Edge intelligent silent voice recognition artificial throat system and method

Abstract

The application discloses an edge intelligent silent voice recognition artificial throat system and method, which comprises a flexible throat strain sensor, an analog front-end circuit, an analog-digital conversion module, an edge computing module and an output module. The flexible throat strain sensor is a system signal acquisition terminal, adopts a PVD silver modified MXene / PVA / MWCNTs / silk fabric microcrack-percolation hybrid network structure or a conventional flexible piezoresistive structure, can be attached to the throat, the lower edge of the lower jaw or the lateral neck of a user, and is electrically connected with the input end of the analog front-end circuit. The application relates to the technical field of wearable intelligent sensing devices, and specifically provides an edge intelligent silent voice recognition artificial throat system and method.

Description

Technical Field

[0001] This invention relates to the field of wearable intelligent sensing device technology, specifically to an edge intelligent silent speech recognition artificial throat system and method. Background Technology

[0002] Silent speech recognition refers to the technology that identifies a user's intentions, words, phrases, or control commands by collecting signals from the throat, mouth, facial muscles, skin vibrations, or other relevant biosignals, without the user uttering any audible sounds or with extremely low vocal intensity. This technology can be applied to scenarios such as assisting communication for the deaf, communication in noisy environments, privacy-preserving interaction, silent communication in military and rescue operations, and smart wearable control and augmented reality input.

[0003] Existing silent speech recognition technologies mainly include camera-based lip reading, ultrasound or electromagnetic sensing-based tongue and oral cavity movement detection, surface electromyography or electroencephalography-based bioelectrical detection, microphone or contact vibration sensing-based weak sound detection, and flexible strain sensor-based laryngeal and facial skin deformation detection. In recent years, the use of flexible strain sensors in silent speech recognition has attracted attention. Existing research has used materials such as graphene, MXene, silver nanowires, and carbon nanotubes to fabricate flexible strain sensors, which are then attached to the larynx to collect skin deformation signals during pronunciation. These sensors are then combined with machine learning or deep learning models to achieve word classification. Recent published work has further introduced dual-modal sEMG / vibration sensing, EMG / EEG fusion, photon sensing, and wearable magnetic sensing into silent speech recognition, indicating that this field is evolving from single-sensor word classification to multimodal, sentence-level, real-time wireless, and scenario-based applications.

[0004] However, existing silent speech recognition solutions using flexible sensors still have the following shortcomings:

[0005] (1) Poor signal stability and weak anti-interference ability. The amplitude of the silent vocalization signal in the throat is small and varies greatly from person to person. It is significantly affected by the position of the sensor, the tightness, the skin condition and sweat. Moreover, the signal fluctuation of non-target actions such as swallowing, coughing, turning the head and breathing is much greater than that of the silent vocalization signal, which can easily cause recognition interference. There is no specific anti-interference mechanism in the existing technology.

[0006] (2) There is no perfect rejection mechanism, resulting in a high false trigger rate. The existing system only classifies and identifies preset target words, and does not distinguish between resting blank state, non-target human body movements, and wearing disturbances as independent categories. When working continuously, it is easy to mistake invalid signals for valid instructions, which cannot meet the actual use needs.

[0007] (3) Lack of identity verification function, resulting in insufficient privacy and security. Existing devices lack user identity verification mechanisms, which can easily trigger erroneous operations when worn by unauthorized users or when the sensors are disturbed by external forces. Furthermore, the physiological signals from the throat contain personal biometric characteristics, and there is a serious risk of privacy leakage when transmitting raw data to the cloud.

[0008] (4) Relying on cloud computing, resulting in poor real-time performance and offline availability. Most existing solutions deploy core recognition and inference in the cloud, which leads to network latency, high power consumption, and inability to work offline, making them unsuitable for all-weather wearable silent interaction scenarios.

[0009] (5) Insufficient adaptability and poor long-term stability. Existing devices lack an online calibration mechanism and cannot adapt to baseline drift and position shift issues after long-term sensor wear. Recognition performance degrades significantly across days and wearing conditions, making it difficult to implement in engineering. Summary of the Invention

[0010] The technical problem to be solved by this invention is that existing silent voice recognition solutions using flexible sensors are inadequate in terms of signal stability, real-time performance, privacy protection, false trigger control, user identity protection, and wearable deployment.

[0011] To solve the above-mentioned technical problems, the present invention provides the following technical solution: The solution provides an edge intelligent silent speech recognition artificial throat system, including a flexible laryngeal strain sensor, an analog front-end circuit, an analog-to-digital conversion module, an edge computing module, an output module, and a power supply and communication module.

[0012] Flexible laryngeal strain sensor: This is the system's signal acquisition terminal. It adopts a PVD silver-modified MXene / PVA / MWCNTs / silk fabric microcrack-percolation hybrid network structure or a conventional flexible piezoresistive structure. It can be attached to the user's throat, lower jaw, or neck. The sensor output is electrically connected to the analog front-end circuit input to convert the skin deformation, muscle traction, and laryngeal micro-vibrations caused by silent vocalization into a continuous resistance change signal.

[0013] Analog front-end circuit: This is the core of signal conditioning, integrating one or more of the following: a reference resistor voltage divider circuit, a constant current source circuit, a Wheatstone bridge circuit, an instrumentation amplifier, a programmable gain amplifier, a filter circuit, and a reference voltage circuit. The input terminal interfaces with the flexible throat strain sensor, and the output terminal connects to the analog-to-digital converter module. This module performs voltage reduction, amplification, filtering, and noise reduction on the sensor's resistance signal, outputting a standardized analog voltage signal. Specifically, the reference resistor or constant current source 210 converts the sensor's resistance change into an acquireable voltage or current change; the filter / amplifier unit 220 suppresses out-of-band noise, amplifies the throat micro-deformation signal, and matches the input range of the analog-to-digital converter module. The reference resistor is 0.1-10 times the initial sensor resistance R0, and the system supply voltage is 1.8-5V.

[0014] Analog-to-digital converter module: a signal conversion unit, with the input end connected to the analog front-end circuit and the output end connected to the edge computing module; configured with a sampling frequency of 100-5000Hz, ADC bit depth of 10-24bit, and analog bandwidth of 0.1-200Hz, used to convert the conditioned analog voltage signal into a digital throat timing signal;

[0015] Edge computing module: The core computing and decision-making unit of the system, employing a low-power MCU, DSP, embedded NPU, Bluetooth SoC, ARM processor, or wearable AI chip. The edge computing module includes a preprocessing unit and a recognition model. The preprocessing unit performs baseline estimation, drift subtraction, sliding window normalization, abnormal peak removal, bandpass filtering, resampling, amplitude clipping, and sensor contact status detection on the throat timing data output by the analog-to-digital conversion module. For throat sensing signals, the system can use a sliding window of 0.2-3 s with an overlap rate of 25%-90%; time-domain, frequency-domain, or depth features are calculated within each window. If the sensor detaches, the initial resistance exceeds the set range, or the baseline drift exceeds the threshold, the system enters a re-attachment prompt or recalibration state instead of directly outputting the recognition result.

[0016] Output module: Result interaction unit, with the input end connected to the edge computing module, including a text display unit, a speech synthesis unit, and an external device control interface. The text display unit is used to display words, short sentences, blank rejection, identity verification, re-wearing, or secondary confirmation results; the speech synthesis unit is used to convert valid recognition results into audible speech; the external device control interface is used to send gate-confirmed control commands to assistive communication devices, smart home devices, rehabilitation training terminals, or other controlled devices.

[0017] Power supply and communication module: Connected to the analog front-end circuit, analog-to-digital conversion module, edge computing module and output module, it is used to provide working power and transmit configuration parameters, identification results or anonymous statistical information via wired or wireless means;

[0018] Optional auxiliary sensing module: includes an inertial sensor, sEMG electrode, temperature and humidity sensor or second flexible throat strain sensor, electrically connected to the edge computing module, used to assist in the detection of head and neck posture, muscle movement, environmental temperature and humidity drift, and multi-position throat deformation, and improve signal anti-interference.

[0019] An edge-based intelligent silent speech recognition method using an artificial throat includes the following steps:

[0020] S201. Device Wearing: Precisely attach the flexible laryngeal strain sensor to the user's throat, lower jaw, or pre-set collection area on the side of the neck.

[0021] S202. Sample collection: Collect target words, target phrases, user resting state, blank inactive state, non-target actions such as swallowing / coughing / turning head, and laryngeal sensor time-series data corresponding to user identity through sensors;

[0022] S203. Signal preprocessing and segmentation: Perform baseline correction, filtering, normalization, and outlier removal preprocessing on the acquired raw time-series data, and complete signal segmentation through a sliding window.

[0023] S204, Edge Model Inference: Input windowed standard data into a locally deployed lightweight multi-task recognition model;

[0024] S205, Multi-dimensional output: The model outputs word / phrase category, blank probability, and user identity probability simultaneously through three output branches;

[0025] S206 Threshold Gating Decision: Based on the word confidence Pword, blank probability Pblank, and identity confidence Pid, ​​a comprehensive judgment is made. If the threshold condition is met, a valid recognition result is output; otherwise, blank rejection, identity protection, or secondary confirmation is performed.

[0026] S207. Result Interaction Output: Push the final recognition result to speech synthesis, text display, intelligent control or auxiliary communication devices to complete human-computer interaction.

[0027] During the model training phase, a multi-label annotation method is adopted to simultaneously label the samples with word categories, blank / non-target action categories, and user identity labels. Multi-task training is completed by combining multiple loss functions such as cross-entropy loss and focus loss. When the device is running, only the recognition results and anonymous data are uploaded, while the original sensor data is stored locally to ensure privacy and security.

[0028] The recognition model of this invention can be a one-dimensional convolutional neural network, a temporal convolutional network (TCN), a lightweight residual network, a recurrent neural network, a Transformer encoder, a Siamese network, a support vector machine, a random forest, or a combination of the above models. Preferably, the model includes a shared temporal feature extraction backbone and at least three output branches: the first output branch is used to recognize words, phrases, or control commands; the second output branch is used to determine non-target actions such as blank states, swallowing, coughing, neck rotation, and wearing perturbations; and the third output branch is used for user identity verification or individual template matching. Through multi-task training, the system can simultaneously incorporate "what was said," "whether recognition should be rejected," and "whether the user is authorized" into the decision-making process, rather than simply post-processing after word classification.

[0029] In a preferred model structure, the input is a one-dimensional resistance change sequence or a multi-channel sensing sequence with N sampling points. First, short-term micro-vibration and slow deformation features are extracted through multi-scale one-dimensional convolution. Then, syllable-level temporal dependence is captured through depthwise separable convolution or TCN. Subsequently, channel attention or temporal attention is used to highlight segments related to laryngeal articulation. Finally, the word probability Pword, blank / interference probability Pblank, and identity probability Pid are output. The model can be deployed at the edge after 8-bit integer quantization, pruning, distillation, or lookup table optimization. The number of model parameters can be controlled in the tens of thousands to millions to meet the real-time inference and low-power operation requirements of wearable devices.

[0030] The training loss function can employ multi-task weighted loss, including word classification cross-entropy, blank / interference binary or multi-class classification loss, identity classification loss, feature center constraint loss, and confidence calibration loss. For auxiliary communication applications with small vocabularies, fixed-category classification can be used; for short sentence or continuous input applications, connection-time classification (CTC), finite-state language models, or small edge-side language models can be further used to merge continuous window results. Regardless of the model used, this invention emphasizes that the output results must pass through blank and identity gating before being allowed to enter the interaction layer.

[0031] When deployed at the edge, different model levels can be selected based on hardware capabilities. The low-power level can employ feature engineering plus a lightweight classifier, with model storage less than several hundred KB; the standard wearable level can use 1D-CNN or TCN, with model storage ranging from several hundred KB to several MB; the enhancement level can use attention networks or small Transformers, and reduce memory and power consumption through quantization. The system can design end-to-end recognition latency within the range of 50-1000 ms, preferably below 300 ms; the energy consumption per inference can be adjusted through window overlap rate, trigger threshold, and sleep strategy to meet all-day or half-day wear requirements.

[0032] The system can also be configured with a tiered power consumption strategy. In resting mode, the edge processing module monitors the sensor baseline only with low sampling rate or low-frequency features; when a throat micro-movement exceeding the trigger threshold is detected, the sampling rate is increased and full model inference is initiated; when a continuous window is determined to be blank or a non-target action, low-power monitoring is resumed. This strategy reduces unnecessary neural network computations during continuous operation, extends battery life, and retains real-time response to valid silent speech.

[0033] The decision logic of this invention includes effective voice triggering, category recognition, rejection, and identity protection. Specifically, when one or more consecutive windows meet the thresholds for signal energy, waveform change rate, and model effective voice probability, the system enters the candidate recognition state. If Pword is higher than the first threshold, Pblank is lower than the second threshold, Pid is higher than the third threshold, and the categories of consecutive windows are consistent, then the corresponding word, phrase, or instruction is output. If Pblank is higher than the threshold, the category confidence is insufficient, identity verification fails, or there is a conflict between window results, then blank space or rejection is output, and external control is not triggered. The first, second, and third thresholds can be set according to the application security level. For example, a lower threshold can be used for ordinary text input, while a higher threshold and repeated confirmation can be used for device control or emergency calls.

[0034] In terms of training methods, the system can first collect basic data for each user, including target words, short sentences, resting state, normal breathing, swallowing, coughing, head turning, head lowering, head nodding, wearing perturbations, and non-target lip movements. Each category can collect 5-200 samples, preferably 20-50. For users with aphonia or weak voices, fewer-sample calibration and template updates can be used to reduce the training burden. During the training phase, time scaling, amplitude perturbation, baseline drift, random pruning, noise superposition, and channel loss enhancement can be applied to improve the model's robustness to changes in wearing tightness, sensor position, and skin condition.

[0035] Dataset construction can also employ a hierarchical labeling system. The first layer of labels represents valid silent speech, blank spaces, and non-target actions; the second layer represents specific words, phrases, or commands; the third layer represents user identity or device binding number; and the fourth layer represents wearing location, attachment pressure, and sensor batch. Through hierarchical labeling, the training phase can simultaneously learn category features and interference features, while the testing phase can separately calculate word accuracy, blank space rejection rate, non-target action false trigger rate, identity verification rate, average latency, and cross-day stability. These metrics are more suitable for evaluating the real-world usability of an artificial throat system than a single accuracy rate.

[0036] The present invention can be configured with the following testing process in an embodiment: After the user wears the flexible sensor, samples are collected sequentially for resting, normal breathing, target words, target phrases, swallowing, coughing, head turning, nodding, walking, and re-attaching; after the system completes training locally or loads a pre-trained model, it runs continuously for recognition. Evaluation results can be represented by a confusion matrix, ROC or PR curve, blank rejection statistics, identity verification curve, real-time latency curve, and power consumption curve. For the patent drawings, the above process can be corresponding to a system block diagram, signal processing flowchart, model structure diagram, decision gating diagram, and test waveform / confusion matrix diagram.

[0037] This invention also includes online calibration and model update mechanisms. When a user first wears the device, the system records the sensor's initial resistance R0, resting baseline, several standard actions, and target word samples to establish a personal template. During daily use, the system updates the baseline in a blank window without outputting commands, and updates the individual feature center after the user confirms the high-confidence recognition result. If a change in sensor position or a deviation in signal distribution from the historical template is detected, the system prompts for recalibration or temporarily increases the rejection threshold. This mechanism can reduce the performance degradation of the flexible sensor caused by sweat, skin condition, and changes in attachment position during long-term wear.

[0038] To further enhance reliability, the system can also be configured with multiple sensors or multi-channel solutions. For example, two flexible strain sensors can be placed above the Adam's apple and on the side of the neck to distinguish between swallowing and silent vocalization; or an IMU can be added to detect large neck movements, sEMG electrodes can be added to enhance muscle action discrimination, and temperature and humidity sensors can be added to compensate for baseline drift. Multi-channel data can be fused at the feature layer or decision layer, but the basic protection of this invention does not rely on multimodal sensing, meaning that a single flexible strain sensor combined with an edge multi-task model can also be implemented.

[0039] Regarding privacy and data security, the system can, by default, save only model parameters, feature templates, and necessary calibration statistics at the edge, without uploading the original laryngeal time-series signals. When remote model updates are required, anonymized features, encrypted gradients, or a small number of user-confirmed samples can be uploaded. Device binding can be accomplished through user identity branches, PIN codes, mobile phone pairing, or biometric templates. For medical assisted communication scenarios, the system can be configured with local logs and retraction mechanisms; for industrial or control scenarios, the system can be configured with command whitelists, secondary confirmation, and security lock status.

[0040] The output interaction module may include a display screen, mobile application, speech synthesizer, Bluetooth module, Wi-Fi module, UART / I2C / SPI interface, smart home control interface, or auxiliary communication terminal. When the recognition result is text, the system can display it on the local screen or mobile device; when the recognition result is a conversational statement, the system can call the local speech synthesis module to play it; when the recognition result is a control command, the system can first perform secondary confirmation or long press / repeat word confirmation to avoid accidental triggering. For medical auxiliary communication scenarios, the system can retain emergency call, common needs, pain expression, and nursing communication vocabulary; for silent control scenarios, the system can retain commands such as device power on / off, volume adjustment, page turning, and confirmation / cancel.

[0041] In the system embodiment, a flexible sensor is attached near the Adam's apple. The analog front end uses a resistor divider and a 12-16 bit ADC with a sampling frequency of 500 Hz. The edge model employs a lightweight one-dimensional convolutional TCN structure with a window length of 1.0 s and a window overlap rate of 50%. After acquiring target word, blank state, and non-target action data, the system outputs the target word recognition result at the edge end. For swallowing, head turning, coughing, or unauthorized user samples, the system outputs rejection or blank. The above parameters can be adjusted according to sensor sensitivity, vocabulary size, user ability, and hardware computing power.

[0042] Compared to solutions that only use cloud-based algorithms, this invention can complete core inference locally, reducing the uploading of raw physiological signals and lowering privacy risks and network dependence. Compared to flexible sensing systems that only perform word classification, this invention adds blank / non-target action rejection and identity verification, which can reduce false triggering in actual use. Compared to solutions that only use sEMG or EEG, this invention can use flexible fabric strain sensors, which are easier to wear and require less skin preparation. Compared to a single artificial throat device, this invention provides a complete closed-loop system from sensing, acquisition, recognition, rejection to output.

[0043] The present invention proposes an edge-intelligent silent speech recognition artificial throat system and method, and the beneficial effects achieved by adopting the above structure are as follows:

[0044] (1) By combining a highly sensitive flexible laryngeal strain sensor with a low-noise analog front-end circuit and a standardized analog-to-digital conversion module, the problem of low signal-to-noise ratio and susceptibility to environmental and wearing status interference in the existing laryngeal micro-signal acquisition is solved, which greatly improves the stability and accuracy of silent vocalization signal acquisition and adapts to the signal acquisition needs of different users and different wearing scenarios.

[0045] (2) A lightweight multi-task recognition model is deployed locally through the edge computing module. Three output branches are set: word classification, blank / non-target action discrimination, and identity discrimination. This solves the defects of existing technologies, such as no anti-interference rejection mechanism and no identity verification. It effectively avoids false triggering caused by actions such as swallowing, coughing, and turning the head, and at the same time prevents unauthorized users from operating illegally, greatly improving the safety and reliability of the device.

[0046] (3) By using a full-process edge local computing architecture to replace the traditional cloud inference mode, the existing technologies can solve the problems of network dependence, high latency, offline unavailability, and leakage of original data privacy, so as to realize the independent operation of the device offline, reduce communication power consumption, and protect the privacy and security of users' biometric data.

[0047] (4) Through the built-in online calibration module, user baseline data can be recorded in real time, personal templates can be updated, and rejection thresholds can be adjusted adaptively to solve the performance degradation problem caused by long-term wear drift and wear position shift of existing sensors, and improve the stability and adaptability of the device for long-term use.

[0048] (5) Through modular hardware design and configurable threshold strategy, combined with multi-source auxiliary sensing expansion interface, the problem of single function and poor scene adaptability of existing technology can be solved. It can be flexibly adapted to various application scenarios such as auxiliary communication for people with hearing loss, silent control of smart home, voice rehabilitation training, wearable identity authentication, etc., and expand the scope of application of the device.

[0049] (6) By using multi-task model training and hierarchical power consumption control strategy, the power consumption of edge devices is reduced while ensuring recognition accuracy, taking into account both recognition real-time performance and device battery life, and meeting the needs of all-weather wearable use. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0051] Figure 1 This is a structural block diagram of an edge-intelligent silent speech recognition artificial throat system proposed in this invention;

[0052] Figure 2 This is a flowchart of an edge-intelligent silent speech recognition artificial throat system proposed in this invention;

[0053] Figure 3 This is a circuit diagram of the analog front-end acquisition circuit of an edge-intelligent silent speech recognition artificial throat system proposed in this invention;

[0054] Figure 4 This is a diagram of the multi-scale attention neural network structure of an edge-intelligent silent speech recognition artificial throat system proposed in this invention, showing the input sequence, multi-scale convolutional branches, feature addition, channel attention, fully connected processing, and the data flow relationship between the three output branches;

[0055] Figure 5 This invention presents an edge-intelligent silent speech recognition artificial throat system and a test diagram of the recognition results.

[0056] Figure 6 This is a test diagram of continuous signal segmentation, blank judgment and confusion matrix of an edge intelligent silent speech recognition artificial throat system proposed in this invention;

[0057] Figure 7This is a feature clustering accuracy distribution diagram of an edge-intelligent silent speech recognition artificial throat system proposed in this invention;

[0058] Figure 8 This is a waveform diagram of the input signal of an edge-intelligent silent speech recognition artificial throat system proposed in this invention.

[0059] in,

[0060] 100-Flexible throat strain sensor;

[0061] 200 - Analog front-end circuit; 210 - Reference resistor or constant current source; 220 - Filter / amplifier unit;

[0062] 300-Analog-to-Digital Conversion Module;

[0063] 400 - Edge computing module; 410 - Preprocessing unit; 420 - Recognition model; 421 - Word or phrase classification output; 422 - Blank / non-target action discrimination output; 423 - Identity discrimination output;

[0064] 500 - Output module; 510 - Text display unit; 520 - Speech synthesis unit; 530 - External device control interface;

[0065] 600-Power supply and communication module. Detailed Implementation

[0066] The technical inventions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0067] It should be noted that the terms “front,” “back,” “left,” “right,” “up,” and “down” used in the following description refer to the directions shown in the attached diagram, while the terms “inside” and “outside” refer to the directions toward or away from the geometric center of a specific component, respectively.

[0068] The power supply voltage is 1.8~5 V, preferably 3.3 V; the reference resistor is 0.1-10R0, preferably 0.5-2R0; the sampling frequency is 50-2000 Hz, preferably 100-500 Hz; the window length is 0.2-5 s, preferably 0.5-2 s; the word threshold is 0.50-0.95, preferably 0.70; the blank threshold is 0.10-0.50, preferably 0.30; and the identity threshold is 0.60-0.99, preferably 0.80. Example 1

[0069] like Figures 1-3 As shown, the technical invention adopted by this invention is as follows: an edge-intelligent silent speech recognition artificial throat system.

[0070] Figure 1 The diagram illustrates a flexible laryngeal strain sensor 100, an analog front-end circuit 200, an analog-to-digital converter (ADC) module 300, an edge computing module 400, an output module 500, a power supply and communication module 600, and the connections between these sub-units. The flexible laryngeal strain sensor 100 is attached to the subject's larynx and converts the micro-deformation of the larynx into a resistance change signal. This resistance change signal is then conditioned by a reference resistor or constant current source 210 and a filtering / amplification unit 220 in the analog front-end circuit 200 before being sent to the ADC module 300. The ADC module 300 converts the analog signal into digital laryngeal timing data and transmits it to the edge computing module 400. The preprocessing unit 410 in the edge computing module 400 performs baseline correction, filtering, normalization, and window segmentation. The recognition model 420, based on the processed window data, generates in parallel word or phrase classification output 421, blank / non-target action discrimination output 422, and identity discrimination output 423. The three outputs jointly participate in the gating decision, and the decision result is transmitted to the output module 500. The corresponding interaction is performed by the text display unit 510, the speech synthesis unit 520, or the external device control interface 530. The power supply and communication module 600 supplies power to the analog front-end circuit 200, the analog-to-digital conversion module 300, the edge computing module 400, and the output module 500, and undertakes the communication of configuration parameters and non-raw physiological data. Figure 1 Solid arrows in the diagram represent the data stream of throat signals or recognition results, while dashed lines indicate power or communication connections.

[0071] like Figure 2 As shown, the method of the present invention includes attaching sensors, collecting silent sound / blank / identity samples, preprocessing, window segmentation, edge reasoning, threshold determination, and output control. Figure 3 This shows the reference resistor voltage divider acquisition circuit. Figure 4 The structure of a multi-scale attention neural network is shown.

[0072] Figure 4 In this context, x(t) represents the throat temporal input sequence sampled at time t and processed by the preprocessing unit 410; K=3, K=5, and K=7 represent three one-dimensional convolutional branches with kernel lengths of 3, 5, and 7, respectively, used to extract local features at different time scales; ADD represents the element-wise addition node, used to add and fuse the same-dimensional features output by the three convolutional branches; SE represents the compression-excitation channel attention unit, used to adaptively calibrate the channel weights of the fused features; and FC represents the fully connected unit, used to map the attention-weighted features to the output space. Figure 4 The solid arrows in the text all represent unidirectional transmission relationships of feature data.

[0073] System Setup and Data Acquisition. A flexible laryngeal strain sensor is attached near the user's Adam's apple, lower jaw, or side of the neck. Resistance change signals are acquired using a reference resistor voltage divider, constant current source, or bridge circuit, and transmitted to the edge computing module via an analog-to-digital converter. The acquired data includes target words, common phrases, resting state, natural breathing, swallowing, coughing, head turning, head lowering, head nodding, wearing disturbances, and identity verification samples. Target words can be commonly used words for auxiliary communication, silent control commands, or rehabilitation training action commands. Blank and non-target action samples should be acquired simultaneously with target words to avoid misclassifying invalid fluctuations as valid words during continuous use. User identity samples can come from multiple wear data from the same user or from comparison samples from different users. Example 2

[0074] The technical invention adopted in this invention is as follows: an edge-intelligent silent speech recognition artificial throat system, further training the edge model and gating decisions. The acquired laryngeal temporal signals undergo baseline correction, filtering, normalization, abnormal peak removal, and window segmentation. The windowed data is then input into a multi-scale one-dimensional convolutional network, a temporal convolutional network, or an attention network. The model is configured with word classification, blank / non-target action discrimination, and identity discrimination output branches, and is trained using a multi-task loss. Figure 4 As shown, the input sequence x(t) enters one-dimensional convolution branches with K=3, K=5, and K=7 respectively along three independent arrows; the outputs of the three branches converge into the ADD node along the arrows. The ADD node only performs element-wise addition of features of the same dimension, and its output enters the SE channel attention unit and the FC fully connected unit sequentially along a single arrow; the output of the FC fully connected unit then forms word or phrase classification output 421, blank / non-target action discrimination output 422, and identity discrimination output 423 respectively along three independent arrows. The loop line below the ADD node in the original figure that was not connected to the previous module has been deleted, and the starting point, ending point, and direction of the revised arrows all correspond to a clear data flow. Figure 6 This is used to illustrate window segmentation and blank determination of continuous signals, as well as the evaluation results of the confusion matrix after target word classification.

[0075] The system performs gating based on word confidence, blank probability, and identity confidence. If the window shows swallowing, head turning, coughing, or wearing disturbances, the system outputs blank or rejects the word; if the word is identified as a target word but the identity confidence is insufficient, the system prohibits the execution of control commands and enters a secondary confirmation process. This gating process enables the system to do more than just classify words; it also provides the ability to control false triggers in real, continuous use. Example 3

[0076] The technical invention adopted in this invention is as follows: an edge intelligent silent speech recognition artificial throat system, which deploys the trained model in the edge computing module to run continuously, and records the recognition accuracy, blank rejection rate, non-target action false trigger rate, identity verification pass rate, end-to-end latency and power consumption. Figures 5 to 8 This system is used to display the overall system test diagram, continuous signal segmentation, confusion matrix, feature clustering accuracy distribution, and input signal waveform. During evaluation, target word samples, blank samples, non-target action samples, and samples from different users can be statistically analyzed to verify the system's usability under continuous use conditions.

[0077] The results can also be compared with those of the blank discrimination branch, the identity discrimination branch, the unquantized model, and the quantized edge model. If adding blank / identity gating reduces false triggering of non-target actions, and the quantized model maintains acceptable recognition performance and low latency on the edge, it indicates that the edge multi-task artificial throat scheme of the present invention has practical application advantages over simple sensor classification or cloud offline classification.

[0078] in, Figure 5 (a) The system is divided into three parts: signal acquisition, deep learning and control, and execution feedback. The throat strain sensor is powered by 3.3 V and converted into an analog voltage by a resistor divider or acquisition circuit. The voltage is then sent to a Raspberry Pi 4B or other edge computing module through an ADC module such as ADS1115. The edge end completes preprocessing segmentation, feature extraction, word classification and multi-task decision-making, and outputs the results to control modules such as text / voice feedback or access control. Figure 5 (b) shows the MS-SE-ResNet network structure. The input node receives original user, new user and newly added word samples. After multi-branch feature extraction and residual SE module, it outputs word classification, user identity and permission related results. Figure 5 (c) shows that after the sensor is attached to the larynx, the micro-deformation signal of the larynx is conditioned and model inference is converted into a silent speech recognition result. Figure 5 (d) shows that a continuous speech signal can be segmented into a target word window and a blank window. Figure 5 The confusion matrix and feature clustering results in (e) are used to illustrate the model evaluation method. The purpose of this overall diagram is to unify hardware acquisition, model structure, application scenarios, and testing and evaluation into a closed-loop process, supporting the connection relationships and synergistic effects between the modules in the system claims.

[0079] Figure 6The horizontal time sequence curve in (a) represents the continuously collected laryngeal strain signal, the dashed lines represent window boundaries, and the arrows represent identified or to-be-determined speech segments; segments such as 'I' and 'feel' in the figure correspond to target speech windows, segments such as 'bad' can serve as adjacent short words, interfering words or non-current target windows, and the '[blank]' segment represents a resting state with no effective pronunciation or a blank interval that should not trigger output. The system jointly determines whether to output a target word through a sliding window, energy / amplitude threshold, word confidence and blank probability. Figure 6 The confusion matrix in (b) takes the true labels as the vertical axis and the predicted labels as the horizontal axis, the dark diagonal units represent the correct recognition proportion, and the off-diagonal units represent the misrecognition proportion between different words. The function of this figure is to prove that the present invention not only recognizes target words, but also can reduce false triggers during continuous use through blank discrimination and threshold gating.

[0080] Figure 7 The clustering distribution of different target word samples is presented in two-dimensional feature coordinates, different colors or markers correspond to different word categories, solid points represent correctly recognized samples, and hollow points represent misrecognized samples; the overall accuracy given in the figure is used to represent the comprehensive recognition effect of the edge model under the corresponding dataset or test conditions. If samples of the same word form a relatively concentrated cluster in the feature space and there is a separable interval between different words, it indicates that multi-scale time sequence feature extraction and the attention / SE module can enhance the distinguishability between different silent speech actions; the misrecognized points are used to analyze the influence of easily confused words, wearing differences or individual differences on recognition performance. This figure is used to support the technical effects of the present invention regarding edge intelligent recognition accuracy, robustness and model interpretable evaluation.

[0081] Figure 8 The normalized input waveforms of target words such as open, close, just, excuse, learn and me under the same window length are presented, and differences exist in the peak and valley positions, duration, waveform trend and local spikes of different words. This figure is used to illustrate that the input recognized by the present invention is not a single amplitude threshold, but a laryngeal micro-deformation signal containing time sequence morphology, duration and dynamic change features, so as to Figure 4 provide the multi-scale time sequence model shown in Figure 6 , the window segmentation shown in Figure 7 and the feature clustering evaluation shown in with original signal basis. Embodiment 4

[0082] The technical invention of this invention is as follows: an edge-intelligent silent speech recognition artificial throat system, implemented using edge devices and storage media; the edge computing module can be implemented by a microcontroller, embedded AI chip, Bluetooth SoC, mobile terminal, or wearable computing module; model parameters, thresholds, user templates, and program instructions can be stored in on-chip Flash, external memory, or mobile terminal memory. When the processor executes the program instructions, it completes baseline correction, window segmentation, three-output branch inference, threshold gating, and output control, thereby forming an edge computing device or storage media implementation that can run locally. Example 5

[0083] The technical invention of this invention is as follows: an edge-intelligent silent speech recognition artificial throat system, with edge deployment and privacy evaluation; the trained recognition model is quantized to 8 bits, pruned or distilled, and then deployed on an edge computing module; the word recognition accuracy, blank rejection rate, identity verification pass rate, single inference time, and continuous operation power consumption are compared before and after quantization. The system by default only outputs recognition results, confidence scores, or anonymized statistical information, and does not continuously upload raw laryngeal time-series data. This embodiment is used to illustrate the technical effects of this invention in local real-time recognition and privacy protection. Example 6

[0084] The technical invention of this invention is as follows: an edge-intelligent silent speech recognition artificial throat system, with robustness testing across wearing and non-target actions; sensors are attached above the larynx, to the side of the larynx, and to the lower edge of the mandible to collect samples of target words, swallowing, coughing, turning the head, lowering the head, natural breathing, and wearing perturbation; the system outputs rejection results through blank / non-target action branches, and restricts unauthorized users from triggering control commands through identity discrimination branches, thereby reducing the risk of false triggering in continuous use scenarios.

[0085] Comparison and testing methods

[0086] Comparative Example 1: A single-task model containing only a word classification output head is used, without setting blank / non-target action discrimination output heads and identity discrimination output heads. This comparative example illustrates that without rejection and identity gating, swallowing, neck turning, wearing perturbations, or unauthorized user samples are more likely to be misidentified as target words.

[0087] Comparative Example 2: An offline classification process using a cloud-based or host computer is employed, with the edge device only responsible for collecting and uploading the raw throat time-series data. This comparative example illustrates the advantages of the present invention's local inference scheme in terms of privacy protection, offline availability, and latency control.

[0088] Comparative Example 3: Fixed threshold energy triggering without baseline adaptive updates. This example illustrates that sensor attachment location, skin condition, and cross-day drift affect continuous recognition stability, while online calibration and threshold adjustment improve practical performance.

[0089] Test Method 1: During data collection, each user will collect target words, target phrases, resting, natural breathing, swallowing, coughing, turning head, lowering head, nodding, wearing disturbance, and identity verification samples; each type of sample can be collected 5-200 times, and the sensor position, initial resistance, and sampling conditions will be recorded.

[0090] Test Method 2: Model evaluation metrics include word recognition accuracy, blank rejection rate, non-target action false trigger rate, identity verification pass rate, end-to-end latency, model storage requirements, single inference time, and continuous operation power consumption. Test results can be represented by a confusion matrix, accuracy histogram, ROC / PR curve, and typical waveform graph.

[0091] Test Method 3: During edge deployment testing, the trained model is quantized, pruned, or distilled and then deployed on a microcontroller, embedded AI chip, mobile terminal, or wearable computing module. The recognition performance and inference latency before and after quantization are compared to prove that the system has the ability to run locally in real time.

[0092] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, material, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, material, or apparatus.

[0093] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An edge-intelligent silent speech recognition artificial throat system, characterized in that, The system includes a flexible laryngeal strain sensor (100), an analog front-end circuit (200), an analog-to-digital conversion module (300), an edge computing module (400), and an output module (500). The flexible laryngeal strain sensor (100) is used to attach to the user's larynx, lower jaw, or neck and convert the micro-vibrations of the larynx, muscle traction, or skin deformation during silent speech into resistance change signals. The edge computing module (400) is used to perform preprocessing, window segmentation, and recognition model (420) inference locally. The output module (500) is used to output word category, sentence category, blank / non-speech discrimination result, rejection result, and / or user identity discrimination result. The flexible throat strain sensor (100) is a PVD silver-modified MXene / PVA / MWCNTs / silk fabric microcrack-percolation hybrid network flexible strain sensor; or a flexible piezoresistive throat strain sensor including a flexible substrate and a conductive sensitive layer, capable of outputting resistance change signals. The analog front-end circuit (200) includes one or more of the following: a reference resistor voltage divider circuit, a constant current source circuit, a Wheatstone bridge circuit, an instrumentation amplifier, a programmable gain amplifier, a filter circuit, and a reference voltage circuit; the reference resistor is 0.1-10 times the initial resistance R0 of the flexible throat strain sensor (100), and the supply voltage is 1.8-5 V; The analog-to-digital converter module (300) has a sampling frequency of 100-5000 Hz, an ADC bit depth of 10-24 bits, and an analog bandwidth of 0.1-200 Hz. The edge computing module (400) is a microcontroller, DSP, embedded NPU, Bluetooth SoC, single-board computer, mobile terminal or wearable computing module. The recognition model (420) is deployed on the edge computing module (400) after 8-bit integer quantization, pruning, distillation or parameter sharing. The preprocessing of the edge computing module (400) includes one or more of the following: baseline drift correction, normalization, filtering, resampling, abnormal peak removal, amplitude clipping, sensor contact state detection, and sliding window segmentation; the sliding window length is 0.2-5s, and the window overlap rate is 0-90%. The recognition model (420) of the edge computing module (400) includes a shared temporal feature extraction backbone and three output branches, namely a word or phrase classification output (421) branch, a blank / non-target action discrimination output (422) branch and a user identity discrimination output (423) branch.

2. The edge-intelligent silent speech recognition artificial throat system according to claim 1, characterized in that, The shared temporal feature extraction backbone includes one or more of the following: multi-scale one-dimensional convolutional module, depthwise separable convolutional module, residual connection module, temporal convolutional network module, recurrent network module, attention module, or Transformer coding module.

3. The edge-intelligent silent speech recognition artificial throat system according to claim 1, characterized in that, The edge computing module (400) makes gating decisions based on word confidence Pword, blank probability Pblank, and identity confidence Pid: when Pword is higher than the first threshold, Pblank is lower than the second threshold, and Pid is higher than the third threshold, it outputs a valid recognition result; otherwise, it outputs blank, rejection, re-wearing prompt, or secondary confirmation prompt.

4. The edge-intelligent silent speech recognition artificial throat system according to claim 1, characterized in that, It also includes an online calibration module, which is used to record the user's resting baseline, sensor initial resistance, target word samples, blank samples, and identity samples, and update the user template or adjust the rejection threshold based on high-confidence samples.

5. The edge-intelligent silent speech recognition artificial throat system according to claim 1, characterized in that, It also includes one or more of an inertial sensor, an sEMG electrode, a temperature and humidity sensor, or a second flexible laryngeal strain sensor (100) for detecting head and neck posture, muscle movement, environmental drift, or multi-position laryngeal deformation.

6. A method for using the edge-intelligent silent speech recognition artificial throat system according to any one of claims 1 to 5, characterized in that, Specifically, the following steps are included: S201, attach the flexible laryngeal strain sensor (100) to the user's throat, lower jaw, or neck; S202, collect sensor time-series data corresponding to target words, target phrases, resting states, blank states, non-target actions, and user identity samples; S203, preprocesses and segments the sensing timing data into windows; S204, input windowed data into the recognition model (420) deployed on the edge computing module (400). S205, obtain the word or phrase category, blank probability, and / or user identity probability; S206, based on the threshold gating, outputs valid recognition results, rejects blank samples, or triggers identity protection; S207 sends the output results to speech synthesis, text display, smart terminal control, or auxiliary communication devices.

7. The edge-intelligent silent speech recognition artificial throat method according to claim 6, characterized in that, When training the recognition model (420), the training samples are simultaneously labeled with word category, blank / non-target action category and user identity label, and one or more of cross-entropy loss, focus loss, triplet loss, center loss or confidence calibration loss are used for multi-task training. The non-target actions include one or more of the following: swallowing, coughing, natural breathing, turning the head, lowering the head, nodding, wearing disturbances, walking disturbances, or non-target lip movements; The edge computing module (400) only uploads the recognition results, confidence scores, model update parameters, or anonymous statistical information, and does not continuously upload the original laryngeal sensing time series data.

8. A device for assisting communication, smart home control, rehabilitation training, silent interaction, or wearable identity authentication, characterized in that, Including the edge-intelligent silent speech recognition artificial throat system as described in any one of claims 1 to 5.

9. An edge computing device, characterized in that, It includes a processor, a memory, and program instructions stored in the memory, which, when executed by the processor, implement the edge intelligent silent speech recognition artificial throat method according to any one of claims 6 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions, which, when executed by the edge computing module (400), cause the edge computing module (400) to perform the method described in any one of claims 6 to 7.

Images