A tongue-machine interaction system and method based on ultrasonic raw radio frequency signals

Abstract

The application discloses a tongue-machine interaction system and method based on an ultrasonic original radio frequency signal. The system comprises an ultrasonic sensing unit, which is attached to the submandibular / mandibular area, emits ultrasonic waves to the tongue tissue, and receives multi-channel original RF echo signals scattered and returned by the tongue muscle; a radio frequency signal processing unit, which amplifies, filters and digitizes the echo signals to form a multi-channel RF signal sequence; a feature construction unit, which extracts spatial and temporal features representing the deformation and contraction of the tongue muscle to form a spatial and temporal feature map; an intention decoding and neural feedback unit, which decodes the spatial and temporal feature map into an interaction intention and outputs a neural feedback signal, and maps the interaction intention into an interaction instruction; and a self-adaptive calibration unit, which updates parameters online to compensate for signal changes. The application uses a non-invasive signal acquisition method, directly constructs a spatial and temporal feature map using multi-channel original RF signals, decodes the intention and maps the interaction instruction, reduces the computational overhead of the image reconstruction link, and realizes low-delay and high-stability interaction.

Description

Technical Field

[0001] This invention belongs to the fields of non-invasive brain-computer interface, disability assistance interaction and rehabilitation technology, and specifically relates to a tongue-computer interaction system and method based on raw ultrasound radio frequency signals, which is used to assist patients with high paraplegia, ALS, ALS and other conditions in multi-terminal assisted control. Background Technology

[0002] For patients with severe limb motor disorders, traditional eye trackers or head trackers suffer from fatigue and poor environmental adaptability. Traditional pressure-based tongue control systems cannot capture complex suspended tongue movements. While patients with high-level spinal cord injuries, ALS, and ALS lose limb function during the course of their disease, the tongue muscles directly innervated by cranial nerves often retain function for a longer period. Although current medical ultrasound technology can observe muscles, its image reconstruction-based processing latency is high, failing to meet the needs of real-time multi-terminal interaction. Summary of the Invention

[0003] To address the shortcomings of the prior art, this invention provides a tongue-to-computer interaction system and method based on raw ultrasound radio frequency signals. This system enables low-latency intent decoding and neural feedback interactive control without image reconstruction processing. Furthermore, it improves long-term usability by resisting wear drift and individual differences through online adaptive calibration.

[0004] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows: A tongue-to-computer interaction system based on raw ultrasound radio frequency signals, comprising: The ultrasound sensing unit is designed to fit against the user's submental / mandibular region, emitting ultrasound waves into the tongue tissue and receiving multi-channel raw RF echo signals scattered back by the internal fiber bundles of the tongue muscles. A radio frequency signal processing unit, connected to the ultrasonic sensing unit, is used to amplify, filter, and perform analog-to-digital conversion on the multi-channel raw RF echo signal to form a multi-channel RF signal sequence. The feature construction unit, connected to the radio frequency signal processing unit, is used to extract spatiotemporal features characterizing tongue muscle deformation and contraction based on the multi-channel RF signal sequence, and form a spatiotemporal feature map; The intent decoding and neural feedback unit, connected to the feature construction unit, is used to decode the spatiotemporal feature map into a standard interactive intent and output a neural feedback signal; and to map the standard interactive intent into a standard human-computer interaction command and send it to the terminal device through the wireless communication unit. An adaptive calibration unit, connected to the radio frequency signal processing unit, feature construction unit, and / or intent decoding and neural feedback unit, is used to perform online calibration to update the parameters of signal processing, feature extraction, or intent decoding, and to compensate for signal and feature changes caused by wearing status or user physiological changes. Furthermore, the ultrasonic sensing unit includes a wearable carrier and an ultrasonic transducer array. The wearable carrier at least covers the user's submental / mandibular region. The ultrasonic transducer array has at least two channels and is encapsulated within the wearable carrier corresponding to the submental / mandibular region.

[0005] Furthermore, the wearable device is a subchin / mandibular support made of medical-grade liquid silicone material.

[0006] Furthermore, the ultrasonic transducer array is arranged in an arc shape, with the main beam focused on the core area where the intralingual and extralingual muscles intersect. The ultrasonic transducer array consists of multiple miniature high-frequency ultrasonic transducers, with the operating frequency set to 0.1-30MHz.

[0007] Furthermore, the method for processing raw radio frequency data by the feature construction unit includes: extracting the deformation features of intralingual and lateral lingual muscle fiber bundles at different depths based on multi-channel RF signal sequences; acquiring signals from different anatomical orientations of the tongue in parallel through multiple channels; identifying the coordinated movement patterns of the tongue muscles by analyzing the spatiotemporal correlation of the RF signal envelopes between channels; and forming a spatiotemporal feature atlas; the coordinated movement patterns include at least one of overall tongue elevation, local curling, or lateral displacement; the spatiotemporal feature atlas is a three-dimensional atlas, the dimensions of which include channels, depth, and time.

[0008] Furthermore, the intent decoding and neural feedback unit is configured to decode a specific coordinated contraction pattern of the tongue muscle fiber bundle into an interactive intent, and map the interactive intent into a standard human-computer interaction command; the neural feedback signal is used to guide the user to form a stable intent output pattern.

[0009] Furthermore, the online calibration steps performed by the adaptive calibration unit include: S1. Static reference acquisition: Acquire signals while the user is at rest and set the zero-point threshold; S2. Dynamic range calibration: Guide users to generate preset intention actions, record the feature distribution and dynamic extreme values ​​corresponding to each intention, and establish personalized intention-feature mapping relationship or decision boundary. S3. Normalization: Map the real-time acquired signal to the standard interval [0, 1]. S4. Dynamic Gain Compensation: Dynamically adjusts the signal amplification gain based on the maximum signal strength change detected when the user performs the calibration action. S5. Real-time closed-loop fine-tuning: Continuously updates mapping parameters and corrects feature drift during the interaction process.

[0010] A tongue-to-computer interaction method based on raw ultrasound radio frequency signals, applying the aforementioned tongue-to-computer interaction system based on raw ultrasound radio frequency signals, includes the following steps: The ultrasonic sensing unit, which is attached to the user's subchinal / mandibular region, emits ultrasound to the tongue tissue and simultaneously acquires multi-channel raw RF echo signals scattered back from the tongue muscles. The original multi-channel RF echo signals are processed to form a multi-channel RF signal sequence; Based on the multi-channel RF signal sequence, spatiotemporal features characterizing tongue muscle deformation and contraction are extracted, and a spatiotemporal feature map is constructed. The spatiotemporal feature map is decoded into a standard interactive intent and a neural feedback signal is generated. Based on the standard interactive intent, interactive instructions corresponding to various terminals are generated. Perform online adaptive calibration to update parameters for signal processing, feature extraction, or intent decoding, compensating for changes caused by wearing conditions or user physiological variations.

[0011] Furthermore, the step of constructing the spatiotemporal feature map includes: The deformation amplitude characteristics of tongue muscle fiber bundles at different depths were extracted from the multi-channel RF signal sequence; By analyzing the spatiotemporal correlation of the envelope of multi-channel RF signals, the coordinated movement patterns of the tongue are identified, and a spatiotemporal feature map is formed; the coordinated movement patterns include at least one of overall tongue elevation, local curling and lateral displacement.

[0012] Furthermore, the step of performing online adaptive calibration includes sequentially performing the following sub-steps: S1. Static reference acquisition: Acquire signals while the user is at rest and set the zero-point threshold; S2. Dynamic range calibration: Guide users to generate preset intention actions, record the feature distribution and dynamic extreme values ​​corresponding to each intention, and establish personalized intention-feature mapping relationship or decision boundary. S3. Normalization: Map the real-time acquired signal to the standard interval [0, 1]. S4. Dynamic Gain Compensation: Dynamically adjusts the signal amplification gain based on the maximum signal strength change detected when the user performs the calibration action. S5. Real-time closed-loop fine-tuning: Continuously updates mapping parameters and corrects feature drift during the interaction process.

[0013] The beneficial effects of this invention are: The tongue-computer interaction system and method of the present invention utilize a multi-channel ultrasonic transducer array integrated into a flexible carrier in the submental / mandibular region to non-invasively acquire multi-channel raw RF echo signals generated by the complex fiber bundle structure inside the tongue muscle during contraction. By analyzing the amplitude changes of the multi-channel RF signals in the spatiotemporal dimensions, the user's interaction intent can be parsed with extremely low latency without the need for image reconstruction processing or beamforming, and accurately mapped into various terminal interaction commands. Through real-time feedback of the intent decoding results, the user can form a stable and repeatable "tongue muscle contraction-interaction intent" biofeedback control paradigm in closed-loop training.

[0014] This invention adopts a non-invasive wearing method in the submental / mandibular region, which does not require intraoral contact, making it comfortable and hygienic; it directly utilizes multi-channel raw RF signals and depth gating to construct feature maps, reducing the computational overhead of the image reconstruction link and achieving low-latency interaction.

[0015] The tongue-computer interface system and method of the present invention support multi-dimensional, multi-intent, and multi-terminal commands, and are suitable for barrier-free input and output at the operating system level. With the built-in five-step adaptive calibration algorithm, the system can dynamically compensate for signal decay caused by disease progression or feature drift caused by sweat or wearing displacement, and maintain sensitivity and stability. It provides a highly effective, long-term stable and comfortable brain-computer interface control solution for patients with severe mental and physical disabilities in complex life scenarios.

[0016] This invention innovatively combines radio frequency signal capture technology for tongue muscle fiber bundle contraction amplitude, intention decoding-feedback closed-loop mechanism, and adaptive calibration algorithm for progressive diseases, significantly improving the intention-driven human-computer interaction ability of patients with high spinal cord injury and ALS. It effectively avoids the invasiveness and high latency limitations of traditional devices, providing a high-tech, high-stability assistive interaction solution for people with disabilities, with significant social value and market prospects. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the system composition of the present invention; Figure 2 This is a schematic diagram of the system architecture and the wearing method in the subchinal / mandibular region of the present invention; Figure 3 This is a schematic diagram of the spatiotemporal characteristics of the original radio frequency (RF) signals of the multi-channel tongue muscle fiber bundles; Figure 4 This is a schematic diagram showing the mapping between changes in radio frequency signals and the characteristics of tongue muscle movement; Figure 5 These are RF signals from different channels of tongue muscle contraction; Figure 6 This is a schematic diagram showing how changes in RF signals after tongue muscle contraction control a multimodal terminal. Figure 7 This is a flowchart illustrating the five-step adaptive automatic calibration algorithm; Figure 8 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0020] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. This embodiment is specifically tailored to the refined interaction needs of patients with high-level spinal cord injury, amyotrophic lateral sclerosis (ALS), and Lou Gehrig's disease.

[0021] A tongue-to-computer interaction system based on raw ultrasound radio frequency signals, such as Figure 1 As shown, it includes: The ultrasound sensing unit is designed to fit against the user's submental / mandibular region, emitting ultrasound waves into the tongue tissue and receiving multi-channel raw radio frequency (RF) echo signals scattered back by the internal fiber bundles of the tongue muscles. A radio frequency signal processing unit, connected to the ultrasonic sensing unit, is used to amplify, filter, and perform analog-to-digital conversion on the multi-channel raw RF echo signal to form a multi-channel RF signal sequence. The feature construction unit, connected to the radio frequency signal processing unit, is used to extract spatiotemporal features characterizing tongue muscle deformation and contraction without performing image reconstruction processing or beamforming on the multi-channel raw RF echo signal, and to form a spatiotemporal feature map. An intent decoding and neural feedback unit, connected to the feature construction unit, is used to decode the spatiotemporal feature map into a standard interactive intent and output a neural feedback signal; map the standard interactive intent into a standard human-computer interaction command and send it to the terminal device through a wireless communication unit; preferably, the wireless communication unit includes a Bluetooth communication module; An adaptive calibration unit, connected to the radio frequency signal processing unit, feature construction unit, and / or intent decoding and neural feedback unit, is used to perform online calibration to update the parameters of signal processing, feature extraction, or intent decoding, and to compensate for signal and feature changes caused by wearing status or user physiological changes.

[0022] Specifically, the ultrasonic sensing unit includes a wearable carrier and an ultrasonic transducer array. The wearable carrier at least covers the user's subchinal / mandibular region. The ultrasonic transducer array has at least two channels and is encapsulated within the wearable carrier corresponding to the subchinal / mandibular region. The wearable carrier is a mandibular support made of medical-grade liquid silicone, possessing good skin-friendliness and flexibility, and adjustable length to ensure that the ultrasonic transducer array fits snugly against the user's subchinal / mandibular region. The ultrasonic transducer array consists of multiple miniature high-frequency ultrasonic transducers, with an operating frequency set between 0.1-30MHz. The ultrasonic transducer array is arranged in an arc shape, with the main beam focused on the core area where the intralingual and extralingual muscles intersect. The system architecture and subchinal / mandibular wearing method of this invention are as follows. Figure 2 As shown.

[0023] The feature construction unit directly processes raw radio frequency data that has not undergone image reconstruction and / or beamforming, including: extracting deformation features of intralingual and lateral lingual muscle fiber bundles at different depths based on multi-channel RF signal sequences, preferably the contraction amplitude of the core lingual muscle fiber bundles; and acquiring signals from different anatomical orientations of the tongue through multiple channels in parallel, identifying the coordinated movement patterns of the tongue muscles by analyzing the spatiotemporal correlation of the RF signal envelopes between channels, and forming a spatiotemporal feature atlas; the coordinated movement patterns include at least one of overall tongue elevation, local curling, or lateral displacement; the spatiotemporal feature atlas is a three-dimensional atlas, the dimensions of which include channels, depth, and time. Figure 3 A schematic diagram of the spatiotemporal characteristics of the original radio frequency (RF) signal of the multi-channel tongue muscle fiber bundle is shown, where the horizontal axis represents the time dimension and the vertical axis represents the different depth channels obtained by time distance resolution. The signal amplitude or energy changes in the figure are used to characterize the contraction state and spatiotemporal coordination characteristics of the intralingual and extralingual muscle fiber bundles at different depths.

[0024] The intent decoding and neural feedback unit is configured to decode specific co-contraction patterns of tongue muscle fiber bundles into human interactive intents, and map these interactive intents into standard human-computer interaction commands; the neural feedback signal is used to guide the user to form a stable intent output pattern. The mapping between radio frequency signal changes and tongue muscle movement characteristics is as follows: Figure 4 As shown, different contraction patterns of the tongue muscles correspond to different radio frequency signals. Figure 5 This is a schematic diagram of RF signals from different channels during tongue muscle contraction. Figure 6 This is a schematic diagram illustrating how changes in RF signals after tongue muscle contraction control a multimodal terminal.

[0025] The interaction intent includes one or more of the following: directional movement intent, click / confirm intent, and page scrolling intent. The decoding results and rewards or error correction prompts are output to the user through visual, auditory, or tactile means. The interaction intent is further mapped to terminal interaction commands such as cursor movement, click, scrolling, selection, or quick operation.

[0026] The intent decoding and neural feedback unit includes at least one of the following intent parsing rules: a) Obtain different multi-channel radiofrequency signals from the offset amount, offset angle and / or multi-channel spatiotemporal correlation characteristics of the lateral coordinated contraction of the tongue muscles, and output the corresponding patient interaction intent vector. b) Based on the longitudinal contraction amplitude of the tongue, the deformation characteristics under depth gating, and / or the change in envelope energy, the intention is determined and the corresponding directional intention vector is output. c) Determine the "rolling intention" and its rolling direction and intensity based on the overall curling amplitude, coordinated contraction pattern and / or periodic characteristics of the tongue; d) The click command is triggered by capturing the high-frequency instantaneous radio frequency pulse generated when the tip of the tongue taps lightly.

[0027] The online calibration steps performed by the adaptive calibration unit are as follows: Figure 7 As shown, it includes: S1. Static baseline acquisition: Acquire signals while the user is at rest, automatically set the zero-point threshold for this wear, and eliminate individual differences in muscle tension.

[0028] S2. Dynamic Range Calibration: Guides users to generate preset intention actions, records the characteristic distribution and dynamic extreme values ​​of signals in each channel, and establishes personalized intention-feature mapping relationships or decision boundaries.

[0029] S3. Normalization: The real-time acquired signals are mapped to the standard interval [0, 1] to shield the influence of different absolute values ​​of muscle strength at different stages of the disease.

[0030] S4. Dynamic Gain Compensation: The signal amplification gain is dynamically adjusted based on the maximum signal strength change monitored when the user performs the calibration action. If the maximum signal value monitored when the user performs the calibration action continues to decrease for several days, the signal amplification gain is automatically increased proportionally to ensure that the patient can still maintain control accuracy with less physical effort.

[0031] S5. Real-time Closed-Loop Fine-Tuning: Continuously updates mapping parameters and corrects feature drift during interaction. It continuously learns the user's intent output habits during operation, automatically correcting feature drift caused by slight displacement of sweat or ultrasonic sensing units, ensuring that the intent decoding-feedback closed-loop latency is consistently below 50ms.

[0032] An interactive method based on capturing the contraction characteristics of tongue muscle fiber bundles using raw ultrasound radio frequency signals, such as... Figure 8 As shown, it includes: The ultrasonic sensing unit, which is attached to the user's subchinal / mandibular region, emits ultrasound to the tongue tissue and simultaneously acquires multi-channel raw radio frequency (RF) echo signals scattered back from the tongue muscles. The original multi-channel RF echo signals are processed to form a multi-channel RF signal sequence; Based on the multi-channel RF signal sequence, spatiotemporal features characterizing tongue muscle deformation and contraction are extracted to construct a spatiotemporal feature atlas. The steps of constructing the spatiotemporal feature atlas include: extracting the deformation amplitude features of tongue muscle fiber bundles at different depths from the multi-channel RF signal sequence; identifying the coordinated movement patterns of the tongue body by analyzing the spatiotemporal correlation of the multi-channel RF signal envelope, and forming a spatiotemporal feature atlas; the coordinated movement patterns include at least one of overall tongue body elevation, local curling, and lateral displacement.

[0033] The spatiotemporal feature map is decoded into a standard interactive intent and a neural feedback signal is generated. Based on the standard interactive intent, interactive instructions corresponding to various terminals are generated. Perform online adaptive calibration to update parameters for signal processing, feature extraction, or intent decoding, compensating for changes caused by wearing conditions or user physiological variations. This includes sequentially executing the following sub-steps: S1. Static reference acquisition: Acquire signals while the user is at rest and set the zero-point threshold; S2. Dynamic range calibration: Guide users to generate preset intention actions, record the feature distribution and dynamic extreme values ​​corresponding to each intention, and establish personalized intention-feature mapping relationship or decision boundary. S3. Normalization: Map the real-time acquired signal to the standard interval [0, 1]. S4. Dynamic Gain Compensation: Dynamically adjusts the signal amplification gain based on the maximum signal strength change detected when the user performs the calibration action. S5. Real-time closed-loop fine-tuning: Continuously updates mapping parameters and corrects feature drift during the interaction process.

[0034] In a clinical application example, taking a patient with high-level paraplegia as an example, the operation steps are as follows: Wearing and connection: The patient wears a submental / mandibular wearable device, and the system is paired with a computer via Bluetooth.

[0035] Intelligent recognition execution: The CNN-LSTM deep learning model built into the processor extracts features from at least two channels of RF signals.

[0036] Fine control: Patients can achieve high-precision cursor positioning and selection by continuously generating control intentions and matching them with the terminal, meeting their fine control needs in social, office and daily entertainment.

[0037] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A tongue-to-computer interaction system based on raw ultrasound radio frequency signals, comprising: The ultrasound sensing unit is designed to fit against the user's submental / mandibular region, emitting ultrasound waves into the tongue tissue and receiving multi-channel raw RF echo signals scattered back by the internal fiber bundles of the tongue muscles. A radio frequency signal processing unit, connected to the ultrasonic sensing unit, is used to amplify, filter, and perform analog-to-digital conversion on the multi-channel raw RF echo signal to form a multi-channel RF signal sequence. The feature construction unit, connected to the radio frequency signal processing unit, is used to extract spatiotemporal features characterizing tongue muscle deformation and contraction based on the multi-channel RF signal sequence, and form a spatiotemporal feature map; The intent decoding and neural feedback unit, connected to the feature construction unit, is used to decode the spatiotemporal feature map into a standard interactive intent and output a neural feedback signal; and to map the standard interactive intent into a standard human-computer interaction command and send it to the terminal device through the wireless communication unit. An adaptive calibration unit, connected to the radio frequency signal processing unit, feature construction unit, and / or intent decoding and neural feedback unit, is used to perform online calibration to update the parameters of signal processing, feature extraction, or intent decoding, and to compensate for signal and feature changes caused by wearing status or user physiological changes.

2. The tongue-to-computer interaction system based on raw ultrasound radio frequency signals according to claim 1, characterized in that: The ultrasonic sensing unit includes a wearable carrier and an ultrasonic transducer array. The wearable carrier covers at least the submental / mandibular region of the user. The ultrasonic transducer array has at least two channels and is encapsulated within the wearable carrier corresponding to the submental / mandibular region.

3. The tongue-to-computer interaction system based on raw ultrasound radio frequency signals according to claim 2, characterized in that: The wearable device is a subchin / mandibular support made of medical-grade liquid silicone.

4. The tongue-to-computer interaction system based on raw ultrasound radio frequency signals according to claim 2, characterized in that: The ultrasonic transducer array is arranged in an arc shape, with the main beam focused on the core area where the intralingual and lateral tongue muscles intersect. The ultrasonic transducer array consists of multiple miniature high-frequency ultrasonic transducers, with the operating frequency set to 0.1-30MHz.

5. The tongue-to-computer interaction system based on raw ultrasound radio frequency signals according to claim 1, characterized in that: The method for processing raw radio frequency data by the feature construction unit includes extracting deformation features of intralingual and lateral lingual muscle fiber bundles at different depths based on a multi-channel RF signal sequence. By acquiring signals from different anatomical positions of the tongue in parallel through multiple channels, and analyzing the spatiotemporal correlation of the RF signal envelopes between channels, the coordinated movement patterns of the tongue muscles are identified, forming a spatiotemporal feature atlas. The coordinated movement patterns include at least one of overall tongue elevation, local curling, or lateral displacement. The spatiotemporal feature atlas is a three-dimensional atlas, whose dimensions include channels, depth, and time.

6. The tongue-to-computer interaction system based on raw ultrasound radio frequency signals according to claim 1, characterized in that: The intent decoding and neural feedback unit is configured to decode a specific co-contraction pattern of tongue muscle fiber bundles into an interactive intent, and map the interactive intent into a standard human-computer interaction command; the neural feedback signal is used to guide the user to form a stable intent output pattern.

7. The tongue-to-computer interaction system based on raw ultrasound radio frequency signals according to claim 1, characterized in that: The online calibration steps performed by the adaptive calibration unit include: S1. Static reference acquisition: Acquire signals while the user is at rest and set the zero-point threshold; S2. Dynamic range calibration: Guide users to generate preset intention actions, record the feature distribution and dynamic extreme values ​​corresponding to each intention, and establish personalized intention-feature mapping relationship or decision boundary. S3. Normalization: Map the real-time acquired signal to the standard interval [0, 1]. S4. Dynamic Gain Compensation: Dynamically adjusts the signal amplification gain based on the maximum signal strength change detected when the user performs the calibration action. S5. Real-time closed-loop fine-tuning: Continuously updates mapping parameters and corrects feature drift during the interaction process.

8. A tongue-to-computer interaction method based on raw ultrasound radio frequency signals, using the tongue-to-computer interaction system based on raw ultrasound radio frequency signals according to any one of claims 1-7, as described in claim 1, characterized in that, Includes the following steps: The ultrasonic sensing unit, which is attached to the user's subchinal / mandibular region, emits ultrasound to the tongue tissue and simultaneously acquires multi-channel raw RF echo signals scattered back from the tongue muscles. The original multi-channel RF echo signals are processed to form a multi-channel RF signal sequence; Based on the multi-channel RF signal sequence, spatiotemporal features characterizing tongue muscle deformation and contraction are extracted, and a spatiotemporal feature map is constructed. The spatiotemporal feature map is decoded into a standard interactive intent and a neural feedback signal is generated. Based on the standard interactive intent, interactive instructions corresponding to various terminals are generated. Perform online adaptive calibration to update parameters for signal processing, feature extraction, or intent decoding, compensating for changes caused by wearing conditions or user physiological variations.

9. The tongue-to-computer interaction method based on raw ultrasound radio frequency signals according to claim 8, characterized in that: The steps for constructing the spatiotemporal feature map include: The deformation amplitude characteristics of tongue muscle fiber bundles at different depths were extracted from the multi-channel RF signal sequence; By analyzing the spatiotemporal correlation of the envelope of multi-channel RF signals, the coordinated movement patterns of the tongue are identified, and a spatiotemporal feature map is formed; the coordinated movement patterns include at least one of overall tongue elevation, local curling and lateral displacement.

10. The tongue-to-computer interaction method based on raw ultrasound radio frequency signals according to claim 8, characterized in that: The step of performing online adaptive calibration includes sequentially executing the following sub-steps: S1. Static reference acquisition: Acquire signals while the user is at rest and set the zero-point threshold; S2. Dynamic range calibration: Guide users to generate preset intention actions, record the feature distribution and dynamic extreme values ​​corresponding to each intention, and establish personalized intention-feature mapping relationship or decision boundary. S3. Normalization: Map the real-time acquired signal to the standard interval [0, 1]. S4. Dynamic Gain Compensation: Dynamically adjusts the signal amplification gain based on the maximum signal strength change detected when the user performs the calibration action. S5. Real-time closed-loop fine-tuning: Continuously updates mapping parameters and corrects feature drift during the interaction process.

Images