Intelligent exoskeleton bone auxiliary conduction hearing aid system capable of being remotely controlled

Abstract

The invention discloses an intelligent exoskeleton bone auxiliary conduction hearing aid system capable of being remotely controlled and having an electroencephalogram control function, and relates to the technical field of intelligent rehabilitation robots and hearing aid devices. The wearable mechanical part comprises a flexible actuator, an elastic fabric base material, an inertial measurement unit IMU, a plantar pressure sensor array, a surface myoelectricity EMG sensor and an electroencephalogram EEG sensor. And the information processing communication part comprises a central controller, a bone conduction hearing aid module, a multi-mode sensor, an embedded SoC, a DSP / AI accelerator, a 4G / 5G / Wi-Fi module and a power management unit. The flexible exoskeleton walking aid system and the bone conduction hearing aid module are integrated, the electroencephalogram EEG sensor is additionally arranged to achieve the electroencephalogram control function, and cooperation of the three functions of movement assistance, auditory perception and neural intention recognition is achieved. A user can directly control the exoskeleton action mode through electroencephalogram signals, meanwhile, intelligent assistance of lower limb joints is obtained in the walking process, and environment sounds and voice prompts are clearly received in a bone conduction mode. According to the system, the perception ability, the control flexibility and the social participation degree of old people or rehabilitation patients who are inconvenient to move and accompanied with hearing impairment in a complex environment are remarkably improved, and more natural and convenient man-machine interaction experience is provided for users.

Description

Technical Field

[0001] This invention belongs to the technical field of intelligent rehabilitation robots and hearing aids, and in particular relates to a remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system and a brain-computer interface. Background Technology

[0002] The problem of mobility impairment caused by musculoskeletal degeneration is becoming increasingly prominent among the elderly. Traditional assistive tools such as canes and wheelchairs have limited functions and cannot meet the diverse needs of daily walking and rehabilitation training.

[0003] Against this backdrop, exoskeleton robots, as wearable assistive devices, have emerged. By providing power assistance from outside the body, they significantly improve the mobility of people with mobility impairments, demonstrating great potential in fields such as medical rehabilitation and elderly mobility assistance. However, existing exoskeleton systems still face several technical bottlenecks in practical applications: First, the accuracy of motion intention recognition is insufficient. Most systems rely on physical signals such as plantar pressure or joint angles, resulting in response lag and making it difficult to achieve seamless human-machine coordination. Brain-computer interface technology, by directly collecting neural signals from the motor cortex of the brain, can reduce the intention recognition delay to less than 100ms. Second, there is a lack of real-time monitoring and risk warning capabilities for the user's health status, especially among the elderly population where falls are common, as existing devices generally lack active fall prevention functions. Third, the level of system intelligence is limited, with control strategies mostly based on preset modes, lacking the adaptive ability to dynamically adjust according to the user's physiological state. On the other hand, hearing impairment is also a common health problem among the elderly. Traditional air conduction hearing aids have limited effectiveness for those with middle or outer ear damage, while bone conduction hearing aids transmit sound waves directly to the inner ear through the skull, providing an effective solution for patients with conductive hearing loss. Currently, there are various bone conduction hearing aids on the market, but most are standalone wearable products and have not yet been integrated with movement assistive systems such as exoskeletons. This results in users having to operate multiple devices simultaneously, causing inconvenience and information fragmentation. Furthermore, falls have become a major public health problem threatening the safety of the elderly. Millions of elderly people worldwide are injured by falls each year. In my country, falls are the leading cause of injury-related deaths among those aged 65 and above. Although research has attempted to apply sensors and control algorithms to wearable devices for fall detection and early warning—for example, the intelligent hip-assisting exoskeleton developed in Switzerland can respond to slip signs and apply corrective torque within 350 milliseconds—such systems are mostly in the early stages of development, suffering from bulky equipment, high costs, and weak algorithm generalization capabilities, making large-scale application difficult.

[0004] To address these issues, we provide a remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system. Summary of the Invention

[0005] The purpose of this invention is to provide a remotely controllable intelligent exoskeleton bone conduction hearing aid system. By integrating a flexible exoskeleton mobility aid device with a bone conduction hearing aid module, and combining multimodal sensor fusion, deep learning intention recognition, real-time health monitoring, fall prediction and protection, and remote communication control technologies, this invention solves the problems in the prior art such as the functional separation of mobility aid devices and hearing aid devices, delayed motion intention recognition, lack of active safety protection mechanisms, and inability to achieve remote monitoring and personalized intervention.

[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0007] This invention relates to a remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system, comprising:

[0008] The wearable mechanical components include flexible actuators, elastic fabric substrates, inertial measurement units (IMUs), plantar pressure sensor arrays, and surface electromyography (EMG) sensors.

[0009] The information processing and communication section includes a central controller, a bone conduction hearing aid module, a brain-computer interface module, a multimodal sensor, an embedded SoC, a DSP / AI accelerator, a 4G / 5G / Wi-Fi module, and a power management unit.

[0010] The flexible actuator is integrated into the elastic fabric substrate and provides joint assistance through pneumatic artificial muscles or flexible cables. The bone conduction hearing aid module includes a vibrator that fits the temporal bone and a shoulder microphone, and is connected to the central controller via Bluetooth technology. The central controller runs a real-time operating system and is connected to various functional modules via a high-speed bus or wireless communication to achieve data acquisition, preprocessing, AI inference, motor drive control, and remote interaction.

[0011] The present invention is further configured such that the multimodal sensor includes a three-axis gyroscope, a three-axis accelerometer, a three-axis magnetometer, a millimeter-wave radar, an EEG brain-computer interface, a heart rate / blood oxygen / temperature sensor, and a joint force sensor.

[0012] The present invention is further configured such that the driving method of the flexible actuator includes:

[0013] Pneumatic artificial muscles are made of fiber-reinforced, highly elastic polymers and contract or extend by controlling internal air pressure.

[0014] The flexible cable drive uses a micro motor in conjunction with a pull wire made of aramid fiber or ultra-high molecular weight polyethylene. Joint assistance is achieved through gear reduction and pull wire traction.

[0015] The present invention is further configured such that the AI ​​algorithm running by the central controller includes:

[0016] The intent recognition model takes IMU signals, plantar pressure center trajectory, sEMG signals, millimeter-wave radar point cloud frame sequences, and EEG neural signals collected by the brain-computer interface as inputs, and outputs the user's current intent state and prediction of the next action.

[0017] The fall prediction model takes IMU signal mutation, abnormal plantar pressure distribution, radar point cloud displacement and attitude deviation angle as inputs, and outputs fall risk status and trigger parameters.

[0018] The personalized enhancement model for bone conduction hearing aids takes ambient microphone sound, user hearing response spectrum and historical wearing data as inputs, and outputs personalized gain configuration parameters.

[0019] The present invention is further configured such that the intent recognition model adopts a structure combining temporal Transformer and graph neural network ST-GCN to model the sensor node topology and learn the temporal action evolution.

[0020] The fall prediction model adopts a strategy that combines mid-level fusion and high-level fusion: mid-level fusion uses a fully connected neural network to nonlinearly combine multiple sensor inputs; high-level fusion uses conditional triggering rules to determine a fall event when acceleration impact, sudden posture change, and abnormal bone vibration are all met simultaneously.

[0021] The miniature audio processor within the bone conduction hearing aid module runs the C3T contrastive learning framework to perform self-supervised clustering of sounds in different scenarios and adjusts the gain of each frequency band through an adaptive convolutional neural network.

[0022] The present invention is further configured such that a remote control assistance strategy model runs between the central controller and the cloud server. The high-level model uses Dynamic Motion Primitives (DMP) in hierarchical reinforcement learning to generate action templates, while the low-level model uses reinforcement learning agents to fine-tune the assistance intensity and joint flexibility.

[0023] The present invention is further configured such that a lightweight student model, compressed by model distillation, is deployed on the embedded SoC. This student model is obtained by transferring knowledge from a teacher model pre-trained on the Isaac Sim simulation platform through Soft-Target distillation and feature alignment techniques, and the number of parameters is less than 10% of the original model.

[0024] The present invention is further configured such that the dynamic modeling of the flexible exoskeleton adopts the Lagrangian method, and the system dynamic equation is:

[0025]

[0026] Where q is the generalized coordinate, and M(q) is the inertial matrix. Let G(q) be the Coriolis force and the centrifugal force matrix, and K be the gravity term. el For the equivalent stiffness, q0 represents the natural state, and τ... exo For the active torque of the exoskeleton, τ human It outputs torque to the human muscles.

[0027] The present invention is further configured such that the control of the exoskeleton adopts an impedance control law:

[0028]

[0029] Where, q d and Given the target joint angle and velocity, q and K is the current measured value. d With B d The target elasticity and damping parameters are given.

[0030] The present invention is further configured such that the system performs the following operations after detecting a fall event:

[0031] Cut off the exoskeleton motor drive output and lock the position of each joint;

[0032] Send an alarm message containing the fall time, GPS location, and sensor data to a preset contact via wireless network;

[0033] The bone conduction hearing aid plays voice prompts and receives user voice or key input to confirm the status.

[0034] The present invention has the following beneficial effects:

[0035] 1. This invention integrates a flexible exoskeleton walking system with a bone conduction hearing aid module, achieving a dual function of motion assistance and auditory perception. Traditional exoskeletons focus on limb assistance, while hearing aids are used independently. This invention breaks down this functional separation, enabling elderly people or rehabilitation patients with mobility impairments and hearing loss to receive intelligent assistance from their lower limb joints while walking, and to clearly receive environmental sounds and voice prompts through bone conduction. This significantly improves the user's perception ability and social participation in complex environments, especially enhancing safety and independence in scenarios such as traffic warnings and interpersonal communication.

[0036] 2. This invention incorporates an active health monitoring and fall prediction and protection mechanism, significantly enhancing user safety. The system monitors physiological and motor parameters such as heart rate, blood oxygen, and gait stability in real time. When fatigue or imbalance is detected, it can issue a voice warning in advance. When a fall risk occurs, the algorithm identifies abnormal postures and drives the exoskeleton to apply a reverse torque to assist the user in regaining balance. If a fall is unavoidable, the joints are immediately locked, power is cut off, and a remote alarm is activated, realizing a complete safety closed loop from prevention and intervention to emergency response, greatly reducing the risk of secondary injuries caused by falls for the elderly and rehabilitation patients.

[0037] 3. This invention employs multimodal sensor fusion and deep learning algorithms, significantly improving the accuracy and response speed of motion intention recognition. The system comprehensively collects multi-source information such as inertial measurement unit (IMU), plantar pressure, surface electromyography (EMG), and millimeter-wave radar. It utilizes temporal Transformer and graph neural network ST-GCN to construct a spatiotemporal joint model, which can predict the user's next movement at the initial stage of muscle activation. This effectively solves the control lag problem caused by traditional exoskeletons relying on signals after the movement is executed, making human-machine collaboration more natural and smooth, and avoiding the discomfort of "humans adapting to machines".

[0038] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0040] Figure 1 This is a schematic diagram of the workflow of a remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system.

[0041] Figure 2 This is a flowchart illustrating the working principle of a remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system.

[0042] Figure 3 The application of knowledge distillation technology in deep learning models for remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid systems.

[0043] Figure 4 This is a schematic diagram illustrating the multimodal data acquisition and fusion of a remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system. Detailed Implementation

[0044] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0045] Example 1

[0046] Please see Figure 1-4 This invention relates to a remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system, comprising:

[0047] The wearable mechanical components include flexible actuators, elastic fabric substrates, inertial measurement units (IMUs), plantar pressure sensor arrays, and surface electromyography (EMG) sensors.

[0048] The information processing and communication section includes a central controller, a bone conduction hearing aid module, a brain-computer interface module, a multimodal sensor, an embedded SoC, a DSP / AI accelerator, a 4G / 5G / Wi-Fi module, and a power management unit.

[0049] The flexible actuator is integrated into the elastic fabric substrate and provides joint assistance through pneumatic artificial muscles or flexible cables. The bone conduction hearing aid module includes a vibrator that fits the temporal bone and a shoulder microphone, and is connected to the central controller via Bluetooth technology. The central controller runs a real-time operating system and is connected to various functional modules via a high-speed bus or wireless communication to achieve data acquisition, preprocessing, AI inference, motor drive control, and remote interaction.

[0050] This invention is based on human-machine collaboration, multimodal perception and intelligent closed-loop control technology. By deeply integrating a wearable flexible exoskeleton system with a bone conduction hearing aid module, and combining embedded artificial intelligence algorithms and remote communication architecture, it constructs a comprehensive intelligent system that integrates motion assistance, perception enhancement, health monitoring and remote intervention.

[0051] This invention further enhances brain-computer interface (BCI) control functionality by acquiring the user's neural activity information through an EEG signal acquisition device and converting it into control commands that can be recognized by the system, thereby enabling direct operation based on the BCI. This function allows users to control the device through thought processes, expanding the ways of human-computer interaction and improving the convenience and intelligence of operation.

[0052] According to claim 1, the specific implementation of the present invention includes the following steps: First, the user's brain activity is monitored in real time using a high-precision EEG signal acquisition device; second, the acquired EEG signals are denoised and feature extracted using signal processing algorithms; then, the extracted features are mapped to corresponding control commands; finally, these commands are transmitted to the target device to achieve the expected operational function. The above process ensures the high efficiency and accuracy of brain-computer interface control.

[0053] Furthermore, this invention also relates to the integration of brain-computer interface modules with other system components, including hardware connection protocols, data transmission standards, and software compatibility design, to ensure the stable operation of the overall system and optimize the user experience.

[0054] The brain-computer interface system of this invention uses a non-invasive electrode array to acquire neural signals and extracts effective features through a multi-channel signal amplifier and filter. Its key components include a signal acquisition module consisting of a 64-channel dry electrode array, covering the motor cortex region of the brain and sampling at a frequency of 2000Hz to ensure high-quality neural electrical signals; and a feature extraction unit that uses wavelet transform and independent component analysis to separate feature vectors related to motor intention from the raw signal. The decoding algorithm includes a motor imagery decoder based on a convolutional neural network (CNN) architecture that maps time-frequency features to specific motor commands; an emotion state classifier that uses a long short-term memory network (LSTM) to analyze the alpha and beta wave patterns of the brain; and a feature extraction unit that analyzes the alpha and beta wave patterns of the brain. The system includes a fatigue monitoring module that assesses the user's mental state by changing the eta / beta ratio; a feedback control system that provides real-time visual, tactile, and auditory feedback, with real-time visual feedback displaying motion intention recognition results via AR glasses, tactile feedback using a vibration motor array to provide operation confirmation signals, and auditory feedback integrating bone conduction headphones for barrier-free sound prompts; and a safety protection mechanism that includes impedance monitoring to detect electrode contact quality in real time, overload protection with current limits set within a safe range (≤100μA), and data encryption using the AES-256 standard to protect neural data transmission. This brain-computer interface system is seamlessly integrated with pneumatic artificial muscles and a flexible cable drive system, and coordinates the operation of each subsystem through a central controller to form a complete intelligent assistive device.

[0055] Specifically, the driving method of the flexible actuator includes:

[0056] Pneumatic artificial muscles are made of fiber-reinforced, highly elastic polymers and contract or extend by controlling internal air pressure.

[0057] The flexible cable drive uses a micro motor in conjunction with a pull wire made of aramid fiber or ultra-high molecular weight polyethylene. Joint assistance is achieved through gear reduction and pull wire traction.

[0058] The motion commands of these flexible actuators can be generated in real time by the central controller based on the neural signals parsed by the brain-computer interface system, realizing a direct mapping from the brain's intention to move to mechanical execution.

[0059] Furthermore, the multimodal sensors include a three-axis gyroscope, a three-axis accelerometer, a three-axis magnetometer, a millimeter-wave radar, an EEG brain-computer interface, a heart rate / blood oxygen / temperature sensor, and a joint force sensor;

[0060] The central controller runs a remote control assistance strategy model between the central controller and the cloud server. The high-level model uses Dynamic Motion Primitives (DMP) in hierarchical reinforcement learning to generate action templates, while the low-level model uses reinforcement learning agents to fine-tune the assistance intensity and joint flexibility.

[0061] The embedded SoC is equipped with a lightweight student model that has been compressed by model distillation. This student model is obtained by transferring knowledge from a teacher model pre-trained on the Isaac Sim simulation platform through soft-target distillation and feature alignment techniques. The number of parameters is less than 10% of the original model.

[0062] Example 2

[0063] Based on the hardware of Specific Embodiment 1, the AI ​​algorithm run by the central controller includes:

[0064] The intent recognition model takes IMU signals, plantar pressure center trajectory, sEMG signals, millimeter-wave radar point cloud frame sequences, and EEG neural signals collected by the brain-computer interface as inputs, and outputs the user's current intent state and prediction of the next action. The intent recognition model adopts a structure that combines temporal Transformer and graph neural network ST-GCN to model the sensor node topology and learn the temporal evolution of actions.

[0065] The fall prediction model takes IMU signal mutations, abnormal plantar pressure distribution, radar point cloud displacement changes, and attitude deviation angles as inputs, and outputs fall risk status and trigger parameters. The fall prediction model adopts a strategy combining mid-level fusion and high-level fusion: mid-level fusion uses a fully connected neural network to nonlinearly combine multiple sensor inputs; high-level fusion uses conditional triggering rules to determine a fall event when acceleration impact, attitude mutation, and abnormal bone vibration are simultaneously satisfied.

[0066] The personalized enhancement model for bone conduction hearing aids takes environmental microphone sound, user hearing response spectrum and historical wearing data as inputs, and outputs personalized gain configuration parameters. The miniature audio processor in the bone conduction hearing aid module runs the C3T contrastive learning framework to perform self-supervised clustering of sounds in different scenarios and adjusts the gain of each frequency band through an adaptive convolutional neural network.

[0067] Specifically, the dynamic modeling of the flexible exoskeleton uses the Lagrangian method, and the system dynamic equations are as follows:

[0068]

[0069] Where q is the generalized coordinate, and M(q) is the inertial matrix. Let G(q) be the Coriolis force and the centrifugal force matrix, and K be the gravity term. el For the equivalent stiffness, q0 represents the natural state, and τ... exo For the active torque of the exoskeleton, τ human It outputs torque to the human muscles.

[0070] The exoskeleton is controlled using an impedance control law:

[0071]

[0072] Where, q d and Given the target joint angle and velocity, q and K is the current measured value. d With B d The target elasticity and damping parameters are given.

[0073] In addition, the system performs the following operations after detecting a fall event:

[0074] Cut off the exoskeleton motor drive output and lock the position of each joint;

[0075] Send an alarm message containing the fall time, GPS location, and sensor data to a preset contact via wireless network;

[0076] The bone conduction hearing aid plays voice prompts and receives user voice or key input to confirm the status.

[0077] Combining Embodiments 1 and 2, the specific usage steps of this invention are as follows: The user first wears a lower limb exoskeleton brace, aligns the flexible actuator with the hip and knee joints, and secures it to the legs using elastic straps or composite fabric substrates. Simultaneously, the user wears bone conduction headphones and a shoulder microphone that conform to the temporal bone, ensuring full contact between the vibrator and the skin. After powering on the system, the central controller executes the initialization calibration process, each sensor performs zero-point calibration, and detects the IMU baseline, plantar pressure distribution, electromyographic signal background value, and brain-computer interface electrode contact quality in a static state. Simultaneously, the bone conduction module plays test audio to verify the normality of the sound conduction pathway. After calibration, the system enters standby mode, continuously acquiring multimodal sensor data with low power consumption, including I... The system collects acceleration and angular velocity signals from the MU (Muscle / Molecular Unit), plantar pressure center of gravity trajectory, surface electromyography (EMG) signals, millimeter-wave radar point cloud frames, brain-computer interface (BCI) EEG neural signals, and physiological parameters such as heart rate and blood oxygenation. When the user expresses the intention to walk, the BCI system first analyzes the characteristics of motor cortex neural activity. Combining this with the EMG activity generated by the quadriceps femoris or tibialis anterior muscle and the forward shift of the body's center of gravity, the system analyzes these aura signals in real time through a multimodal intention recognition model. Within 50 to 100 milliseconds, it determines the "ready to start" state and switches the control mode to walking assistance mode. The central controller then drives the flexible actuators of the hip and knee joints to apply a preset assist torque to assist the user in taking the first step. During walking, the intention recognition model continuously analyzes the sensors. The system outputs data on the current gait phase (such as the support phase and swing phase) and movement state. Based on this, the controller dynamically adjusts the assist intensity and output timing of each joint to achieve synchronization and coordination with the user's steps. Simultaneously, the bone conduction hearing aid module continuously collects ambient sounds. After processing by a personalized enhancement model, key audio signals such as car horns and human voices are conducted through the skull to the inner ear, ensuring that the user can still perceive external information even with hearing impairment. During system operation, the health monitoring module analyzes data such as heart rate and gait stability in real time. If the heart rate consistently exceeds a preset threshold or gait variability significantly increases, a voice prompt is issued through the bone conduction headphones. If the system identifies pre-fall warning signs such as sudden acceleration changes, excessive posture angles, and abnormal foot pressure... Within 300 milliseconds, the system activates a protective mechanism, correcting body posture by increasing hip joint extension torque and simultaneously issuing an acoustic warning of "Caution: Fall Prevention." If a fall actually occurs, the system immediately cuts off motor output and locks all joints to prevent secondary injury. At the same time, it sends an emergency alarm containing the fall time, GPS location, and sensor snapshots to a preset contact via 4G / 5G network and plays a reassuring voice message through headphones, waiting for the user's voice response or remote intervention. In rehabilitation training areas and fitness settings, after logging into the system, remote client users can send mode switching commands to the central controller via the cloud server, such as switching the system to "Rehabilitation Training Mode" and adjusting the assistance level, joint range of motion, or setting specific movement trajectories.The exoskeleton executes control strategies based on new parameters, assisting patients in completing training movements such as sitting-to-standing transfers and stepping in place. During the process, data collected, including electromyography (EMG), joint torque, and electroencephalography (EEG) characteristics, are uploaded to the cloud in real time for therapists to assess training effectiveness. After each day's use, the system automatically saves operation logs, physiological data, and event records, and can wirelessly synchronize these to a remote platform to generate analysis reports.

[0078] The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system provided by this invention is based on flexible actuation technology. It uses pneumatic artificial muscles (PAM) or flexible cables as a power source. These actuators have high power density, lightweight and biomimetic flexibility, and can conform to the biomechanical characteristics of the human body, providing joint assistance without restricting natural movement. These flexible actuators are integrated into elastic fabrics or composite substrates to form a low-invasive wearable mechanical structure, which not only ensures the flexibility and wearing comfort of the system, but also effectively protects the internal components from external environmental interference through a sealed design.

[0079] At the perception level, the system deploys a multimodal sensor network, including a brain-computer interface (EEG) electrode array distributed on the head, an inertial measurement unit (IMU) on the waist, lower limbs and hearing aids, a plantar pressure sensor array, surface electromyography (EMG) electrodes, millimeter-wave radar, and physiological parameter sensors (heart rate, blood oxygen, body temperature). This enables the synchronous acquisition of human posture, movement status, neural activity, muscle activation signals and environmental information. The brain-computer interface electrode array covers the motor cortex of the brain and acquires neural electrical signals at a sampling frequency of 2000Hz. Wavelet transform and independent component analysis techniques are used to extract motion intention feature vectors. All sensor data converges to the central controller—a high-performance edge computing platform integrating an embedded SoC, DSP / AI accelerator, 4G / 5G / Wi-Fi communication module and power management unit. This controller runs a real-time operating system and is responsible for preprocessing the raw data, such as time synchronization and filtering normalization, and calling on locally deployed lightweight deep learning models to complete key decision-making tasks.

[0080] In terms of algorithm architecture, the system adopts a layered fusion strategy: the lower layer improves the accuracy of posture estimation by fusing multi-channel IMU data through Kalman filtering; the middle layer integrates radar point clouds, IMU motion features, foot pressure trajectories, physiological indicators, and brain-computer interface neural features into a comprehensive feature vector; the upper layer combines a rule engine and a neural network to output the final judgment result, such as motion intention classification or fall event determination; specifically, the intention recognition model uses a temporal Transformer and a graph neural network (ST-GCN) for joint modeling, extracting spatiotemporal dynamic features from EEG signals and multi-channel temporal signals to achieve millisecond-level prediction of actions such as "starting", "walking", and "stopping"; the fall prediction model triggers a protection mechanism within 300ms by detecting the coupling conditions of acceleration abrupt changes, posture imbalance, and abnormal pressure distribution; at the same time, the bone conduction hearing aid module works independently but is logically linked, its shoulder microphone collects ambient sound, which is processed by a personalized enhancement model and transmitted to the inner ear by the temporal bone oscillator through bone conduction, ensuring that hearing-impaired users can clearly receive voice prompts and alarm information.

[0081] The control logic of the entire system follows a closed-loop framework of "perception-decision-execution": the central controller generates target torque commands based on the neural signals parsed by the brain-computer interface and the intention state output by the AI ​​model, and adjusts the output of the flexible actuators in combination with impedance control or model predictive control (MPC) algorithms to make the exoskeleton movements highly coordinated with human movements; when a potential risk is detected, the system immediately adjusts the joint stiffness, applies a reaction torque and activates an acoustic warning to achieve active balance maintenance.

[0082] Furthermore, the system connects to a cloud server via 4G / 5G / Wi-Fi modules to establish a remote monitoring and interaction channel. This allows authorized clients to access device status, upload control policies, receive emergency alarms, and retrieve brain-computer interface neural feature data, thereby expanding the system's application dimensions in rehabilitation medicine and home care. The overall structure utilizes model distillation technology to compress complex AI models to a scale executable at the edge, solving the limitations of computing power and power consumption. This enables the efficient deployment of high-performance intelligent algorithms on wearable devices, ultimately forming a compact, highly integrated, and rapidly responsive intelligent human-machine fusion system.

[0083] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

Claims

1. A remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system, characterized in that, include: The wearable mechanical components include flexible actuators, elastic fabric substrates, inertial measurement units (IMUs), plantar pressure sensor arrays, and surface electromyography (EMG) sensors. The information processing and communication section includes a central controller, a bone conduction hearing aid module, a brain-computer interface module, a multimodal sensor, an embedded SoC, a DSP / AI accelerator, a 4G / 5G / Wi-Fi module, and a power management unit. The flexible actuator is integrated into the elastic fabric substrate and provides joint assistance through pneumatic artificial muscles or flexible cables. The bone conduction hearing aid module includes a vibrator that fits the temporal bone and a shoulder microphone, and is connected to the central controller via Bluetooth technology. The central controller runs a real-time operating system and is connected to various functional modules via a high-speed bus or wireless communication to achieve data acquisition, preprocessing, AI inference, motor drive control, and remote interaction.

2. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 1, characterized in that, The multimodal sensors include a three-axis gyroscope, a three-axis accelerometer, a three-axis magnetometer, a millimeter-wave radar, an EEG brain-computer interface, a heart rate / blood oxygen / temperature sensor, and a joint force sensor.

3. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 1, characterized in that, The driving method of the flexible actuator includes: Pneumatic artificial muscles are made of fiber-reinforced, highly elastic polymers and contract or extend by controlling internal air pressure. The flexible cable drive uses a micro motor in conjunction with a pull wire made of aramid fiber or ultra-high molecular weight polyethylene. Joint assistance is achieved through gear reduction and pull wire traction.

4. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 1, characterized in that, The AI ​​algorithms run by the central controller include: The intent recognition model takes IMU signals, plantar pressure center of gravity and gravity center COG trajectory, sEMG signals, millimeter-wave radar point cloud frame sequences and EEG neural signals collected by brain-computer interface as inputs, and outputs the user's current action features as intent state and prediction of the next action. The fall prediction model takes IMU signal mutation, abnormal plantar pressure distribution, radar point cloud displacement and attitude deviation angle as inputs, and outputs fall risk status and trigger parameters. The bone conduction hearing aid embeds an IMU personalized enhancement model, with inputs including ambient microphone sound, user hearing response spectrum, and historical wearing data, and outputs personalized gain configuration parameters.

5. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 4, characterized in that, The intent recognition model adopts a structure that combines temporal Transformer and graph neural network ST-GCN to model the sensor node topology and learn the temporal evolution of actions. The fall prediction model adopts a strategy that combines mid-level fusion and high-level fusion: mid-level fusion uses a fully connected neural network to nonlinearly combine multiple sensor inputs; The high-level integration uses conditional triggering rules to determine a fall event when acceleration impact, sudden posture change, and abnormal bone vibration are all met simultaneously. The miniature audio processor within the bone conduction hearing aid module runs the C3T contrastive learning framework to perform self-supervised clustering of sounds in different scenarios and adjusts the gain of each frequency band through an adaptive convolutional neural network.

6. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 1, characterized in that, The central controller runs a remote control assistance strategy model between itself and the cloud server. The high-level model uses Dynamic Motion Primitives (DMP) in hierarchical reinforcement learning to generate action templates, while the low-level model uses reinforcement learning agents to fine-tune the assistance intensity and joint flexibility.

7. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 1, characterized in that, The embedded SoC is equipped with a lightweight student model that has been compressed by model distillation. This student model is obtained by transferring knowledge from a teacher model pre-trained on the Isaac Sim simulation platform through soft-target distillation and feature alignment techniques. The number of parameters is less than 10% of the original model, and the exoskeleton control signals are enhanced by hearing aids.

8. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 1, characterized in that, The dynamic modeling of the flexible exoskeleton uses the Lagrange method, and the system dynamic equations are as follows: Where q is the generalized coordinate, and M(q) is the inertial matrix. Let G(q) be the Coriolis force and the centrifugal force matrix, and K be the gravity term. el For the equivalent stiffness, q0 represents the natural state, and τ... exo For the active torque of the exoskeleton, τ human It outputs torque to the human muscles.

9. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 1, characterized in that, The exoskeleton is controlled using an impedance control law: Where, q d and Given the target joint angle and velocity, q and K is the current measured value. d With B d The target elasticity and damping parameters are given.

10. The remotely controllable intelligent exoskeleton bone-assisted conduction hearing aid system according to claim 1, characterized in that, The system performs the following operations after detecting a fall event: Cut off the exoskeleton motor drive output and lock the position of each joint; Send an alarm message containing the fall time, GPS location, and sensor data to a preset contact via wireless network; The bone conduction hearing aid plays voice prompts and receives user voice or key input to confirm the status.

Images