Non-inductive interaction control method and system based on multi-sensor fusion

Abstract

The application relates to the field of smart home control, and provides a non-sensing interaction control method and system based on multi-sensor fusion, which comprises the following steps: acquiring inertial sensing data and electroencephalogram signal data; determining an effective direction intention and generating a direction control instruction based on the inertial sensing data through direction keeping judgment; determining a concentration state and generating a permission control signal for limiting movement based on the electroencephalogram signal data; recognizing at least two kinds of trigger mechanism emergency stop intentions and generating an emergency stop signal based on the electroencephalogram signal data; arbitrating the direction control instruction, the permission control signal and the emergency stop signal according to a priority rule, and outputting a final control instruction, wherein the priority of the emergency stop signal is the highest, and the priority of the permission control signal is higher than that of the direction control instruction. The application aims to solve the problems of high false trigger rate of existing auxiliary equipment, lack of active safety guarantee and unreliable emergency stop. Through multi-sensor fusion and multi-level priority arbitration, the application realizes high safety, anti-interference and non-sensing interaction control.

Description

Technical Field

[0001] This invention relates to the field of smart home control technology, and in particular to a seamless interactive control method and system based on multi-sensor fusion. Background Technology

[0002] In the field of smart home control, equipment such as wheelchairs and nursing beds for the elderly and people with mobility impairments are gradually developing towards intelligence and high security.

[0003] Currently, the operation of wheelchairs, nursing beds, and other devices mainly relies on hand controls (joysticks, buttons), voice control, or single inertial sensors (such as head posture gyroscopes). Among these, single inertial sensor solutions have attracted attention due to their natural operation and lack of hand intervention. However, existing technologies have the following problems: First, they are sensitive to slight head tremors, and small deviations are easily misinterpreted as valid commands, resulting in a high false trigger rate. Second, they cannot automatically lock the movement when the user is not focused, drowsy, or distracted, posing a safety hazard. Third, they rely on a single action or button, which may fail to trigger or trigger falsely in dangerous situations, and some solutions rely on machine learning models, making it difficult to guarantee reliability on low-channel devices.

[0004] Therefore, how to reduce the false trigger rate, provide proactive safety guarantees, and realize a reliable emergency stop mechanism that does not require machine learning, thereby achieving highly secure and interference-resistant seamless interactive control, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] This invention provides a contactless interactive control method and system based on multi-sensor fusion to solve the problems of high false triggering rate, lack of active safety protection, and unreliable emergency stop mechanism in existing auxiliary equipment control schemes.

[0006] This invention provides a non-contact interactive control method based on multi-sensor fusion. The method includes: acquiring the user's inertial sensor data and EEG signal data; determining a valid directional intention based on the inertial sensor data through orientation maintenance determination, and generating a corresponding directional control command; determining the user's attention state based on the EEG signal data, and generating an access control signal for restricting movement according to the attention state; identifying at least two different triggering mechanisms for emergency stop based on the EEG signal data, and generating corresponding emergency stop signals; arbitrating the directional control command, the access control signal, and the emergency stop signal according to a preset priority rule to output a final control command; wherein the emergency stop signal has the highest priority, and the access control signal has a higher priority than the directional control command.

[0007] According to the present invention, a non-sensory interactive control method based on multi-sensor fusion is provided. The method determines a valid directional intention based on the inertial sensing data and through direction maintenance determination. The method includes: processing the inertial sensing data to obtain direction information; determining whether the direction information remains within a preset direction range within a continuous determination time window; if so, determining that there is a valid directional intention corresponding to the direction information.

[0008] According to the present invention, a non-sensory interactive control method based on multi-sensor fusion is provided, wherein the length of the determination time window is dynamically adjusted according to the rate of change of the directional information.

[0009] According to the present invention, a non-sensory interactive control method based on multi-sensor fusion is provided, which determines the user's attention state based on the EEG signal data and generates an access control signal for restricting movement according to the attention state. The method includes: calculating a real-time attention index based on the EEG signal data; when the real-time attention index is lower than the attention threshold, determining the attention state as insufficient attention and generating the access control signal for restricting movement.

[0010] According to the present invention, a non-sensory interactive control method based on multi-sensor fusion is provided, wherein the attention threshold is set based on the individual user's baseline value and dynamically adjusted according to the device's operating status or environmental information.

[0011] According to the present invention, a non-sensor-based interactive control method is provided, wherein the generation of the permission control signal for restricting movement includes: when the real-time attention index is lower than the attention threshold, generating a first permission control signal for controlling the device to decelerate; and if the real-time attention index continues to be lower than the attention threshold for a first preset duration, generating a second permission control signal for controlling the device to pause.

[0012] According to the present invention, a non-sensory interactive control method based on multi-sensor fusion is provided, which identifies at least two different triggering mechanisms for emergency stop based on the EEG signal data and generates corresponding emergency stop signals, including: identifying a first emergency stop intention based on a first triggering mechanism corresponding to the user's involuntary physiological response and generating a first emergency stop signal; identifying a second emergency stop intention based on a second triggering mechanism corresponding to the user's voluntary physiological operation and generating a second emergency stop signal; wherein the first emergency stop signal has a higher priority than the second emergency stop signal.

[0013] According to the present invention, a non-sensory interactive control method based on multi-sensor fusion is provided, which identifies a first emergency stop intention based on a first trigger mechanism corresponding to a user's involuntary physiological response. The method includes: extracting high-frequency energy features, temporal amplitude change features, and duration features from the EEG signal data of at least two preset EEG channels; when the energy features, the temporal amplitude change features, and the duration features simultaneously satisfy preset teeth-grinding action recognition rules, the first emergency stop intention is determined to be identified.

[0014] According to the present invention, a non-sensory interactive control method based on multi-sensor fusion is provided, wherein the second emergency stop intention is identified based on the second triggering mechanism corresponding to the user's autonomous physiological operation, including: identifying a single blink event that conforms to the preset blink waveform characteristics from the EEG signal data; and determining that the second emergency stop intention is identified when two consecutive single blink events are identified within a preset time interval.

[0015] This invention also provides a contactless interactive control system based on multi-sensor fusion. The system includes: a data acquisition module for acquiring the user's inertial sensing data and EEG signal data; a directional intention determination module for determining a valid directional intention based on the inertial sensing data and through direction maintenance determination, and generating a corresponding directional control command; a focus gating module for determining the user's focus state based on the EEG signal data, and generating an access control signal to restrict movement according to the focus state; an emergency stop recognition module for recognizing at least two different triggering mechanisms of emergency stop intentions based on the EEG signal data, and generating a corresponding emergency stop signal; and a safety arbitration and control module for arbitrating the directional control command, access control signal, and emergency stop signal according to preset priority rules to output a final control command; wherein the emergency stop signal has the highest priority, and the access control signal has a higher priority than the directional control command.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the multi-sensor fusion-based contactless interactive control method described above.

[0017] The present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the multi-sensor fusion-based contactless interactive control method described above.

[0018] The present invention provides a non-contact interactive control method and system based on multi-sensor fusion. By simultaneously acquiring the user's inertial sensor data and EEG signal data, a unified multimodal information foundation is established for subsequent intention recognition and safety monitoring. Based on this, on the one hand, orientation maintenance is determined based on inertial sensor data, and directional control commands are generated only when the determination is valid. On the other hand, the user's attention state is determined in parallel based on EEG signal data, and at least two emergency stop intentions are identified. This links the generation of control commands with the user's safety status in real time. When attention is insufficient, the access control signal actively restricts movement, and once any emergency stop intention is detected, the emergency stop signal immediately interrupts regular control. Furthermore, the directional control command, access control signal, and emergency stop signal are arbitrated according to preset priority rules, with the emergency stop signal having the highest priority and the access control signal having a higher priority than the directional control command. This forms a rigid decision chain of "emergency stop takes precedence over access control, and access control takes precedence over directional control," ensuring that the most dangerous event receives the first response in any situation. This integrates anti-jitter control, active safety gating, redundant emergency stop mechanism, and hierarchical arbitration logic into a closed-loop system, achieving highly safe and interference-resistant seamless interactive control. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the contactless interactive control method based on multi-sensor fusion provided by the present invention. Figure 2 This is a schematic diagram of the structure of the contactless interactive control system based on multi-sensor fusion provided by the present invention; Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] The following is combined Figure 1This invention describes a lifecycle management method for a personalized intent recognition model. For consistency, the entity executing this method will be uniformly named "System," and will not be described further thereafter.

[0023] Figure 1 This is a schematic diagram of the personalized intent recognition model lifecycle management method provided in an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following: S101. Acquire the user's inertial sensor data and EEG signal data.

[0024] In some embodiments, the target user's head inertial sensing data and multi-channel EEG signal data are simultaneously acquired by a multimodal acquisition unit.

[0025] Inertial sensing data is used to detect changes in the user's head posture, while electroencephalogram (EEG) signal data is used to monitor the user's neurophysiological state. The two types of data are synchronized and aligned using a unified timestamp, providing a data foundation for subsequent multimodal fusion processing.

[0026] For example, inertial sensing data can be acquired through a six-axis inertial measurement unit (IMU), which includes a three-axis gyroscope and a three-axis accelerometer, and is installed at the headrest of the user's head-mounted device or mobile assistive device; EEG signal data is acquired through multi-channel EEG electrodes covering the user's forehead and central area, with acquisition channels including Fp1, Fp2, Fz, and Cz.

[0027] Specifically, the IMU sampling rate can be set to 100Hz to collect angular velocity and acceleration data related to the user's head pitch, roll, and yaw; the EEG electrode sampling rate can be set to 250Hz to collect raw EEG signals that can reflect the user's concentration and physiological movement characteristics.

[0028] S102. Based on inertial sensing data, determine the valid directional intent through direction-keeping determination, and generate corresponding directional control commands.

[0029] In some embodiments, inertial sensing data can be processed to obtain orientation information.

[0030] For example, the original inertial sensing data is filtered and attitude calculation is performed to remove high-frequency noise caused by head micro-shaking, so as to obtain stable head attitude direction information.

[0031] Specifically, a second-order Butterworth low-pass filter is used to filter the original gyroscope signal, with the cutoff frequency set to 5Hz to filter out high-frequency jitter noise and obtain smoothed angular velocity data. The filtered angular velocity is then integrated to obtain the head attitude Euler angles, which include pitch, roll, and yaw angles. The Euler angles are then mapped to a three-dimensional direction vector as direction information.

[0032] Furthermore, it is determined whether the direction information remains within a preset direction range within a continuous determination time window; if so, it is determined that there is a valid directional intention corresponding to the direction information.

[0033] In this embodiment of the invention, a preset directional information angle tolerance range can be used as a reference for determining whether the head is in the same direction.

[0034] Specifically, the angular tolerance for the same direction is defined as ±5°. When the three-dimensional direction vector always falls within this angular tolerance range within a continuous time window, it is determined to be a valid direction maintenance, that is, there is a valid direction intention.

[0035] Optionally, in this embodiment of the invention, the length of the determination time window is dynamically adjusted according to the rate of change of the direction information.

[0036] Specifically, the rate of change of direction information is the angular velocity of the user's head rotation. The direction holding time threshold is dynamically adjusted according to the magnitude of the head rotation angular velocity: when the head rotation angular velocity is greater than the preset angular velocity threshold, the judgment time threshold is shortened; when the head rotation angular velocity is less than the preset angular velocity threshold, the judgment time threshold is shortened; when the head rotation angular velocity is in the middle range, the basic time threshold is used.

[0037] For example, when the head rotation angular velocity is greater than 30° / second, the judgment time threshold is shortened to 0.5 seconds to adapt to large-angle rapid rotation scenarios; when the head rotation angular velocity is less than 10° / second, the judgment time threshold is extended to 1.5 seconds to adapt to fine-tuning scenarios; when the head rotation angular velocity is in the middle range, a basic time threshold of 1 second is used.

[0038] Thus, this invention achieves anti-shake direction maintenance determination through filtering and noise reduction, attitude calculation, angle tolerance determination, and dynamic time threshold adjustment, effectively reducing the false trigger rate caused by micro-shakes of the head.

[0039] S103. Based on EEG signal data, determine the user's attention state and generate a control signal for restricting movement permissions according to the attention state.

[0040] In some embodiments, a real-time focus index can be calculated based on electroencephalogram (EEG) signal data.

[0041] For example, the EEG signal data is processed by framing and sliding windowing to extract the power features of EEG signals in different frequency bands, and a real-time focus index that reflects the user's attention concentration level is calculated based on a preset formula.

[0042] Specifically, after preprocessing the acquired multi-channel EEG signal data, extract... Wave (4-8Hz) and The concentration index is calculated based on the power characteristics of the wave (13-30Hz) according to the following formula (1): (1).

[0043] in, express Wave power value, express Wave power value; Attention index is updated per second using a sliding window with a window length of 2 seconds and a step size of 0.5 seconds.

[0044] Furthermore, when the real-time focus index is lower than the focus threshold, the focus state is determined to be insufficient, and a control signal for restricting movement permissions is generated.

[0045] In this embodiment of the invention, the focus threshold can be set based on an individual user baseline and dynamically adjusted according to the device operating status or environmental information.

[0046] Specifically, when a user uses the system for the first time, it collects 30 seconds of resting-state EEG data to establish an individual baseline focus value. A fixed percentage of this baseline focus value is used as the basic focus threshold. The basic threshold is then dynamically adjusted based on the device's real-time movement speed, the density of obstacles in the environment, and the current time period.

[0047] For example, the base threshold is set to 70% of the individual baseline value; when the device's movement speed increases or the density of environmental obstacles increases, the focus threshold is increased; when it is nighttime or afternoon, when users are more likely to be drowsy, the focus threshold is decreased to increase system sensitivity.

[0048] In one alternative implementation, when the real-time attention index is below the attention threshold, a first-authority control signal for controlling the device to decelerate is generated.

[0049] For example, when the real-time focus index is lower than the focus threshold and the duration is less than the first preset duration, a first permission control signal is generated to control the device to enter the "insufficient focus - deceleration" state and reduce the device's operating speed to a preset ratio of normal speed.

[0050] Specifically, when the focus level is below the focus threshold for 2 seconds, a first permission control signal is generated, reducing the device's movement speed to 50% of the normal speed, and issuing a safety warning to remind the user to concentrate.

[0051] In another alternative implementation, if the real-time focus index remains below the focus threshold for a first preset duration, a second permission control signal is generated to control the device to pause.

[0052] For example, when the real-time focus index is below the focus threshold for a duration exceeding a first preset duration, a second permission control signal is generated to control the device to enter the "insufficient focus - pause" state, completely stopping the device's movement and waiting for user confirmation before resuming.

[0053] Specifically, when the user's focus level is below the focus threshold for more than 5 seconds, a second permission control signal is generated, which completely stops the device's movement. The device can only resume normal operation after the user's focus level returns to above the threshold and a confirmation operation is completed.

[0054] Thus, this invention achieves an active safety gating mechanism through individual benchmark calibration of focus calculation, dynamic threshold adjustment, and hierarchical permission control. When the user's attention is not focused, or when they are drowsy or distracted, the device's movement is automatically restricted in a hierarchical manner, thereby avoiding safety hazards at the source.

[0055] S104. Based on EEG signal data, identify at least two different triggering mechanisms for emergency stop intentions and generate emergency stop signals.

[0056] In one alternative implementation, the first emergency stop intention can be identified based on the user's involuntary physiological response corresponding to the first triggering mechanism, and a first emergency stop signal can be generated.

[0057] In some embodiments, high-frequency energy features, temporal amplitude abrupt change features, and duration features can be extracted from EEG signal data from at least two preset EEG channels.

[0058] For example, an EEG acquisition channel sensitive to chewing muscle activity is selected, and the energy features, temporal amplitude change features, and duration features of the preset high-frequency gamma band signal are extracted. The user's subconscious teeth-grinding action is recognized based on pure rule logic without the need for machine learning model training.

[0059] Specifically, the Cz and FCz two central EEG channels were selected, and the energy features of the 30-45Hz γ band signal were extracted. The energy of this band will increase sharply when the teeth clenching action occurs. At the same time, the temporal amplitude change features and duration features of the signal were extracted for subsequent action recognition.

[0060] Furthermore, when the energy characteristics, temporal amplitude abrupt change characteristics, and duration characteristics simultaneously meet the preset teeth-gritting action recognition rules, the first emergency stop intention is determined to be recognized.

[0061] For example, when multiple features simultaneously meet the preset triggering conditions and pass multi-channel verification and anti-accidental touch verification, it is determined to be a valid teeth-gripping emergency stop action, and a first emergency stop signal is generated.

[0062] Specifically, when the signal amplitude rises by more than 50μV within 200ms, the signal duration is ≥150ms and ≤500ms, and at least two channels simultaneously meet the above conditions, and the Cz channel has the highest energy proportion, it is determined to meet the teeth-gripping action recognition rules; at the same time, the target action is distinguished from the interference action by waveform characteristics: blinking is a single spike with a width of 80-120ms and an amplitude of 100-150μV, while teeth gritting is a continuous energy burst; chewing is a periodic fluctuation of 1-2Hz, while teeth gritting is a single continuous burst, thereby eliminating false triggers.

[0063] Thus, this invention uses pure rule-based multi-feature fusion to recognize teeth-gritting actions, covering subconscious reactions in emergency scenarios. It requires no machine learning, has a fast response speed, and is highly interpretable. At the same time, it ensures recognition reliability through multi-dimensional anti-accidental touch rules.

[0064] In another alternative implementation, a second emergency stop intention can be identified based on a second triggering mechanism corresponding to the user's autonomous physiological operation, and a second emergency stop signal can be generated.

[0065] In some embodiments, single blink events conforming to preset blink waveform characteristics can be identified from electroencephalogram (EEG) signal data.

[0066] For example, blink spike waveform features are extracted from EEG signals, and single blink events that meet preset amplitude and width requirements are selected for subsequent double blink recognition.

[0067] Specifically, spike-shaped blink waveforms were identified from EEG signals, and waveforms with an amplitude ≥100μV and a width of 80-120ms were selected and marked as single blink events.

[0068] Furthermore, when two consecutive single blink events are detected within a preset time interval, a second emergency stop intention is determined.

[0069] For example, when two consecutive single blink events that meet the characteristics are detected, and the time interval between the two blinks falls within a preset range, and after passing the anti-accidental touch verification, it is determined to be a valid double blink emergency stop action, and a second emergency stop signal is generated.

[0070] Specifically, when the time interval between two consecutive blinks is within the range of 200-500ms, it is determined that the double blink trigger condition is met; at the same time, the anti-accidental touch rule is set: three consecutive single blinks will not trigger, and if a double blink is detected again within 1 second after a double blink, it is considered an accidental touch and ignored.

[0071] Thus, this invention, by recognizing the waveform shape and interval double blink rule, covers the user's conscious emergency stop operation and forms a dual-channel redundancy with the subconscious teeth-gritting emergency stop, greatly improving the reliability of emergency braking.

[0072] It should be noted that in this invention, the first emergency stop signal has a higher priority than the second emergency stop signal. Upon triggering either emergency stop signal, the system immediately enters an emergency stop lockout state, ceasing all motion output.

[0073] In this embodiment of the invention, the emergency stop lock state needs to be released by a manual reset operation, which includes pressing a physical button or confirming through a terminal application.

[0074] Specifically, the emergency stop signal is implemented using a hardware interrupt method to ensure that the system response delay is lower than a preset value, with a response delay of ≤80ms from the occurrence of the emergency stop action to the output of the emergency stop command.

[0075] S105. Arbitrate the direction control command, authority control signal and emergency stop signal according to the preset priority rules to output the final control command.

[0076] Among them, the emergency stop signal has the highest priority, and the access control signal has a higher priority than the direction control command.

[0077] In some embodiments, the three types of signals are sequentially arbitrated according to a fixed control cycle, based on a preset hierarchical security state machine and fixed priority arbitration rules, and finally output compliant device control commands.

[0078] For example, the control cycle can be set to 50ms. Within each cycle, arbitration is strictly performed in the order of "emergency stop signal → access control signal → direction control command". The priority rule is: emergency stop signal > access control signal > direction control command. The arbitration logic is implemented through a hierarchical safety state machine.

[0079] Specifically, the hierarchical safety state machine defines five operating states: idle state, direction control state, inattentive-deceleration state, inattentive-pause state, and emergency stop lockout state. The entry conditions, exit conditions, and priorities of each state are shown in Table 1 below: Table 1 state describe Entry conditions Exit conditions Priority idle The system is in standby mode and there is no motion output. System startup / reset complete The focus level and direction retention criteria were met. - Direction control Normal movement, outputting directional commands based on head movements. Attention level ≥ threshold + direction retention pass the test Attention level < threshold / emergency stop trigger / idle timeout 3 Lack of focus - slowdown Speed ​​drops to 50% of normal, issuing a warning. Focus duration < threshold for 2 seconds Attention recovery ≥ threshold / emergency stop trigger 2 Lack of focus - pause The exercise has completely stopped; user confirmation is required to resume. Focus level < threshold for 5 seconds Attention recovery ≥ threshold + user confirmation 2 Emergency stop lock System locked, all movement stopped. Teeth clenching / double blinking trigger Manual reset 1 In actual control operations: within each control cycle, the arbitration process is as follows: 1) Check the emergency stop signal: If present, immediately output a stop command and enter the emergency stop lockout state.

[0080] 2) If there is no emergency stop, check the focus gating: if the focus level is less than the threshold, enter the deceleration or pause state according to the duration.

[0081] 3) If the focus level meets the standard, check the direction maintenance judgment: if it passes, output the corresponding direction command; if it fails, maintain the current state.

[0082] The emergency stop signal is implemented through hardware interrupts (such as general purpose input / output (GPIO)) to ensure a response latency of <10ms. The inattentive state is implemented through software arbitration, which polls every 100ms.

[0083] In the multi-sensor fusion-based contactless interactive control method provided by this invention, a unified multimodal information foundation is established for subsequent intention recognition and safety monitoring through user inertial sensor data and EEG signal data. Based on this, on the one hand, orientation maintenance is determined based on inertial sensor data, and directional control commands are generated only when the determination is valid; on the other hand, the attention state is determined in parallel based on EEG signal data, and at least two emergency stop intentions are identified. This links the generation of control commands with the user's safety status in real time. When attention is insufficient, the access control signal actively restricts movement, and once any emergency stop intention is detected, the emergency stop signal immediately interrupts regular control. Furthermore, the directional control command, access control signal, and emergency stop signal are arbitrated according to preset priority rules, with the emergency stop signal having the highest priority and the access control signal having a higher priority than the directional control command. This forms a rigid decision chain of "emergency stop takes precedence over access control, and access control takes precedence over directional control," ensuring that the most dangerous event receives the first response in any situation. Thus, anti-jitter control, active safety gating, redundant emergency stop mechanisms, and hierarchical arbitration logic are integrated into a closed-loop system, achieving highly secure and interference-resistant contactless interactive control.

[0084] The following describes the contactless interactive control system based on multi-sensor fusion provided by the present invention. The contactless interactive control system based on multi-sensor fusion described below can be referred to in correspondence with the contactless interactive control method based on multi-sensor fusion described above.

[0085] Figure 2 This is a structural diagram of a contactless interactive control system based on multi-sensor fusion provided in an embodiment of the present invention. The contactless interactive control system based on multi-sensor fusion includes: a data acquisition module 201, a direction / intention determination module 202, a focus gating module 203, an emergency stop recognition module 204, and a safety arbitration and control module 205.

[0086] The system includes: a data acquisition module 201 for acquiring the user's inertial sensing data and EEG signal data; a directional intention determination module 202 for determining valid directional intentions based on inertial sensing data and directional maintenance determination, and generating corresponding directional control commands; a focus gating module 203 for determining the user's focus state based on EEG signal data, and generating permission control signals to restrict movement according to the focus state; an emergency stop recognition module 204 for recognizing at least two different emergency stop intentions based on EEG signal data, and generating corresponding emergency stop signals; and a safety arbitration and control module 205 for arbitrating the directional control commands, permission control signals, and emergency stop signals according to preset priority rules to output the final control command; wherein the emergency stop signal has the highest priority, and the permission control signal has a higher priority than the directional control command.

[0087] In some embodiments, the directional intent determination module 202 is specifically used to: process the inertial sensing data to obtain directional information; determine whether the directional information remains within a preset directional range within a continuous determination time window; if so, determine that there is a valid directional intent corresponding to the directional information.

[0088] In some embodiments, the attention gating module 203 is specifically used to: calculate a real-time attention index based on the EEG signal data; when the real-time attention index is lower than the attention threshold, determine that the attention state is insufficient, and generate the permission control signal for restricting movement.

[0089] In some embodiments, the attention gating module 203 is specifically used to: generate a first permission control signal for controlling the device to decelerate when the real-time attention index is lower than the attention threshold; and generate a second permission control signal for controlling the device to pause if the real-time attention index continues to be lower than the attention threshold for a first preset duration.

[0090] In some embodiments, the above-mentioned emergency stop recognition module 204 is specifically used to: recognize a first emergency stop intention based on a first triggering mechanism corresponding to the user's involuntary physiological reaction and generate a first emergency stop signal; recognize a second emergency stop intention based on a second triggering mechanism corresponding to the user's voluntary physiological operation and generate a second emergency stop signal; wherein the first emergency stop signal has a higher priority than the second emergency stop signal.

[0091] In some embodiments, the above-mentioned emergency stop recognition module 204 is specifically used to: extract high-frequency energy features, temporal amplitude change features and duration features from the EEG signal data of at least two preset EEG channels; when the energy features, the temporal amplitude change features and the duration features simultaneously satisfy the preset teeth-grinding action recognition rules, the first emergency stop intention is determined to be recognized.

[0092] In some embodiments, the above-mentioned emergency stop recognition module 204 is specifically used to: recognize a single blink event that conforms to a preset blink waveform feature from the EEG signal data; when two consecutive single blink events are recognized within a preset time interval, it is determined that the second emergency stop intention has been recognized.

[0093] In the multi-sensor fusion-based contactless interactive control system provided by this invention, a unified multimodal information foundation is established for subsequent intention recognition and safety monitoring through user inertial sensor data and EEG signal data. Based on this, on the one hand, orientation maintenance is determined based on inertial sensor data, and directional control commands are generated only when the determination is valid; on the other hand, the attention state is determined in parallel based on EEG signal data, and at least two emergency stop intentions are identified. This allows for real-time correlation between the generation of control commands and the user's safety status. When attention is insufficient, the access control signal actively restricts movement, and once any emergency stop intention is detected, the emergency stop signal immediately interrupts regular control. Furthermore, the directional control command, access control signal, and emergency stop signal are arbitrated according to preset priority rules, with the emergency stop signal having the highest priority and the access control signal having a higher priority than the directional control command. This forms a rigid decision chain of "emergency stop takes precedence over access control, and access control takes precedence over directional control," ensuring that the most dangerous event receives the first response in any situation. Thus, anti-jitter control, active safety gating, redundant emergency stop mechanisms, and hierarchical arbitration logic are integrated into a closed-loop system, achieving highly secure and interference-resistant contactless interactive control.

[0094] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communications bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other through the communications bus 340. The processor 310 can call logic instructions in the memory 330 to execute a non-sensory interaction control method based on multi-sensor fusion. The method includes: acquiring the user's inertial sensing data and EEG signal data; determining a valid directional intention based on the inertial sensing data through orientation maintenance determination, and generating a corresponding directional control command; determining the user's attention state based on the EEG signal data, and generating an access control signal for restricting movement according to the attention state; identifying at least two different triggering mechanisms for emergency stop based on the EEG signal data, and generating a corresponding emergency stop signal; arbitrating the directional control command, the access control signal, and the emergency stop signal according to a preset priority rule to output a final control command; wherein the emergency stop signal has the highest priority, and the access control signal has a higher priority than the directional control command.

[0095] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the non-sensory interaction control method based on multi-sensor fusion provided by the above methods, the method comprising: acquiring the user's inertial sensing data and EEG signal data; determining a valid directional intention based on the inertial sensing data by direction-keeping determination, and generating a corresponding directional control instruction; determining the user's attention state based on the EEG signal data, and generating an access control signal for restricting movement according to the attention state; identifying at least two different triggering mechanisms of emergency stop intention based on the EEG signal data, and generating a corresponding emergency stop signal; arbitrating the directional control instruction, the access control signal, and the emergency stop signal according to a preset priority rule, to output a final control instruction; wherein the emergency stop signal has the highest priority, and the access control signal has a higher priority than the directional control instruction.

[0097] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the aforementioned contactless interactive control methods based on multi-sensor fusion. The method includes: acquiring the user's inertial sensor data and EEG signal data; determining a valid directional intention based on the inertial sensor data through orientation maintenance determination, and generating a corresponding directional control command; determining the user's attention state based on the EEG signal data, and generating an access control signal for restricting movement according to the attention state; identifying at least two different triggering mechanisms for emergency stop intention based on the EEG signal data, and generating a corresponding emergency stop signal; arbitrating the directional control command, the access control signal, and the emergency stop signal according to a preset priority rule to output a final control command; wherein the emergency stop signal has the highest priority, and the access control signal has a higher priority than the directional control command.

[0098] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A seamless interactive control method based on multi-sensor fusion, characterized in that, The method includes: Acquire the user's inertial sensor data and electroencephalogram (EEG) signal data; Based on the inertial sensing data, the effective directional intention is determined through orientation hold determination, and the corresponding directional control command is generated. Based on the EEG signal data, the user's attention state is determined, and an access control signal is generated to restrict movement according to the attention state. Based on the EEG signal data, identify at least two different triggering mechanisms for emergency stop intentions and generate corresponding emergency stop signals; According to a preset priority rule, the direction control command, the access control signal, and the emergency stop signal are arbitrated to output a final control command; wherein, the emergency stop signal has the highest priority, and the access control signal has a higher priority than the direction control command.

2. The contactless interactive control method based on multi-sensor fusion according to claim 1, characterized in that, The determination of a valid directional intent based on the inertial sensing data through orientation maintenance judgment includes: The inertial sensing data is processed to obtain orientation information; Determine whether the direction information remains within a preset direction range within a continuous determination time window; If so, it is determined that there is a valid directional intention corresponding to the directional information.

3. The contactless interactive control method based on multi-sensor fusion according to claim 2, characterized in that, The length of the determination time window is dynamically adjusted according to the rate of change of the direction information.

4. The contactless interactive control method based on multi-sensor fusion according to claim 1, characterized in that, The process of determining the user's attention state based on the EEG signal data and generating an access control signal for restricting movement according to the attention state includes: Real-time focus index is calculated based on the aforementioned EEG signal data; When the real-time focus index is lower than the focus threshold, the focus state is determined to be insufficient, and the permission control signal for restricting movement is generated.

5. The contactless interactive control method based on multi-sensor fusion according to claim 4, characterized in that, The focus threshold is set based on individual user baseline values ​​and is dynamically adjusted according to device operating status or environmental information.

6. The contactless interactive control method based on multi-sensor fusion according to claim 4, characterized in that, The generation of the access control signal for restricting movement includes: When the real-time focus index is lower than the focus threshold, a first permission control signal is generated to control the device to slow down. If the real-time focus index remains below the focus threshold for a first preset duration, a second permission control signal is generated to control the device to pause.

7. The non-sensored interactive control method based on multi-sensor fusion according to any one of claims 1 to 6, characterized in that, The step of identifying at least two different triggering mechanisms for emergency stop based on the EEG signal data and generating corresponding emergency stop signals includes: The first emergency stop intention is identified based on the first triggering mechanism corresponding to the user's involuntary physiological response, and a first emergency stop signal is generated. The second emergency stop intention is identified based on the second triggering mechanism corresponding to the user's autonomous physiological operation, and a second emergency stop signal is generated. The first emergency stop signal has a higher priority than the second emergency stop signal.

8. The contactless interactive control method based on multi-sensor fusion according to claim 7, characterized in that, The first triggering mechanism based on the user's involuntary physiological response to identify the first emergency stop intention includes: Extract high-frequency energy characteristics, temporal amplitude abrupt change characteristics, and duration characteristics from the EEG signal data of at least two preset EEG channels; When the energy characteristic, the temporal amplitude change characteristic, and the duration characteristic simultaneously satisfy the preset teeth-gritting action recognition rules, the first emergency stop intention is determined to be recognized.

9. The contactless interactive control method based on multi-sensor fusion according to claim 7, characterized in that, The second triggering mechanism based on the user's autonomous physiological operation to identify the second emergency stop intention includes: Identify single blink events that conform to preset blink waveform characteristics from the electroencephalogram (EEG) signal data; When two consecutive single blink events are detected within a preset time interval, it is determined that the second emergency stop intention has been detected.

10. A seamless interactive control system based on multi-sensor fusion, characterized in that, The system includes: The data acquisition module is used to acquire the user's inertial sensor data and electroencephalogram (EEG) signal data; The direction intent determination module is used to determine the valid direction intent based on the inertial sensing data through direction holding determination, and generate the corresponding direction control command. The attention gating module is used to determine the user's attention state based on the EEG signal data, and generate an access control signal for restricting movement according to the attention state; An emergency stop recognition module is used to identify emergency stop intentions with at least two different triggering mechanisms based on the EEG signal data, and generate corresponding emergency stop signals; The safety arbitration and control module is used to arbitrate the direction control command, the access control signal, and the emergency stop signal according to preset priority rules, so as to output the final control command; wherein, the emergency stop signal has the highest priority, and the access control signal has a higher priority than the direction control command.

Images