A bluetooth mesh whole-house roaming adaptive remote control system and method based on multi-sensor fusion

Abstract

The application discloses a Bluetooth Mesh whole-house roaming adaptive remote control system and method based on multi-sensor fusion. The application collects the angular velocity, acceleration and spatial inclination angle data of a remote control device in real time through a 6-axis IMU attitude sensing module, accurately identifies whether a user is in a handheld walking roaming state, and when the walking roaming state is identified, an adaptive control module automatically unbinds remote room controlled intelligent devices, binds the nearest intelligent devices in the current room according to a whole-house device space mapping library constructed by a Bluetooth Mesh communication module, and switches touch mapping logic without sensing. When the user handheld remote controller moves between different rooms, no manual operation is needed, the control target automatically follows the current room, and the whole-house roaming and sensing following experience of "walking to where and controlling to where" is realized.

Description

Technical Field

[0001] This invention relates to the technical field of smart homes, specifically to a Bluetooth Mesh whole-house roaming adaptive remote control system and method based on multi-sensor fusion. Background Technology

[0002] In the current field of whole-house smart home, Bluetooth Mesh self-organizing network control technology has been widely used in the linkage control of terminal devices such as smart lights, home appliances, curtains, and air conditioners. Most existing smart remote control terminals use physical buttons to trigger pairing and control. After the devices are paired, the binding relationship is fixed, which only supports fixed-point static control and does not have the ability to adapt to handheld mobile roaming.

[0003] Existing conventional solutions rely solely on signal strength to select the nearest device, failing to consider user movement, device placement, and handheld carrying scenarios for intelligent identification. This results in an inability to differentiate between various usage scenarios, such as walking / roaming, stationary placement, pocket carrying, and close-range operation, leading to poor intelligent adaptation capabilities. For example, they cannot effectively distinguish between handheld walking / roaming and stationary placement, causing cross-room tracking to either fail to trigger or trigger accidentally; or they cannot effectively distinguish between remote control being carried in a pocket / backpack and stationary on a table, causing the anti-accidental touch mechanism to fail, resulting in numerous accidental touch commands due to fabric friction when the user walks; severely impacting the user experience. Summary of the Invention

[0004] This application provides a Bluetooth Mesh whole-house roaming adaptive remote control system and method based on multi-sensor fusion, which solves the problem mentioned in the background art that existing smart remote control terminals do not combine user movement posture, device placement status and handheld carrying scenario to carry out intelligent discrimination, and cannot distinguish different usage scenarios such as walking roaming, stationary placement, pocket carrying, and close-range operation, resulting in poor intelligent adaptation capability.

[0005] To address the aforementioned technical problems, firstly, this application provides a Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion, comprising: A capacitive touch interaction module, connected to a touch panel, is used to collect touch commands from the touch panel, including multi-touch, swipe, long press, and tap. A 6-axis IMU attitude sensing module is used to collect data information from the remote control device in real time and accurately identify the current state of the remote control device; the data information includes the angular velocity, acceleration, and spatial tilt angle data of the remote control device; the current state includes the user's handheld walking roaming state, the device's stationary state, and the fixed-point control state. The proximity sensing module accurately acquires information about the area of ​​the touch panel that is obstructed and the distance between the object and the touch panel; The Mesh Bluetooth communication module is used to connect to the whole-house Bluetooth Mesh network, collect RSSI signals of smart devices in each room in real time, complete spatial ranging, and build a spatial mapping library of whole-house devices. It also includes an adaptive control module, which dynamically unbinds remotely controlled smart devices and automatically binds nearby smart devices in the current room based on the whole-house device space mapping library data, and seamlessly switches the touch mapping logic; at the same time, based on the current status of the remote control device and the information on the panel obstruction area and the distance to the object obtained by the proximity sensing module, it adaptively switches between anti-accidental touch sleep mode, low-power standby mode and fixed-point precise control mode; finally, it responds to user interaction based on the touch commands, realizing a buttonless, all-scenario adaptive intelligent remote control.

[0006] In one embodiment, the capacitive touch interaction module includes: The trajectory filtering unit, with a built-in digital filter, is used to smooth continuous touch points and eliminate jitter and abnormal jump points; The micro-motion interference filtering unit is used to adaptively filter out invalid touch signals caused by hand tremors, table vibrations, electromagnetic interference, etc., to avoid false triggering; The gesture recognition unit performs pattern matching on the original touch sequence to identify specific command types such as multi-touch, swipe trajectory, long press, and tap.

[0007] In one embodiment, the 6-axis IMU attitude sensing module includes: The triaxial accelerometer unit acquires the linear acceleration of the remote-controlled device along the X, Y, and Z directions; The three-axis gyroscope unit acquires the angular velocity of the remote-controlled device around the X, Y, and Z axes; The attitude determination unit calculates the spatial tilt angle by using the values ​​of linear acceleration and angular velocity, and accurately identifies the current state of the remote control device by combining the features of linear acceleration, angular velocity and spatial tilt angle.

[0008] In one embodiment, the attitude determination unit includes: The spatial tilt angle calculation subunit integrates accelerometer and gyroscope data and uses a complementary filtering algorithm to calculate the pitch and roll angles of the remote control device in three-dimensional space. The status recognition subunit receives raw data from the three-axis accelerometer, raw data from the three-axis gyroscope, and spatial tilt angle data output by the spatial tilt angle calculation subunit. Through feature extraction and threshold judgment, it outputs the current status of the remote control device.

[0009] In one embodiment, the proximity sensing module includes: The occlusion area quantization unit calculates the percentage of the area covered by the touch panel through multi-channel sensing or imaging sensing. Distance quantization unit accurately measures the vertical distance between the object and the touch panel; The scene pre-judgment unit performs preliminary combined analysis of occlusion area and distance, and outputs coarse judgment results of suspected occlusion or suspected alignment for reference by the adaptive control module.

[0010] In one embodiment, the Mesh Bluetooth communication module includes: The RSSI real-time acquisition unit periodically scans surrounding Bluetooth Mesh devices and acquires the received signal strength indication of each device. The spatial ranging unit converts RSSI values ​​into spatial distances using a path loss model. The room device cluster analysis unit divides the smart devices in the whole house into device clusters in different rooms based on RSSI strength and spatial distance, and builds and maintains a spatial mapping library.

[0011] In one embodiment, the adaptive control module includes: The multi-sensor fusion decision unit is used to fuse four types of data from different sources, frequencies, and formats: touch commands from the capacitive touch interaction module, data information from the 6-axis IMU attitude perception module, occluded area information and distance information between the object and the touch panel from the proximity sensing module, and smart device RSSI signal and spatial distance from the Mesh Bluetooth communication module. Through timestamp alignment and feature extraction, the data is fused into a unified fusion feature vector, providing a synchronous, reliable, and multi-dimensional input basis for subsequent decisions. The four-scenario intelligent decision model unit, based on the fusion feature vector output by the multi-sensor fusion decision unit, classifies the current composite state of the system into four predefined semantic scenarios, and provides a unified decision basis for subsequent device binding, power consumption control, and touch mapping switching. The dynamic device binding and unbinding unit is used to execute the logic of unbinding remote smart devices and binding the nearest smart devices in the current room, and to maintain the binding status table.

[0012] In one embodiment, the main control adaptive control module further includes an anomaly self-healing fault-tolerant unit. When any unit communication timeout or data anomaly is detected, the unit's soft reset and re-initialization operations are executed sequentially. If three consecutive resets fail, the system is downgraded to the basic remote control mode.

[0013] Secondly, this application also provides a Bluetooth Mesh whole-house roaming adaptive remote control method based on multi-sensor fusion, which is used to implement a Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion, including the following steps: The touch commands on the touch panel are collected through the capacitive touch interaction module. The touch commands include multi-touch, swipe trajectory, long press and tap. The 6-axis IMU attitude sensing module collects angular velocity, acceleration, and spatial tilt angle data of the remote control device in real time, and accurately identifies the current state of the remote control device based on the data. The current state includes user hand-held walking roaming state, device stationary state, and fixed-point control state. The proximity sensing module accurately obtains information about the area of ​​the touch panel that is obscured and the distance between the object and the touch panel. By connecting to the whole-house Bluetooth Mesh network through the Mesh Bluetooth communication module, the RSSI signals of smart devices in each room are collected in real time to complete spatial ranging and build a spatial mapping library of whole-house devices; Based on the data in the whole-house device space mapping library, the controlled smart devices in remote rooms are dynamically unbound, the nearby smart devices in the current room are automatically bound, and the touch mapping logic is switched seamlessly. Based on the current status of the remote control device and the distance information between the panel's obstruction area and the object, the system adaptively switches between anti-accidental touch sleep mode, low-power standby mode, or fixed-point precise control mode. The system responds to user interactions based on the touch commands.

[0014] In one embodiment, the method for accurately identifying the current state of the remote control device based on the data is as follows: When the detected acceleration fluctuation frequency is between the first preset threshold and the second preset threshold and the spatial tilt angle changes continuously beyond the preset angle, it is determined that the user is in a handheld walking roaming state. When the acceleration modulus is detected to be close to the third preset threshold, the angular velocity is close to 0, and this condition persists for more than 30 seconds, the device is determined to be in a stationary state. When the acceleration is stable, the spatial tilt angle is stable in a certain direction and accompanied by a slight angular velocity correction, the system is judged to be in a fixed-point control state.

[0015] The beneficial effects of the aforementioned Bluetooth Mesh whole-house roaming adaptive remote control system and method based on multi-sensor fusion are as follows: 1. This application uses a 6-axis IMU attitude sensing module to collect real-time data on the angular velocity, acceleration, and spatial tilt angle of the remote-controlled device, accurately identifying whether the user is in a handheld walking / roaming state. When walking / roaming is detected, the adaptive control module, based on the whole-house device spatial mapping library built by the Bluetooth Mesh communication module, automatically unbinds the remotely controlled smart device and binds the nearest smart device in the current room, seamlessly switching the touch mapping logic. When the user moves between different rooms with the remote control, no manual operation is required; the controlled target automatically follows the current room, achieving a seamless whole-house roaming experience of "wherever you go, you control wherever you go." 2. This application accurately acquires information about the area of ​​the touch panel being obscured and the distance between the object and the panel through a proximity sensing module. Combined with the device status identified by the 6-axis IMU attitude sensing module: when the panel obscuration area is ≥80%, the object distance is ≤2mm, and the IMU detects walking swaying characteristics, it is determined to be a pocket / backpack carrying scenario, and the adaptive control module automatically switches to an anti-accidental touch sleep mode, disabling touch command response; when the obscuration area is ≤10%, the object distance is ≥5cm, and the IMU detects a stationary state, it switches to a low-power standby mode. This effectively avoids numerous accidental touch commands caused by fabric rubbing against the panel during walking, protecting user privacy and device security. 3. This application uses a proximity sensing module to quantitatively detect the distance between an object and the panel, as well as the changing trend of the occlusion area: when the object distance is detected to gradually decrease from greater than 5cm to less than 2cm, and the occlusion area is a small, localized area, combined with the fixed-point control posture detected by the IMU, it is determined to be an active alignment control scenario, and the adaptive control module triggers precise device pairing; when a rapid and significant change in the occlusion area is detected, or the object distance is unstable, it is determined to be unintentional approach, and the pairing process is not triggered. This effectively avoids accidental pop-up windows interfering with normal use when the user's hand unintentionally brushes over or places objects. 4. This application integrates touch commands from the capacitive touch interaction module, data from the 6-axis IMU attitude sensing module, information on the obstructed area and distance between the object and the touch panel from the proximity sensing module, and the RSSI signal and spatial distance of the smart device from the Mesh Bluetooth communication module through an adaptive control module. It runs a multi-sensor fusion decision algorithm and a four-scene intelligent decision model to comprehensively determine the user's current scene (handheld walking roaming, device stationary placement, stationary carrying, fixed-point operation), and automatically switches device binding, touch mapping, and power consumption mode according to the scene, realizing full-scene adaptive intelligent remote control, significantly improving the system's intelligence level and user experience. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the structure of a Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion, as shown in an embodiment of this application. Figure 2 This is a flowchart illustrating a Bluetooth Mesh whole-house roaming adaptive remote control method based on multi-sensor fusion, as shown in an embodiment of this application. Detailed Implementation

[0017] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0018] It should be noted that when a component is considered to be "connected" to another component, it can be directly connected to the other component or there may be an intermediary component present. Conversely, when a component is said to be "directly" connected to another component, there is no intermediary component.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0020] like Figure 1 As shown, this application provides a Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion, including: The capacitive touch interaction module 1 is connected to the touch panel and is used to collect touch commands from the touch panel, including multi-touch, swipe trajectory, long press and tap. The capacitive touch interaction module 1 includes: The trajectory filtering unit, with a built-in digital filter, is used to smooth continuous touch points and eliminate jitter and abnormal jump points; The micro-motion interference filtering unit is used to adaptively filter out invalid touch signals caused by hand tremors, table vibrations, electromagnetic interference, etc., to avoid false triggering; The gesture recognition unit performs pattern matching on the original touch sequence to identify specific command types such as multi-touch, swipe trajectory, long press, and tap.

[0021] In this way, the capacitive touch interaction module 1 removes invalid signals such as environmental noise and hand tremors through the micro-motion interference filtering unit, smooths the effective touch trajectory through the trajectory filtering unit, and then matches the smoothed touch sequence pattern into standardized commands such as multi-touch, swipe trajectory, long press, and tap through the gesture recognition unit. Finally, it outputs the commands to the adaptive control module for execution, thus achieving a precise, anti-interference, and natural touch interaction experience.

[0022] The 6-axis IMU attitude sensing module 2 is used to collect data information of the remote control device in real time and accurately identify the current state of the remote control device; the data information includes the angular velocity, acceleration, and spatial tilt angle data of the remote control device; the current state includes the user's handheld walking roaming state, the device's stationary state, and the fixed-point control state. The 6-axis IMU attitude sensing module 2 includes: The triaxial accelerometer unit acquires the linear acceleration of the remote-controlled device along the X, Y, and Z directions; The three-axis gyroscope unit acquires the angular velocity of the remote-controlled device around the X, Y, and Z axes; The attitude determination unit calculates the spatial tilt angle by using the values ​​of linear acceleration and angular velocity, and accurately identifies the current state of the remote control device by combining the features of linear acceleration, angular velocity and spatial tilt angle.

[0023] The three-axis accelerometer unit outputs raw acceleration data in three axes, denoted as (Ax_raw, Ay_raw, Az_raw). The three-axis gyroscope unit uses a MEMS vibrating gyroscope and outputs raw angular velocity data in three axes, denoted as (Gx_raw, Gy_raw, Gz_raw).

[0024] The attitude determination unit includes a spatial tilt calculation subunit and a state recognition subunit. The spatial tilt calculation subunit receives the outputs (Ax_raw, Ay_raw, Az_raw) from the three-axis accelerometer and the outputs (Gx_raw, Gy_raw, Gz_raw) from the three-axis gyroscope, and fuses the two data through a complementary filtering algorithm to calculate the pitch angle and roll angle of the remote control device in three-dimensional space.

[0025] First, the accelerometer data is normalized to obtain the components of gravity along the three axes (gx, gy, gz). In a stationary state, the tilt angle can be calculated using only the accelerometer data: Pitch_acc=arctan2(gy,sqrt(gx 2 +gz 2 )) Roll_acc=arctan2(-gx,gz) However, when the remote-controlled device is in motion, the accelerometer data is affected by the motion acceleration, which can cause significant errors in the above calculations. Therefore, this embodiment introduces gyroscope data for compensation.

[0026] The change in angle can be obtained by integrating the physical angular velocity values ​​(Gx, Gy, Gz) output by the gyroscope over time after zero-point offset calibration and sensitivity conversion. This embodiment uses a first-order complementary filtering algorithm for fusion. Pitch=α×(Pitch+Gx×Δt)+(1-α)×Pitch_acc; Roll=α×(Roll+Gy×Δt)+(1-α)×Roll_acc; Where Δt is the sampling interval (5ms in this embodiment, corresponding to a 200Hz sampling rate), and α is the fusion coefficient, with a value of 0.98, which ensures the accuracy of the gyroscope's short-term integration while utilizing the accelerometer's long-term stability to eliminate drift. Through the above fusion calculation, stable spatial tilt data is obtained, denoted as (Pitch, Roll).

[0027] The status recognition subunit receives raw data from the triaxial accelerometer, raw data from the triaxial gyroscope, and spatial tilt angle data output by the spatial tilt angle calculation subunit. Through feature extraction and threshold judgment, it outputs the current status of the remote control device.

[0028] The state recognition subunit first extracts the step frequency feature from the resultant acceleration signal. The resultant acceleration is then calculated. A_mag=sqrt(Ax, high-pass filtering (cutoff frequency 0.5Hz) is applied to the resultant acceleration sequence to remove the DC component of gravity and obtain the AC component A_ac. 2 +Ay 2 +Az 2 ) The peak detection method is used to calculate the step frequency: A local maximum point of A_ac is found within a sliding window. This point must have a value greater than five sampling points before and after it, and its amplitude must exceed a preset threshold, for example, 0.2g in this embodiment. The time interval Δt_i between two adjacent peaks is calculated, and the instantaneous step frequency f_i = 1 / Δt_i. The average step frequency f_avg is obtained by averaging all instantaneous step frequencies within the window.

[0029] Simultaneously, the change in tilt angle is extracted from the spatial tilt angle data. The difference in spatial tilt angle between the start and end times of the calculation window is: ΔPitch = |Pitch_end - Pitch_start|, ΔRoll = |Roll_end - Roll_start|.

[0030] When the average step frequency f_avg is within the range of 0.5Hz to 3Hz, the change in spatial tilt angle ΔPitch or ΔRoll is greater than 15°, and the standard deviation of the resultant acceleration signal is greater than 0.1g, the state recognition subunit determines that the remote control device is in a handheld walking roaming state.

[0031] When the resultant acceleration value A_mag is in the range of 0.98g to 1.02g, the angular velocity magnitude output by the three-axis gyroscope is less than 5° / s, and the change rate of the spatial tilt angle (Pitch, Roll) is less than 0.5° / s, and these three conditions are met simultaneously and last for more than 30 seconds, the remote control device is judged to be in a stationary state.

[0032] Fixed-point control refers to a scenario where a user holds a remote control and precisely controls a target device. When the resultant acceleration value A_mag approaches 1g (between 0.95g and 1.05g), the pitch (roll) angle stabilizes in a certain direction with a rate of change of less than 10° / s, and the three-axis gyroscope exhibits a small angular velocity output (between 5° / s and 50° / s), and this state persists for more than 1 second, the state recognition subunit determines that the remote-controlled device is in fixed-point control mode.

[0033] The 6-axis IMU attitude perception module 2 outputs the decision results in a structured data format for use by the adaptive control module 5.

[0034] The proximity sensing module 3 accurately acquires information about the area of ​​the touch panel that is obstructed and the distance between the object and the touch panel; The proximity sensing module 3 includes: The occlusion area quantization unit calculates the percentage of the area covered by the touch panel through multi-channel sensing or imaging sensing. Distance quantization unit accurately measures the vertical distance between the object and the touch panel; The scene pre-judgment unit performs preliminary combined analysis of occlusion area and distance, and outputs coarse judgment results of suspected occlusion or suspected alignment for reference by the adaptive control module.

[0035] In this embodiment, the proximity sensing module 3 employs a capacitive or infrared optical proximity sensor array, deployed in the non-display area of ​​the touch panel or below the panel. The occlusion area quantization unit is used to calculate the percentage of the touch panel area covered.

[0036] M×N independent capacitive sensing electrodes are deployed below or inside the touch panel to form a grid-like sensing array. In this embodiment, both M and N are 16, resulting in a total of 256 sensing nodes. Each node independently detects the proximity of objects above it.

[0037] When an object (such as a finger, palm, or fabric) approaches or touches the touch panel, a coupling capacitance is formed between the sensing electrode in the corresponding area and the object, causing a change in the capacitance value of that node. The amount of capacitance change ΔC is positively correlated with the area covered by the object.

[0038] The occlusion area quantization unit scans all sensing nodes at a fixed frequency to obtain the capacitance change value of each node. For a node located in the i-th row and the j-th column, its capacitance change is denoted as ΔC_ij.

[0039] First, binarization processing is performed on each node: when ΔC_ij≥Th_c, it indicates that the node is occluded, then S_ij=1; when ΔC_ij<Th_c, it indicates that the node is not occluded, then S_ij=0, wherein Th_c is an occlusion detection threshold.

[0040] Then, the number of occluded nodes is calculated: N_occluded=Σ_{i=1}^{M}Σ_{j=1}^{N}S_ij Finally, the percentage of occlusion area is calculated: Occlusion_Ratio=(N_occluded / (M×N))×100%.

[0041] Capacitive ranging performs distance estimation by using the non-linear relationship between the induced capacitance change and the distance when an object approaches. The capacitance value C between the sensing electrode and the object can be approximately expressed as: C≈ε×A / d, wherein ε is the dielectric constant, A is the opposite area between the electrode and the object provided by the occlusion area quantization unit, and d is the distance between the object and the panel. The distance quantization unit inversely calculates the distance d according to the known occlusion area A and the actually measured capacitance value C: d=ε×A / C. The scene pre-decision unit outputs an occlusion situation determination table for the adaptive control module to refer to, as shown in Table 1:

[0042] Table 1 The Bluetooth Mesh communication module 4 is configured to access the whole-house Bluetooth Mesh network, collect RSSI signals of intelligent devices in each room in real time, complete spatial ranging and construct a spatial mapping library of whole-house devices; The Bluetooth Mesh communication module 4 comprises: an RSSI real-time acquisition unit, configured to periodically scan surrounding Bluetooth Mesh devices and acquire the received signal strength indicator of each device; a spatial ranging unit, configured to convert RSSI values into spatial distances through a path loss model; a room device cluster analysis unit, configured to divide the whole-house intelligent devices into device clusters of different rooms according to RSSI strength and spatial distance, and construct and maintain the spatial mapping library.

[0043] Mesh Bluetooth communication module 4 uses a wireless communication chip that supports the BLE5.4 protocol and Bluetooth Mesh self-organizing network protocol. Mesh Bluetooth communication module 4 is connected to adaptive control module 5 through SPI or UART interface and is responsible for accessing the whole house Bluetooth Mesh network, device discovery, RSSI acquisition, spatial ranging, and the construction and maintenance of spatial mapping library.

[0044] The RSSI real-time acquisition unit is responsible for periodically scanning the surrounding Bluetooth Mesh devices and acquiring the Received Signal Strength Indicator (RSSI) of each device, providing basic data for subsequent spatial ranging and room identification.

[0045] The spatial ranging unit receives the RSSI value output by the RSSI real-time acquisition unit and converts it into an estimated spatial distance between the remote control device and each smart device through the path loss model.

[0046] This embodiment uses a logarithmic distance path loss model, which describes the logarithmic relationship between signal strength and distance: RSSI(d) = RSSI(d). RSSI(d) is the received signal strength at distance d; RSSI(d) is the signal strength at a reference distance d; d is the spatial distance to be estimated; d is the reference distance; n is the path loss exponent. The above model is transformed into the distance calculation formula: d = d. In this embodiment, the parameters are set as follows: d meters, RSSI(d), n = 2.5. Substituting into the formula, we get: d = 10^((-50-RSSI) / 25).0) - 10 × n × log 10 (d / d0)+X_σ0) 000 ×10^((RSSI(d0)-RSSI(d)) / (10×n))0=1.00)=-50dBm In this system, the room device cluster analysis unit uses the K-Means clustering algorithm to automatically divide all smart devices in the house into device clusters in different rooms based on RSSI strength and spatial distance. Specifically, the system first acquires the received signal strength values ​​of the remote control to each smart device through the real-time RSSI acquisition unit, and then converts them into estimated spatial distances by the spatial ranging unit. Each device is represented as a feature vector composed of RSSI strength, spatial distance, and signal stability. Subsequently, the algorithm presets the number of rooms K (i.e., the number of clusters), randomly selects K devices as initial cluster centers, calculates the weighted Euclidean distance from each device to each cluster center, and assigns the device to the room represented by the nearest cluster center. Then, it recalculates the feature mean of all devices in each room as the new cluster center, and repeats the "assignment-update" two steps until the cluster centers converge or the maximum number of iterations is reached. After clustering is completed, each device is assigned a room label, forming an initial spatial mapping library.

[0047] Each device is represented by the following feature vector: Feature_vector=[RSSI_device,distance_device,time_stability]; where: RSSI_device: the currently measured signal strength; distance_device: the estimated spatial distance; time_stability: the historical variance of RSSI, used to distinguish between fixed devices and mobile devices; fixed devices (such as lamps, sockets, and air conditioners) are included in the spatial mapping library; mobile devices (such as user mobile phones) do not participate in cluster analysis.

[0048] When it is necessary to identify the room where the remote control device is currently located, the room device cluster analysis unit first obtains the real-time RSSI measurement values ​​of each fixed device from the remote control. Then, for each room cluster, it uses weighted Euclidean distance to calculate the similarity between the current RSSI vector and the RSSI baseline vector of the devices in that room. The calculation formula is: Score_room=Σw_i×(RSSI_current_i-RSSI_baseline_i) 2 RSSI_current_i: Current RSSI vector; RSSI_baseline_i: Room device RSSI baseline vector; w_i: Room device weight, the weight is related to the device type, with fixed devices such as lamps having a higher weight than movable devices. The room with the lowest similarity matching score is the current room. In this way, the K-Means clustering algorithm can automatically divide all the devices in the house into device clusters in different rooms without manual configuration, which greatly reduces the user's installation and deployment costs.

[0049] And adaptive control module 5, which is used to dynamically unbind remote room controlled smart devices and automatically bind nearby smart devices in the current room based on the whole house device space mapping library data, and seamlessly switch touch mapping logic; at the same time, based on the current status of the remote control device and the panel occlusion area and object distance information obtained by the proximity sensing module, it adaptively switches between anti-accidental touch sleep mode, low power standby mode and fixed-point precise control mode; finally, it responds to user interaction based on the touch command, realizing full-scene adaptive intelligent remote control without physical buttons.

[0050] Adaptive control module 5 includes: The multi-sensor fusion decision unit is used to fuse four types of data from different sources, frequencies, and formats: touch commands from the capacitive touch interaction module, data information from the 6-axis IMU attitude perception module, occluded area information and distance information between the object and the touch panel from the proximity sensing module, and smart device RSSI signal and spatial distance from the Mesh Bluetooth communication module. Through timestamp alignment and feature extraction, the data is fused into a unified fusion feature vector, providing a synchronous, reliable, and multi-dimensional input basis for subsequent decisions. The four-scenario intelligent decision model unit, based on the fusion feature vector output by the multi-sensor fusion decision unit, classifies the current composite state of the system into four predefined semantic scenarios, and provides a unified decision basis for subsequent device binding, power consumption control, and touch mapping switching. The dynamic device binding and unbinding unit is used to execute the logic of unbinding remote smart devices and binding the nearest smart devices in the current room, and to maintain the binding status table.

[0051] The adaptive control module 5 uses a low-power 32-bit embedded MCU as its core processor, and communicates with the I / O controller via I / O. 2 The C, SPI, and UART interfaces are connected to the capacitive touch interaction module 1, the 6-axis IMU attitude sensing module 2, the proximity sensing module 3, and the Mesh Bluetooth communication module 4, respectively, serving as the central decision-making and control unit of the entire system.

[0052] The four predefined semantic scenarios are: handheld walking roaming, where the user walks between different rooms holding the remote control; device stationary placement, where the remote control is placed on a table or TV cabinet and not in use; stationary carrying, where the remote control is placed in a pocket or backpack and the user is stationary; and fixed-point control, where the user holds the remote control and performs precise control on the device. The 6-axis IMU attitude sensing module 2 cannot distinguish between stationary placement and stationary carrying, so the adaptive control module 5 needs to combine information about the area of ​​the touch panel being obscured and the distance between the object and the touch panel to make the judgment.

[0053] 1. When the step frequency is in the range of 0.5Hz to 3Hz, the rate of change of spatial tilt angle is greater than 15° / second, the occlusion level is unobstructed or partially obstructed, the standard deviation of acceleration value is greater than 0.1g, and the above state lasts for more than 1 second, the remote control device is a handheld walking roaming device. 2. The acceleration value is in the range of 0.98g to 1.02g, the angular velocity modulus is less than 5° / second, the occlusion level is unobstructed or partially obstructed, the spatial tilt angle (Pitch, Roll) is stable with a change rate of less than 0.5° / second, the tilt angle is at a typical placement angle, such as near a horizontal table or a vertical wall, and the above state lasts for more than 30 seconds, that is, the remote control device is placed still. 3. The acceleration value is in the range of 0.98g to 1.02g, the angular velocity modulus is less than 5° / second, the occlusion level is complete occlusion, the spatial tilt angle is tilt angle, and the above state lasts for more than 5 seconds, that is, the remote control device is a stationary carrying scenario; 4. When the acceleration value is in the range of 0.95g to 1.05g, the spatial tilt angle is stable in a certain direction with a rate of change of less than 10° / second, the angular velocity modulus is in the range of 5° / second to 50° / second, and the above state lasts for more than 1 second, the remote control device is a fixed-point control device.

[0054] The binding and unbinding execution process is as follows: Retrieve the current room ID and the list of devices in the room from the space mapping library of the Mesh Bluetooth communication module; Sort the devices in the current room by estimated distance, and select the closest device that is online and in good working order as the target binding device; Send an unbinding notification to the original bound remote device and clear the original binding record from the binding status table; Send a binding request to the target device, wait for confirmation, update the binding status table, and mark the target device as the current binding.

[0055] In one embodiment, the main control adaptive control module 5 further includes an anomaly self-healing fault-tolerant unit. When any unit communication timeout or data anomaly is detected, the soft reset and re-initialization operations of the unit are executed in sequence. If the reset fails three times in a row, the system is downgraded to the basic remote control mode.

[0056] The basic remote control mode only retains basic touch command responses and controls only the last bound device, without automatic switching. This three-level recovery strategy—soft reset, re-initialization, and system degradation—ensures the system's stable availability in complex environments and effectively addresses fault scenarios such as sensor malfunctions and communication interruptions.

[0057] like Figure 2 As shown, in a second aspect, this application also provides a Bluetooth Mesh whole-house roaming adaptive remote control method based on multi-sensor fusion, a method for implementing a Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion, comprising the following steps: S1. Collect touch commands on the touch panel through the capacitive touch interaction module. The touch commands include multi-touch, swipe trajectory, long press and tap. S2. The angular velocity, acceleration, and spatial tilt angle data of the remote control device are collected in real time through the 6-axis IMU attitude sensing module, and the current state of the remote control device is accurately identified based on the data. The current state includes the user's handheld walking roaming state, the device's stationary state, and the fixed-point control state. S3. Accurately obtain information about the area of ​​the touch panel that is obscured and the distance between the object and the touch panel through the proximity sensing module; S4. Connect to the whole-house Bluetooth Mesh network through the Mesh Bluetooth communication module, collect RSSI signals of smart devices in each room in real time, complete spatial ranging, and build a whole-house device spatial mapping library. S5. Based on the whole-house equipment space mapping library data, dynamically unbind the controlled smart devices in remote rooms, automatically bind the nearest smart devices in the current room, and seamlessly switch the touch mapping logic. S6. Based on the current status of the remote control device and the distance information between the panel's obstruction area and the object, adaptively switch between anti-accidental touch sleep mode, low-power standby mode, or fixed-point precise control mode. S7. Respond to user interaction based on the touch command.

[0058] In one embodiment, the method for accurately identifying the current state of the remote control device based on the data is as follows: When the detected acceleration fluctuation frequency is between the first preset threshold and the second preset threshold and the spatial tilt angle changes continuously beyond the preset angle, it is determined that the user is in a handheld walking roaming state. When the acceleration modulus is detected to be close to the third preset threshold, the angular velocity is close to 0, and this condition persists for more than 30 seconds, the device is determined to be in a stationary state. When the acceleration is stable, the spatial tilt angle is stable in a certain direction and accompanied by a slight angular velocity correction, the system is judged to be in a fixed-point control state.

[0059] The options described in the above system embodiments are also applicable to this embodiment, and will not be detailed here. The remaining content of this application's embodiments can be found in the above system embodiments, and will not be repeated in this embodiment.

[0060] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion, characterized in that: include: A capacitive touch interaction module, connected to a touch panel, is used to collect touch commands from the touch panel, including multi-touch, swipe, long press, and tap. A 6-axis IMU attitude sensing module is used to collect data information from the remote control device in real time and accurately identify the current state of the remote control device; the data information includes the angular velocity, acceleration, and spatial tilt angle data of the remote control device; the current state includes the user's handheld walking roaming state, the device's stationary state, and the fixed-point control state. The proximity sensing module accurately acquires information about the area of ​​the touch panel that is obstructed and the distance between the object and the touch panel; The Mesh Bluetooth communication module is used to connect to the whole-house Bluetooth Mesh network, collect RSSI signals of smart devices in each room in real time, complete spatial ranging, and build a spatial mapping library of whole-house devices. It also includes an adaptive control module, which dynamically unbinds remotely controlled smart devices and automatically binds nearby smart devices in the current room based on the whole-house device space mapping library data, and seamlessly switches the touch mapping logic; at the same time, based on the current status of the remote control device and the information on the panel obstruction area and the distance to the object obtained by the proximity sensing module, it adaptively switches between anti-accidental touch sleep mode, low-power standby mode and fixed-point precise control mode; finally, it responds to user interaction based on the touch commands, realizing a buttonless, all-scenario adaptive intelligent remote control.

2. The Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion according to claim 1, characterized in that: The capacitive touch interaction module includes: The trajectory filtering unit, with a built-in digital filter, is used to smooth continuous touch points and eliminate jitter and abnormal jump points; The micro-motion interference filtering unit is used to adaptively filter out invalid touch signals caused by hand tremors, table vibrations, electromagnetic interference, etc., to avoid false triggering; The gesture recognition unit performs pattern matching on the original touch sequence to identify specific command types such as multi-touch, swipe trajectory, long press, and tap.

3. The Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion according to claim 2, characterized in that: The 6-axis IMU attitude sensing module includes: The triaxial accelerometer unit acquires the linear acceleration of the remote-controlled device along the X, Y, and Z directions; The three-axis gyroscope unit acquires the angular velocity of the remote-controlled device around the X, Y, and Z axes; The attitude determination unit calculates the spatial tilt angle by using the values ​​of linear acceleration and angular velocity, and accurately identifies the current state of the remote control device by combining the features of linear acceleration, angular velocity and spatial tilt angle.

4. The Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion according to claim 3, characterized in that: The attitude decision unit includes: The spatial tilt angle calculation subunit integrates accelerometer and gyroscope data and uses a complementary filtering algorithm to calculate the pitch and roll angles of the remote control device in three-dimensional space. The status recognition subunit receives raw data from the three-axis accelerometer, raw data from the three-axis gyroscope, and spatial tilt angle data output by the spatial tilt angle calculation subunit. Through feature extraction and threshold judgment, it outputs the current status of the remote control device.

5. The Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion according to claim 3, characterized in that: The proximity sensing module includes: The occlusion area quantization unit calculates the percentage of the area covered by the touch panel through multi-channel sensing or imaging sensing. Distance quantization unit accurately measures the vertical distance between the object and the touch panel; The scene pre-judgment unit performs preliminary combined analysis of occlusion area and distance, and outputs coarse judgment results of suspected occlusion or suspected alignment for reference by the adaptive control module.

6. The Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion according to claim 5, characterized in that: The Mesh Bluetooth communication module includes: The RSSI real-time acquisition unit periodically scans surrounding Bluetooth Mesh devices and acquires the received signal strength indication of each device. The spatial ranging unit converts RSSI values ​​into spatial distances using a path loss model. The room device cluster analysis unit divides the smart devices in the whole house into device clusters in different rooms based on RSSI strength and spatial distance, and builds and maintains a spatial mapping library.

7. The Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion according to claim 1, characterized in that: The adaptive control module includes: The multi-sensor fusion decision unit is used to fuse four types of data from different sources, frequencies, and formats: touch commands from the capacitive touch interaction module, data information from the 6-axis IMU attitude perception module, occluded area information and distance information between the object and the touch panel from the proximity sensing module, and smart device RSSI signal and spatial distance from the Mesh Bluetooth communication module. Through timestamp alignment and feature extraction, the data is fused into a unified fusion feature vector, providing a synchronous, reliable, and multi-dimensional input basis for subsequent decisions. The four-scenario intelligent decision model unit, based on the fusion feature vector output by the multi-sensor fusion decision unit, classifies the current composite state of the system into four predefined semantic scenarios, and provides a unified decision basis for subsequent device binding, power consumption control, and touch mapping switching. The dynamic device binding and unbinding unit is used to execute the logic of unbinding remote smart devices and binding the nearest smart devices in the current room, and to maintain the binding status table.

8. The Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion according to claim 7, characterized in that: The main control adaptive control module also includes an anomaly self-healing fault-tolerant unit. When any unit is detected to have a communication timeout or data anomaly, the unit will be executed in sequence for soft reset and re-initialization. If the reset fails three times in a row, the system will be downgraded to the basic remote control mode.

9. A Bluetooth Mesh whole-house roaming adaptive remote control method based on multi-sensor fusion, used to implement the Bluetooth Mesh whole-house roaming adaptive remote control system based on multi-sensor fusion as described in any one of claims 1-8, characterized in that, Includes the following steps: The touch commands on the touch panel are collected through the capacitive touch interaction module. The touch commands include multi-touch, swipe trajectory, long press and tap. The 6-axis IMU attitude sensing module collects angular velocity, acceleration, and spatial tilt angle data of the remote control device in real time, and accurately identifies the current state of the remote control device based on the data. The current state includes user hand-held walking roaming state, device stationary state, and fixed-point control state. The proximity sensing module accurately obtains information about the area of ​​the touch panel that is obscured and the distance between the object and the touch panel. By connecting to the whole-house Bluetooth Mesh network through the Mesh Bluetooth communication module, the RSSI signals of smart devices in each room are collected in real time to complete spatial ranging and build a spatial mapping library of whole-house devices; Based on the data in the whole-house device space mapping library, the controlled smart devices in remote rooms are dynamically unbound, the nearby smart devices in the current room are automatically bound, and the touch mapping logic is switched seamlessly. Based on the current status of the remote control device and the distance information between the panel's obstruction area and the object, the system adaptively switches between anti-accidental touch sleep mode, low-power standby mode, or fixed-point precise control mode. The system responds to user interactions based on the touch commands.

10. The Bluetooth Mesh whole-house roaming adaptive remote control method based on multi-sensor fusion according to claim 9, characterized in that, The method for accurately identifying the current state of the remote control device based on the data is as follows: When the detected acceleration fluctuation frequency is between the first preset threshold and the second preset threshold and the spatial tilt angle changes continuously beyond the preset angle, it is determined that the user is in a handheld walking roaming state. When the acceleration modulus is detected to be close to the third preset threshold, the angular velocity is close to 0, and this condition persists for more than 30 seconds, the device is judged to be in a stationary state. When the acceleration is detected to be stable, the spatial tilt angle is stable in a certain direction, and there is a slight angular velocity correction, the device is judged to be in a fixed-point control state.

Images