Multi-sensor track fusion control method and system based on UWB technology

Abstract

The invention discloses a multi-sensor track fusion control method and system based on a UWB technology. The UWB technology is adopted, high-precision remote controller positioning can be achieved, the precision reaches the centimeter level, and the accuracy of gesture recognition is ensured; the gesture operation history of the user is automatically analyzed through centimeter-level positioning and a deep learning algorithm, rapid response, real-time dynamic gesture recognition and rapid television function and application starting can be achieved, operation delay is reduced, and the timeliness and accuracy of gesture control are greatly improved. Furthermore, the method also supports user-defined gestures and function mapping, so that the individuation of the operation is improved.

Description

Technical Field

[0001] The present invention relates to the field of household appliance control technology, and more specifically, to a multi-sensor trajectory fusion control method and system based on UWB technology. Background Art

[0002] With the development of smart TV technology, traditional remote control operation can no longer meet users' demand for quick and intuitive operation. Many TVs already use gesture control, including radar and visual solutions. However, direct gesture control using the remote control is more in line with user habits and provides more precise results.

[0003] The UWB technology used in existing remote control gesture recognition lacks accuracy in dynamic environments, especially in environments with multiple interference sources or multipath effects. Positioning errors and response delays often affect user experience, resulting in problems such as low positioning accuracy and slow response speed. Summary of the Invention

[0004] The present invention overcomes the shortcomings of the existing technology in solving the problems of cumbersome operation and insufficient gesture recognition accuracy of existing remote controls, and provides a multi-sensor trajectory fusion control method and system based on UWB technology, in the hope of solving the problems existing in the existing technology.

[0005] In order to solve the above technical problems, the present invention provides a multi-sensor trajectory fusion control method based on UWB technology:

[0006] A multi-sensor trajectory fusion control method based on UWB technology includes the following steps:

[0007] S1: The remote control has a built-in UWB module, which uses UWB technology to calculate the relative position of the remote control and the controlled appliance, and obtain the remote control's position in space;

[0008] S2: The user draws a graphic gesture in the air using the remote control, and the controlled electrical system captures and records the trajectory of the gesture;

[0009] S3: The remote controller is equipped with a position sensor to synchronously collect trajectory data, and the collected trajectory data is integrated with the trajectory recorded by the controlled electrical system;

[0010] S4: The controlled electrical appliance determines the type of gesture based on the drawn gesture graphic through a graphic recognition algorithm;

[0011] S5: The controlled electrical appliance triggers a controlled electrical appliance function or application corresponding to the gesture according to the judgment result.

[0012] A further technical solution is that the relative position of the remote control and the controlled appliance is calculated by using UWB technology, specifically:

[0013] S11: After collecting the UWB signal, the signal is processed to improve positioning accuracy and low-latency response;

[0014] The signal processing includes a noise filtering algorithm.

[0015] A further technical solution is to perform the following steps after executing S5:

[0016] S6: The controlled appliance records the user's gesture graphics and operation results, automatically analyzes the user's gesture operation history through a deep learning algorithm, and optimizes the mapping relationship between the gesture and the controlled appliance function in real time.

[0017] A further technical solution is that the S4 is specifically:

[0018] The controlled appliance recognizes dynamic gestures in real time based on the drawn gesture graphics and combines them with a deep learning model, and determines the type of gesture through a graphic recognition algorithm.

[0019] The types of gestures dynamically recognized according to the gesture graph include single gesture types and multiple gesture types.

[0020] A further technical solution is that S5 specifically includes the following steps:

[0021] S51: The controlled appliance executes the TV function or application corresponding to the gesture graphic based on the judgment result, and provides a successful execution feedback if the judgment result is that the gesture graphic is executable;

[0022] S52: If the judgment result is that the gesture graphic cannot be executed, the TV function or application is not executed, and only an execution failure feedback is given.

[0023] A further technical solution is that S2 specifically includes the following steps:

[0024] S21: The user inputs a gesture control activation signal, wherein the gesture control activation signal includes a set trajectory, a voice signal, and a remote control designated signal;

[0025] S22: After the gesture control activation signal is recognized, the user draws a graphic gesture in the air using the remote control, and the controlled electrical system captures and records the trajectory of the gesture;

[0026] A further technical solution is that the number of the controlled electrical systems is more than 2.

[0027] A further technical solution is to perform the following steps after executing S5:

[0028] S7: Identify whether there is a withdrawal step;

[0029] S71: The user inputs a withdrawal control signal, wherein the gesture control activation signal includes a set track, a voice signal, and a remote control designated signal;

[0030] S72: After the withdrawal control signal is recognized, the controlled electrical appliance function or application triggered by the previous graphic gesture is returned to a state where the graphic gesture is not executed.

[0031] On the other hand, the present invention also provides a multi-sensor trajectory fusion control system based on UWB technology, including a remote controller and a controlled electrical appliance;

[0032] The remote control has a built-in UWB module and a position sensor. The UWB module is used to communicate with the controlled appliance and calculate the relative position of the remote control and the controlled appliance. The position sensor is used to record the movement trajectory of the remote control.

[0033] The controlled electrical appliance is used to calculate the relative position of the remote control and the controlled electrical appliance, fuse the collected trajectory data with the trajectory recorded by the controlled electrical appliance system, determine the type of gesture based on the drawn gesture graphic through a graphic recognition algorithm, and trigger the controlled electrical appliance function or application corresponding to the gesture based on the judgment result.

[0034] Compared with existing technologies, this invention offers at least the following advantages: It utilizes UBW technology to achieve high-precision remote control positioning, reaching centimeter-level accuracy, ensuring accurate gesture recognition. Through centimeter-level positioning and automatic analysis of a user's gesture history using deep learning algorithms, it enables rapid response, real-time recognition of dynamic gestures, and rapid activation of TV functions and applications, reducing operational delays and significantly improving the timeliness and accuracy of gesture control. Furthermore, this invention supports user-defined gestures and function mapping, enhancing personalized operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of the first embodiment of the present invention; DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] Example 1

[0038] A multi-sensor trajectory fusion control method based on UWB technology, see Figure 1 , including the following steps:

[0039] S1: The remote control has a built-in UWB module, which uses UWB technology to calculate the relative position of the remote control and the controlled appliance, and obtain the remote control's position in space;

[0040] The remote control transmits signals through UWB technology, and the smart TV receives and analyzes these signals to monitor the position, speed and trajectory of the remote control in real time.

[0041] Through the high-precision positioning technology of UWB signals, the spatial position changes of the remote control can be accurately captured, ensuring the accuracy and response speed of gesture drawing.

[0042] The relative position of the remote control and the controlled appliance is calculated using UWB technology, specifically:

[0043] In a preferred embodiment, S11: after collecting the UWB signal, processing the signal to improve positioning accuracy and low-latency response;

[0044] The signal processing includes a noise filtering algorithm.

[0045] Exemplarily, the noise filtering algorithm includes the following steps:

[0046] 1. Multipath Interference Signal Identification: Based on the arrival time characteristics of the UWB signal, the system pre-segments the received waveform using a time-domain extended window function to perform a preliminary classification of the main signal and multipath reflection signals. Reflection paths are eliminated using a signal strength difference threshold method.

[0047] 2. Adaptive environmental noise modeling: After the system is started, a 30-second static environment sample is collected, a background noise model is constructed based on the Gaussian mixture model (GMM), and abnormal signal detection is performed using KL divergence.

[0048] 3. Direction coupling correction: The attitude vector is calculated based on the gyroscope angular velocity output, and cosine similarity correction is performed with the UWB ranging direction to achieve attitude error compensation.

[0049] 4. Self-learning weighted filtering processing: Utilizes a neural weighted filtering algorithm to dynamically adjust signal point weights. Inputs include signal strength, position increment, posture change, and confidence score to achieve dynamic filtering optimization.

[0050] 5. Curve reconstruction and trajectory smoothing: Use high-order B-spline curves to fit the filtered point sequence, retaining velocity direction inflection points and graphic sharp points, improving the smoothness and expressiveness of gesture recognition.

[0051] S2: The user draws a graphic gesture in the air using the remote control, and the controlled electrical system captures and records the trajectory of the gesture;

[0052] It should be noted that the number of the controlled electrical systems can be one or more than two.

[0053] For example, the remote control not only controls the smart TV, but also other smart home devices (such as air conditioners, speakers, lights, etc.) through gestures. For example, when the user draws a "V" gesture, the remote control will adjust the TV volume and the air conditioner temperature simultaneously.

[0054] In a preferred embodiment, the step S2 specifically includes the following steps:

[0055] S21: The user inputs a gesture control activation signal, wherein the gesture control activation signal includes a set trajectory, a voice signal, and a remote control designated signal;

[0056] S22: After the gesture control activation signal is recognized, the user draws a graphic gesture in the air using the remote control, and the controlled electrical system captures and records the trajectory of the gesture;

[0057] Because users also use and move the remote control in daily life, in order to avoid misoperation caused by misrecognition of non-gesture control operations, a step of inputting a gesture control activation signal is added before inputting gesture control operations to reduce the possibility of misoperation.

[0058] S3: The remote controller is equipped with a position sensor to synchronously collect trajectory data, and the collected trajectory data is integrated with the trajectory recorded by the controlled electrical system;

[0059] Exemplarily, the data fusion of the collected trajectory data and the recorded trajectory of the controlled electrical system includes the following steps:

[0060] 1. Timing synchronization and frame loss compensation: The system time synchronization is used to control the error within 5ms. If there is data loss, a second-order prediction function is used to compensate based on the IMU historical data.

[0061] 2. Coordinate system normalization conversion: Use the quaternion attitude solution algorithm to map the IMU local coordinates to the UWB global space to unify the three-dimensional coordinate system.

[0062] 3. Multi-source dynamic weighted fusion: Dynamic fusion weights are calculated based on sensor stability, drift rate, user motion pattern, and environmental recognition results, and extended Kalman filtering is used to achieve spatiotemporal position fusion.

[0063] 4. Trajectory anomaly detection and dynamic optimization: Detect sudden changes in trajectory direction and speed anomalies, and smooth out abnormal sections through Hermite interpolation to ensure trajectory continuity.

[0064] 5. Semantic layer trajectory mapping optimization: Extract features such as strokes and closure from the fused trajectory, segment and label it as input for deep learning, and improve the accuracy of subsequent gesture recognition.

[0065] Exemplarily, the position sensor may be a single accelerometer, a gyroscope, a magnetometer, or a combination of the above sensors, which can accurately identify the gesture path drawn by the user in the air.

[0066] In this embodiment, the position sensor is a gyroscope. The remote control uses UWB technology to obtain the spatial position of the user's gestures in real time. It also combines data from the accelerometer and gyroscope to accurately track the speed and direction of the gestures. If the UWB positioning signal is affected by interference or multipath, the system automatically corrects the positioning error based on signals from other sensors, ensuring high-precision gesture recognition.

[0067] S4: The controlled electrical appliance determines the type of gesture based on the drawn gesture graphic through a graphic recognition algorithm;

[0068] In a preferred embodiment, the S4 is specifically:

[0069] The controlled appliance recognizes dynamic gestures in real time based on the drawn gesture graphics and combines them with a deep learning model, and determines the type of gesture through a graphic recognition algorithm.

[0070] The types of gestures dynamically recognized according to the gesture graph include single gesture types and multiple gesture types.

[0071] For example, the gesture graphic may be a preset standard one-stroke graphic or subtitle (such as a circle, a triangle, a Z, etc.), or may be a freely drawn one-stroke graphic customized by the user.

[0072] S5: The controlled electrical appliance triggers a controlled electrical appliance function or application corresponding to the gesture according to the judgment result.

[0073] It should be noted that the system has preset mappings between different graphics and TV functions. For example, drawing a circle will launch an application, drawing a triangle will launch the App Store, and drawing the letter J will mute the TV.

[0074] Users can also customize the correspondence between graphics and functions through the settings menu to suit their personal habits.

[0075] In a preferred embodiment, the step S5 specifically includes the following steps:

[0076] S51: The controlled appliance executes the TV function or application corresponding to the gesture graphic based on the judgment result, and provides a successful execution feedback if the judgment result is that the gesture graphic is executable;

[0077] S52: If the judgment result is that the gesture graphic cannot be executed, the TV function or application is not executed, and only an execution failure feedback is given.

[0078] The execution success feedback and execution failure feedback may be different prompt sounds, light signals, vibration signals, or a combination of the above signals.

[0079] It should be noted that the customer's daily operations, such as picking up, or simple trajectory movement within a certain range, should be identified as the customer's normal movement, and there is no need to give execution failure feedback.

[0080] However, when the customer has performed an uncommon daily action and the image recognized by the action cannot correspond to the TV function or application, an execution failure feedback is given.

[0081] After executing S5, perform the following steps:

[0082] S6: The controlled appliance records the user's gesture graphics and operation results, automatically analyzes the user's gesture operation history through a deep learning algorithm, and optimizes the mapping relationship between the gesture and the controlled appliance function in real time.

[0083] S7: Identify whether there is a withdrawal step;

[0084] S71: The user inputs a withdrawal control signal, wherein the gesture control activation signal includes a set track, a voice signal, and a remote control designated signal;

[0085] S72: After the withdrawal control signal is recognized, the controlled electrical appliance function or application triggered by the previous graphic gesture is returned to a state where the graphic gesture is not executed.

[0086] Example 2

[0087] A multi-sensor trajectory fusion control system based on UWB technology, including a remote controller and controlled electrical appliances;

[0088] The remote control has a built-in UWB module and a position sensor. The UWB module is used to communicate with the controlled appliance and calculate the relative position of the remote control and the controlled appliance. The position sensor is used to record the movement trajectory of the remote control.

[0089] The controlled electrical appliance is used to calculate the relative position of the remote control and the controlled electrical appliance, fuse the collected trajectory data with the trajectory recorded by the controlled electrical appliance system, determine the type of gesture based on the drawn gesture graphic through a graphic recognition algorithm, and trigger the controlled electrical appliance function or application corresponding to the gesture based on the judgment result.

[0090] Although the present invention has been described herein with reference to illustrative embodiments of the present invention, it will be appreciated that those skilled in the art may devise numerous other modifications and implementations that fall within the scope and spirit of the principles disclosed herein. More specifically, within the scope disclosed herein, various variations and improvements may be made to the components and / or layout of the subject combination layout. In addition to variations and improvements made to the components and / or layout, other uses will be apparent to those skilled in the art.

Claims

1. A multi-sensor trajectory fusion control method based on UWB technology, characterized in that: The following steps are involved: S1: The remote control has a built-in UWB module, which uses UWB technology to calculate the relative position of the remote control and the controlled appliance, and obtain the remote control's position in space; S2: The user draws a graphic gesture in the air using the remote control, and the controlled electrical system captures and records the trajectory of the gesture; S3: The remote controller is equipped with a position sensor to synchronously collect trajectory data, and the collected trajectory data is integrated with the trajectory recorded by the controlled electrical system; S4: The controlled electrical appliance determines the type of gesture based on the drawn gesture graphic through a graphic recognition algorithm; S5: The controlled electrical appliance triggers a controlled electrical appliance function or application corresponding to the gesture according to the judgment result.

2. The multi-sensor trajectory fusion control method based on UWB technology according to claim 1, characterized in that: The relative position of the remote control and the controlled appliance is calculated using UWB technology, specifically: S11: After collecting the UWB signal, the signal is processed to improve positioning accuracy and low-latency response; The signal processing includes a noise filtering algorithm.

3. The multi-sensor trajectory fusion control method based on UWB technology as claimed in claim 2, characterized in that: The noise filtering algorithm includes one or more of the following steps: Multipath interference signal identification; Adaptive environmental noise modeling; Direction coupling correction: calculate the attitude vector by combining the gyroscope angular velocity output and perform cosine similarity correction with the UWB ranging direction; Self-learning weighted filtering processing uses a neural weighted filtering algorithm to dynamically adjust the weights of signal points. Inputs include signal strength, position increment, posture change, and confidence score. Curve reconstruction and trajectory smoothing use high-order B-spline curves to fit the filtered point sequence, retaining the velocity direction inflection points and graphic peaks.

4. The multi-sensor trajectory fusion control method based on UWB technology according to claim 1, characterized in that: The S2 specifically includes the following steps: S21: The user inputs a gesture control activation signal, wherein the gesture control activation signal includes a set trajectory, a voice signal, and a remote control designated signal; S22: After the gesture control activation signal is recognized, the user draws a graphic gesture in the air through the remote control, and the controlled electrical system captures and records the trajectory of the gesture.

5. The multi-sensor trajectory fusion control method based on UWB technology according to claim 1, characterized in that: The S3 specifically includes the following steps: S31: Timing synchronization and frame loss compensation; S32: Coordinate system normalization conversion, using the quaternion attitude solution algorithm to map the IMU local coordinates to the UWB global space and unify the three-dimensional coordinate system; S33: Multi-source dynamic weighted fusion, calculates dynamic fusion weights, and uses extended Kalman filtering to achieve spatiotemporal position fusion; S34: Trajectory anomaly detection and dynamic optimization, detecting sudden changes in trajectory direction and speed anomalies, and smoothing abnormal sections through Hermite interpolation; S35: Semantic layer trajectory mapping optimization: Extract stroke, closure and other features from the fused trajectory, segment and label it as the input for deep learning.

6. The multi-sensor trajectory fusion control method based on UWB technology as claimed in claim 3, characterized in that: The S4 is specifically: The controlled appliance recognizes dynamic gestures in real time based on the drawn gesture graphics and combines them with a deep learning model, and determines the type of gesture through a graphic recognition algorithm. The types of gestures dynamically recognized according to the gesture graph include single gesture types and multiple gesture types.

7. The multi-sensor trajectory fusion control method based on UWB technology according to claim 1, characterized in that: The S5 specifically includes the following steps: S51: The controlled appliance executes the TV function or application corresponding to the gesture graphic based on the judgment result, and provides a successful execution feedback if the judgment result is that the gesture graphic is executable; S52: If the judgment result is that the gesture graphic cannot be executed, the TV function or application is not executed, and only an execution failure feedback is given.

8. The multi-sensor trajectory fusion control method based on UWB technology according to claim 1, characterized in that: After executing S5, perform the following steps: S6: The controlled appliance records the user's gesture graphics and operation results, automatically analyzes the user's gesture operation history through a deep learning algorithm, and optimizes the mapping relationship between the gesture and the controlled appliance function in real time.

9. The multi-sensor trajectory fusion control method based on UWB technology according to claim 1, characterized in that: After executing S5, perform the following steps: S7: Identify whether there is a withdrawal step; S71: The user inputs a withdrawal control signal, wherein the gesture control activation signal includes a set track, a voice signal, and a remote control designated signal; S72: After the withdrawal control signal is recognized, the controlled electrical appliance function or application triggered by the previous graphic gesture is returned to a state where the graphic gesture is not executed.

10. A multi-sensor trajectory fusion control system based on UWB technology, characterized in that: Including remote controls and controlled electrical appliances; The remote control has a built-in UWB module and a position sensor. The UWB module is used to communicate with the controlled appliance and calculate the relative position of the remote control and the controlled appliance. The position sensor is used to record the movement trajectory of the remote control. The controlled electrical appliance is used to calculate the relative position of the remote control and the controlled electrical appliance, fuse the collected trajectory data with the trajectory recorded by the controlled electrical appliance system, determine the type of gesture based on the drawn gesture graphic through a graphic recognition algorithm, and trigger the controlled electrical appliance function or application corresponding to the gesture based on the judgment result.

Images