Bluetooth remote control based on gesture recognition

Abstract

The application relates to the technical field of gesture control devices, and discloses a Bluetooth remote controller based on gesture recognition. The Bluetooth remote controller comprises a gesture acquisition module, a feature extraction module, an instruction mapping module and a Bluetooth communication module. The gesture acquisition module acquires user hand action data, determines an acquisition frequency and a data dimension, acquires space-time features to identify a motion trajectory and a posture change; the feature extraction module extracts key feature points based on the motion trajectory and the posture change, positions relative coordinates of the key feature points, and determines a feature vector in combination with the space-time features; the instruction mapping module identifies an effective feature set and a feature weight, constructs an instruction matching model, and sets an instruction mapping rule; and the Bluetooth communication module encodes a control instruction in combination with the feature vector and the instruction mapping rule, selects a transmission protocol, and sets a data packet format. The remote controller realizes convenient gesture-based control through comprehensive capture, accurate analysis and efficient transmission of user hand actions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of gesture control devices, in particular to a Bluetooth remote controller based on gesture recognition. BACKGROUND

[0002] Remote controllers, as a common control device, are widely used in various electronic devices such as televisions, air conditioners, and smart speakers. Traditional remote controllers rely on physical buttons to achieve control functions, and users need to send corresponding instructions by pressing different buttons. However, the layout of physical buttons is fixed, and when the number of device functions increases, the number of buttons also increases, which makes it difficult for users to quickly find the required buttons during operation, especially for the elderly or people unfamiliar with the device, which has a high operation threshold.

[0003] With the development of technology, some remote controllers based on voice or touch have appeared, but these remote controllers also have limitations. Voice remote controllers are easily disturbed by environmental noise, and their recognition accuracy decreases significantly in noisy environments, and there is a risk of privacy leakage; touch remote controllers require direct contact between the user's fingers and the screen, and the operation sensitivity will be affected when the hands are wet or have stains, and frequent touch can also cause screen wear.

[0004] Some remote controllers that use motion sensing technology have obvious shortcomings in gesture recognition. They can only recognize simple gestures, and it is difficult to accurately capture complex continuous gestures or subtle posture changes. In the data collection process, the extraction of space-time features is not comprehensive enough, resulting in poor stability of gesture recognition. At the same time, the communication between such remote controllers and the controlled devices mostly uses fixed transmission protocols, which can easily cause delays or packet loss during data transmission, affecting the user's experience.

[0005] In terms of instruction mapping, the correspondence between gestures and instructions in existing technologies is relatively single and cannot be adjusted according to the usage habits of different users, lacking flexibility. Moreover, most remote controllers do not effectively analyze feature vectors during data processing and transmission, resulting in low instruction encoding efficiency and difficulty in meeting real-time control requirements. SUMMARY

[0006] The purpose of the present application is to provide a Bluetooth remote controller based on gesture recognition to solve the problems raised in the background.

[0007] To achieve the above purpose, the present application provides a Bluetooth remote controller based on gesture recognition, which comprises:

[0008] A gesture acquisition module is used to acquire user hand movement data, determine the acquisition frequency and data dimension of the hand movement data, collect the space-time features of the hand movement data, and identify the motion trajectory and posture change of the hand movement according to the space-time features.

[0009] a feature extraction module configured to extract key feature points of the hand action based on the motion trajectory and the posture change, position relative coordinates of the key feature points in the motion trajectory, and determine a feature vector of the hand action based on the spatiotemporal features and the relative coordinates;

[0010] an instruction mapping module configured to identify an effective feature set in the feature vector and a feature weight of the effective feature set, construct an instruction matching model of the effective feature set based on the feature weight, and set an instruction mapping rule of the hand action according to the instruction matching model and the feature weight;

[0011] a Bluetooth communication module configured to encode a control instruction of the hand action in real time in combination with the feature vector and the instruction mapping rule, select a transmission protocol of the Bluetooth communication based on the control instruction, and set a data packet format of the Bluetooth communication in combination with the transmission protocol and the control instruction.

[0012] Preferably, the Bluetooth remote controller further comprises:

[0013] the interference detection module is configured to monitor a signal strength of the Bluetooth communication based on the data packet format, analyze an interference type of the Bluetooth communication according to the signal strength and the transmission protocol, and set an anti-interference mode of the Bluetooth communication in combination with the interference type and the signal strength;

[0014] the adaptive learning module is configured to identify a gesture habit preference of the user based on the anti-interference mode, optimize an allocation strategy of the feature weight according to the gesture habit preference and the instruction matching model, and perform instruction conversion processing on the hand action to optimize a control instruction according to the anti-interference mode, the allocation strategy, and the instruction mapping rule.

[0015] Preferably, the determination of the feature vector of the hand action based on the spatiotemporal features and the relative coordinates comprises:

[0016] separating dynamic features and static features of the hand action based on the spatiotemporal features;

[0017] extracting a speed parameter and an acceleration parameter of the hand action according to the dynamic features;

[0018] calculating a spatial distribution density of the key feature points based on the static features and the relative coordinates;

[0019] determining the feature vector of the hand action in combination with the speed parameter, the acceleration parameter, and the spatial distribution density.

[0020] Preferably, the instruction matching model of the effective feature set is constructed based on the feature weight, including:

[0021] Screening the core feature component in the effective feature set based on the feature weight;

[0022] Identifying the standard instruction template corresponding to the core feature component;

[0023] Analyzing the similarity threshold of the standard instruction template and the core feature component;

[0024] Extracting the feature change trend of the effective feature set, and analyzing the influence degree of the feature change trend on instruction matching according to the similarity threshold;

[0025] Constructing the instruction matching model of the effective feature set according to the similarity threshold and the influence degree.

[0026] Preferably, the instruction mapping rule of the hand action is set according to the instruction matching model and the feature weight, including:

[0027] Real-time comparison of the feature vector and the standard feature vector in the instruction matching model;

[0028] According to the instruction matching model, the matching degree of the feature vector and the standard feature vector is calculated;

[0029] The association relationship data of the feature weight and the matching degree is scheduled, and the validity threshold of the matching degree is determined based on the association relationship data;

[0030] According to the matching degree and the validity threshold, the instruction mapping priority of the hand action is set;

[0031] The duration of the hand action is identified, and the instruction mapping rule of the hand action is set based on the duration and the instruction mapping priority.

[0032] Preferably, the transmission protocol of the Bluetooth communication is selected based on the control instruction, including:

[0033] The control instruction is classified by instruction type to obtain a classified instruction;

[0034] The real-time requirement of the classified instruction is extracted, and the protocol performance parameters of the Bluetooth communication are collected based on the real-time requirement;

[0035] Identifying the transmission rate and delay index in the protocol performance parameters;

[0036] According to the transmission rate, the bandwidth occupancy rate of the Bluetooth communication is analyzed;

[0037] analyzing a response time range of the Bluetooth communication based on the delay index;

[0038] selecting a transmission protocol of the Bluetooth communication in combination with the bandwidth occupancy, the response time range and the real-time requirement.

[0039] Preferably, the setting the anti-interference mode of the Bluetooth communication in combination with the interference type and the signal strength comprises:

[0040] determining a communication distance interval of the Bluetooth communication based on the signal strength;

[0041] analyzing an interference source distribution feature within the communication distance interval;

[0042] identifying a vulnerable interference frequency band of the Bluetooth communication and identifying an interference intensity of the vulnerable interference frequency band according to the interference type and the interference source distribution feature;

[0043] setting a frequency hopping strategy of the Bluetooth communication according to the signal strength and the interference intensity;

[0044] setting the anti-interference mode of the Bluetooth communication in combination with the interference intensity and the frequency hopping strategy.

[0045] Preferably, the identifying the gesture habit preference of the user based on the anti-interference mode comprises:

[0046] collecting historical gesture instruction data of the user based on the anti-interference mode;

[0047] counting a high-frequency gesture type of the user according to the historical gesture instruction data;

[0048] identifying an action amplitude and an execution speed of the high-frequency gesture type, and analyzing a gesture feature rule of the user in combination with the action amplitude and the execution speed;

[0049] setting an identification threshold of the gesture habit preference based on the high-frequency gesture type;

[0050] identifying the gesture habit preference of the user according to the identification threshold and the gesture feature rule.

[0051] Preferably, the optimizing the feature weight distribution strategy according to the gesture habit preference and the instruction matching model comprises:

[0052] determining a user-sensitive feature in the effective feature set according to the gesture habit preference;

[0053] Analyze the recognition accuracy of the user sensitive feature, and set a weight adjustment coefficient of the user sensitive feature based on the recognition accuracy;

[0054] According to the instruction matching model, the relevance of the user sensitive feature and other features is identified;

[0055] Based on the relevance, a weight compensation coefficient of the other features is set;

[0056] The allocation strategy of the feature weight is optimized in combination with the weight adjustment coefficient and the weight compensation coefficient.

[0057] Preferably, according to the anti-interference mode, the allocation strategy and the instruction mapping rule, the hand action is subjected to instruction conversion processing, and the control instruction is optimized, comprising:

[0058] According to the anti-interference mode, the signal stability of the Bluetooth communication is confirmed;

[0059] Based on the signal stability, a trigger condition of the instruction mapping rule is set, and an instruction anti-shake mechanism of the hand action is set;

[0060] According to the trigger condition and the instruction anti-shake mechanism, the effectiveness of the feature vector is verified, and a verification result is obtained;

[0061] Based on the verification result and the allocation strategy, the hand action is subjected to instruction conversion processing, and a control instruction is obtained.

[0062] Compared with the prior art, the present application has the following advantages:

[0063] When the gesture acquisition module acquires user hand action data, the acquisition frequency and data dimension can be determined, and the space-time feature can be acquired to identify the motion trajectory and posture change. This way makes the capture of user hand action more comprehensive, whether it is a simple single gesture or a complex continuous gesture, it can be recorded and analyzed in detail, so that the range of gesture recognition is expanded, and it is no longer limited to a few fixed actions.

[0064] The feature extraction module extracts key feature points based on the motion trajectory and posture change, and locates their relative coordinates, and determines the feature vector in combination with the space-time feature. This process can accurately filter out representative information from the hand action data, and eliminate redundant data, so that the subsequent instruction processing is more efficient. Through the analysis of key feature points, the essential features of gestures can be more clearly reflected, which helps to improve the accuracy of gesture recognition and makes the distinction between different gestures more explicit.

[0065] The instruction mapping module identifies the effective feature set and the feature weight, constructs an instruction matching model and sets an instruction mapping rule. This makes the correspondence between gestures and control instructions more flexible, and can adapt to the operation habits of different users. Users can realize personalized matching of gestures and instructions according to their own needs, without being limited by fixed mapping relationships, thereby enhancing the adaptability of the remote controller.

[0066] The Bluetooth communication module combines the feature vector and the instruction mapping rule to encode the control instruction in real time, and selects a transmission protocol and sets a data packet format according to the control instruction. This approach can make the data transmission more in line with actual needs, different control instructions correspond to suitable transmission methods, reduce unnecessary resource consumption in the data transmission process, and at the same time make the entire process from generation to transmission of the instruction more smooth, thereby improving the real-time performance of the control.

[0067] The remote controller combines gesture recognition with Bluetooth communication technology, and breaks free from the shackles of physical buttons. Users only need to perform hand actions to control the device, thereby simplifying the operation process. In the use process, there is no need to directly touch the remote controller, thereby reducing various restrictions caused by contact. Whether it is to control kitchen appliances while cooking or to operate entertainment devices while exercising, the remote controller can provide a convenient use experience.

[0068] The remote controller systematically designs information collection, extraction, mapping and transmission at each link of data processing, so that the entire control process forms an organic whole. The modules work cooperatively from capturing hand actions to sending instructions, each step is closely connected, the interference of intermediate links is reduced, and the remote controller can maintain stable performance in various use scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 A working principle diagram of the Bluetooth remote controller based on gesture recognition described in the present application;

[0070] Figure 2 A flowchart for constructing the instruction matching model;

[0071] Figure 3 A flowchart for selecting the transmission protocol;

[0072] Figure 4 A flowchart for recognizing gesture habit preferences;

[0073] Figure 5 A flowchart for feature weight distribution optimization. DETAILED DESCRIPTION

[0074] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0075] Please refer to Figure 1 The present application provides a Bluetooth remote controller based on gesture recognition.

[0076] The gesture acquisition module acquires user hand action data, determines the acquisition frequency and data dimension of the hand action data, acquires the space-time characteristics of the hand action data, and identifies the motion trajectory and posture change of the hand action according to the space-time characteristics.

[0077] The feature extraction module extracts key feature points of the hand action based on the motion trajectory and posture change, locates the relative coordinates of the key feature points in the motion trajectory, and determines the feature vector of the hand action based on the space-time characteristics and the relative coordinates.

[0078] The instruction mapping module identifies the effective feature set in the feature vector and the feature weight of the effective feature set, constructs an instruction matching model of the effective feature set based on the feature weight, and sets the instruction mapping rule of the hand action according to the instruction matching model and the feature weight.

[0079] The Bluetooth communication module combines the feature vector and the instruction mapping rule to encode the control instruction of the hand action in real time, selects the transmission protocol of Bluetooth communication based on the control instruction, and sets the data packet format of Bluetooth communication in combination with the transmission protocol and the control instruction.

[0080] Embodiment 1: In the Bluetooth remote controller based on gesture recognition, the collaborative work of the interference detection module and the adaptive learning module constitutes an important part of the stable operation of the system in a complex environment and the adaptation to the user's operation habits.

[0081] The running process of the interference detection module starts with the analysis of the data packet format. The data packet format contains information such as the encoding structure of the control instruction, the check bit and the data length, and the module constructs a benchmark framework for signal monitoring based on these information. When monitoring the signal strength of Bluetooth communication, a real-time sampling method is adopted, and the signal strength value is obtained once every preset time period to form a continuous signal strength sequence. This sequence can reflect the fluctuation of the signal in the transmission process, for example, when the number of obstructions increases or other wireless devices are turned on, the signal strength will show different degrees of attenuation or fluctuation.

[0082] According to the signal strength and transmission protocol, the interference type of Bluetooth communication is analyzed. The characteristics of the transmission protocol itself will affect the form of interference, for example, some protocols are susceptible to interference from wireless signals in the same frequency band when operating at high frequencies, while others may be more sensitive to noise in low-power mode. Combined with the characteristics of signal strength changes, interference types can be distinguished, including co-channel interference, adjacent channel interference, and multipath interference. Co-channel interference usually manifests as a sharp fluctuation in signal strength over a short period of time, with the fluctuation frequency consistent with the transmission frequency of the interference source; adjacent channel interference causes a sustained slow decline in signal strength, accompanied by some signal distortion; and multipath interference presents a periodic fluctuation in signal strength, related to the distribution of reflecting objects in the environment.

[0083] Combined with the interference type and signal strength, the anti-interference mode of Bluetooth communication is set. For co-channel interference, the frequency hopping interval of the communication channel is adjusted according to the attenuation amplitude of the signal strength, and when the signal strength attenuation exceeds a certain range, the frequency hopping interval is shortened to reduce the residence time on the interfered channel; for adjacent channel interference, a channel filtering mechanism is enabled based on the stability of the signal strength to filter out interference signals outside a certain frequency band; for multipath interference, the sampling time of signal reception is optimized based on the fluctuation period of signal strength, and data sampling is performed at times when signal strength is relatively stable.

[0084] The adaptive learning module works based on the anti-interference mode. After the anti-interference mode is determined, the module begins to collect historical gesture command data of the user. These data include the user's hand movements, corresponding feature vectors, generated control commands, and feedback results of command execution under different anti-interference modes. During the collection process, the data is labeled in chronological order to distinguish the operation records of different time periods.

[0085] According to the historical gesture command data, the user's high-frequency gesture types are counted. During the counting, the motion trajectory and posture change of the hand movement are used as the classification basis, gestures with high feature vector similarity are classified into the same type, and then the proportion of each type of gesture in the total number of operations is calculated. The type whose proportion exceeds the preset value is the high-frequency gesture type.

[0086] The action amplitude and execution speed of the high-frequency gesture type are identified. The action amplitude is represented by the maximum displacement of the key feature points in the motion trajectory, i.e., the maximum difference of the relative coordinates of the key feature points; the execution speed is determined by the ratio of the time required to complete the entire motion trajectory to the length of the trajectory. Combined with the action amplitude and execution speed, the user's gesture characteristic law is analyzed, for example, some users have a larger action amplitude but slower execution speed when performing high-frequency gestures, while others may have a smaller action amplitude but faster execution speed.

[0087] Based on the high-frequency gesture type, the recognition threshold of gesture habit preference is set. The recognition threshold includes the similarity threshold of the feature vector, the deviation range of the action amplitude, and the allowed fluctuation interval of the execution speed, etc. For the high-frequency gesture type, the similarity threshold of the feature vector is set relatively low to accommodate a larger range of action deformation, while the deviation range of the action amplitude and the allowed fluctuation interval of the execution speed are adjusted according to the statistical results of the gesture type, so that most actions consistent with the user's habits can be included in the recognition range.

[0088] According to the recognition threshold and the gesture feature rule, the user's gesture habit preference is recognized. In the real-time recognition process, the feature vector of the current hand action is compared with the standard feature vector of the high-frequency gesture type. When the similarity exceeds the recognition threshold, and the action amplitude and execution speed fall within the corresponding deviation range, it is determined that the current action conforms to the user's gesture habit preference.

[0089] According to the gesture habit preference and the instruction matching model, the allocation strategy of the feature weight is optimized. The instruction matching model contains the basic weight of various features in the instruction recognition, and the weights are adjusted in combination with the user's feature dependence on the high-frequency gesture type. For example, if the user's hand action speed parameter consistency is high when executing a certain high-frequency gesture, the weight of the speed parameter in the feature vector is increased; if the user relies more on the posture change of the hand to distinguish different instructions, the proportion of the static feature in the feature weight is increased.

[0090] According to the anti-interference mode, the allocation strategy and the instruction mapping rule, the hand action is processed for instruction conversion, and the control instruction is optimized. When the anti-interference mode is the frequency hopping mode, the instruction conversion will preferentially select the control instruction with shorter encoding length to reduce the data transmission time; based on the optimized feature weight allocation strategy, the user-sensitive feature components are highlighted when generating the feature vector, so that the instruction matching is more in line with the user's habits; at the same time, according to the instruction mapping rule, the converted control instruction is format-verified to ensure that it meets the requirements of the current transmission protocol and data packet format, and finally the optimized control instruction is formed.

[0091] Embodiment 2: see Figure 2 In the Bluetooth remote controller based on gesture recognition, the feature vector of the hand action needs to go through multiple processing steps. Based on the space-time feature, the dynamic feature and the static feature of the hand action are separated. The space-time feature contains the change information in the time dimension and the position information in the space dimension. The dynamic feature mainly reflects the change of the hand action with time, such as the flexion and extension speed of the fingers, the change of the palm trajectory, etc. The static feature reflects the fixed posture of the hand at a certain moment, such as the bending degree of the fingers, the inclination angle of the palm, etc. By analyzing the space-time feature, the dynamic feature and the static feature can be distinguished and processed separately.

[0092] According to the dynamic characteristics, the speed parameter and the acceleration parameter of the hand action are extracted. The speed parameter is obtained by calculating the displacement of the hand action in a unit time, and the displacement is determined based on the coordinate changes of the key feature points in the motion trajectory. The instantaneous speed at the corresponding time is obtained by comparing the coordinate difference of the adjacent two sampling time points with the sampling time interval, and then the speed parameter in the whole action process is integrated. The acceleration parameter is calculated by the change of the speed parameter, that is, the ratio of the speed difference of the adjacent two time points to the time interval, so as to reflect the change of the speed of the hand action.

[0093] Based on the static characteristics and the relative coordinates, the spatial distribution density of the key feature points is calculated. The static characteristics provide the position information of the key feature points in a certain static posture, and the relative coordinates clarify the position relationship between the key feature points. A certain spatial range is selected as the calculation region, the number of the key feature points in the region is counted, and then the number is compared with the volume of the region to obtain the spatial distribution density, which can reflect the aggregation degree of the key feature points of the hand in the static posture.

[0094] The feature vector of the hand action is determined by combining the speed parameter, the acceleration parameter and the spatial distribution density. The speed parameter, the acceleration parameter and the spatial distribution density are quantitatively processed and converted into numerical form, and these numerical values are arranged in a predetermined order to form a multi-dimensional array, which is the feature vector of the hand action. The feature vector contains the key information of the hand action in dynamic and static aspects, and can be used for subsequent instruction recognition and matching.

[0095] In constructing the instruction matching model of the effective feature set, the core feature components in the effective feature set are screened based on the feature weights. The feature weights reflect the importance of each feature in the hand action recognition, and the features with higher weight values have greater influence on the action recognition. By comparing the weight values of each feature, the features with weight values exceeding a predetermined threshold are selected as the core feature components, which are the keys to distinguish different hand actions.

[0096] The standard instruction template corresponding to the core feature components is recognized. The standard instruction template is pre-set, and each standard instruction corresponds to a specific combination of core feature components. These templates are trained based on a large amount of sample data, and contain the numerical range and distribution of the typical core feature components corresponding to each type of instruction. By comparing the core feature components with the feature information in the standard instruction template, the corresponding standard instruction template can be found.

[0097] The similarity threshold of the standard instruction template and the core feature component is analyzed. The similarity threshold is used to judge the matching degree of the core feature component and the standard instruction template. The difference degree of the core feature component and the corresponding feature in the standard instruction template is calculated, such as the Euclidean distance, the cosine similarity, etc. A critical value is set as the similarity threshold. When the difference degree is less than the threshold, it is considered that the two have a high similarity.

[0098] The feature change trend of the effective feature set is extracted, and the influence degree of the feature change trend on instruction matching is analyzed according to the similarity threshold. The feature change trend refers to the change law of each feature in the effective feature set over time, such as the numerical value of some features showing an upward trend and the numerical value of some other features showing a downward trend. Combined with the similarity threshold, the influence on the instruction matching result when the feature change trend conforms to the trend in the standard instruction template is observed, and the influence on the matching result when the trend deviates is observed, so as to determine the influence degree.

[0099] According to the similarity threshold and the influence degree, an instruction matching model of the effective feature set is constructed. The similarity threshold is used as the matching judgment condition in the model, and the influence degree of the feature change trend is also included in the calculation process of the model. When the similarity of the core feature component and the standard instruction template reaches the threshold, and the influence degree of the feature change trend is within the acceptable range, the model judges that the effective feature set matches the corresponding standard instruction, so as to complete the construction of the instruction matching model. The model can be used for instruction recognition of the feature vector of the hand action.

[0100] Embodiment 3: see Figure 3 When setting the instruction mapping rule of the hand action, the feature vector is compared with the standard feature vector in the instruction matching model in real time. The feature vector contains multi-dimensional information of the hand action, such as the speed parameter, the acceleration parameter, and the spatial distribution density of the key feature point. The standard feature vector is the feature benchmark corresponding to each type of preset instruction. In the comparison process, the numerical difference between the two is compared dimension by dimension, the matching situation of each dimension is recorded, and a preliminary comparison result is formed.

[0101] According to the instruction matching model, the matching degree of the feature vector and the standard feature vector is calculated. The calculation of the matching degree covers the difference degree of each dimension feature. The difference value of each dimension is weighted according to the weight of the dimension in the feature vector, and then a normalized processing is performed to obtain a value between 0 and 1. The value is the matching degree.

[0102] The scheduling feature weight and the matching degree are associated with the association data, and the validity threshold of the matching degree is determined based on the association data. The association data records the corresponding relationship between the matching degree and the instruction recognition accuracy under different feature weight combinations. By analyzing these data, the matching degree critical value that can distinguish between valid instructions and invalid instructions is found, which is the validity threshold. When the matching degree is higher than the validity threshold, the corresponding hand movement is considered as a valid movement, and subsequent instruction mapping can be performed; when the matching degree is lower than the validity threshold, the hand movement is determined as an invalid movement, and no corresponding control instruction is generated.

[0103] According to the matching degree and the validity threshold, the instruction mapping priority of the hand movement is set. For the hand movement with a matching degree much higher than the validity threshold, a higher mapping priority is given, and instruction conversion and transmission are performed preferentially; for the hand movement with a matching degree slightly higher than the validity threshold, the mapping priority is relatively low, and the processing may be delayed when the system resources are tight; for the hand movement with a matching degree close to the validity threshold, the validity needs to be further verified, and then the mapping priority is determined.

[0104] The duration of the hand movement is recognized, and based on the duration and the instruction mapping priority, the instruction mapping rule of the hand movement is set. The duration of the hand movement refers to the time interval from the start of the movement to the end of the movement, which is calculated by recording the start time and the end time of the hand movement data obtained by the gesture acquisition module. In the instruction mapping rule, different durations correspond to different instruction mapping methods, for example, the same hand movement corresponds to one control instruction when it is completed in a short time, and corresponds to another control instruction when it is executed continuously for a long time. In combination with the instruction mapping priority, when multiple hand movements are recognized at the same time, the movement with high priority and duration meeting the preset condition is mapped first, and the movement with low priority or duration not meeting the condition is queued or filtered according to the rule.

[0105] When selecting the transmission protocol of Bluetooth communication, the control instructions are classified by instruction type to obtain classified instructions. The control instructions can be divided into different types such as adjustment type instructions, switching type instructions, confirmation type instructions, etc. according to functions, and in the classification process, the control instructions corresponding to the feature vectors are classified into corresponding categories according to the operation object and execution effect of the instructions.

[0106] The real-time requirement of the classified instructions is extracted, and based on the real-time requirement, the protocol performance parameters of Bluetooth communication are collected. The real-time requirement reflects the sensitivity of the control instruction to transmission delay, for example, instructions such as adjusting volume and switching channel require fast response speed, and the real-time requirement is high; while the real-time requirement of instructions such as querying device status is relatively low. The protocol performance parameters include transmission rate, delay, power consumption, transmission distance and other indicators related to communication performance, which are obtained by querying the protocol specification document supported by the Bluetooth communication module.

[0107] The transmission rate and delay indicators in the protocol performance parameters are identified. The transmission rate refers to the amount of data that can be transmitted per unit of time, usually measured in bits per second; the delay indicator includes the time required for data to travel from the sending end to the receiving end, covering the time spent in data processing, signal transmission, and reception confirmation.

[0108] According to the transmission rate, the bandwidth occupancy rate of Bluetooth communication is analyzed. The formula for calculating the bandwidth occupancy rate is:

[0109]

[0110] where B represents the bandwidth occupancy rate, D represents the data volume of the control command, R represents the transmission rate, and T represents the transmission time. Through this formula, the proportion of bandwidth occupied by the transmission of a specific data volume of control command at a certain transmission rate can be calculated.

[0111] Based on the delay indicator, the response time range of Bluetooth communication is analyzed. The response time range refers to the time interval from the generation of a control command to the execution of the command at the receiving end. By combining the minimum delay and maximum delay in the delay indicator, the fluctuation range of the response time under different transmission protocols can be determined.

[0112] Based on the bandwidth occupancy rate, response time range, and real-time requirement, the transmission protocol for Bluetooth communication is selected. For classification commands with high real-time requirements, a transmission protocol with a small response time range and an acceptable bandwidth occupancy rate is selected to ensure that the command can be quickly transmitted and executed. For classification commands with low real-time requirements, a transmission protocol with low bandwidth occupancy rate or low power consumption can be selected to optimize the overall performance of the system. In the protocol selection process, other performance parameters such as transmission distance should also be considered to ensure that the selected protocol can meet the communication requirements of the actual use scenario.

[0113] Example 4: see Figure 4 When setting the anti-interference mode of Bluetooth communication, the communication distance interval of Bluetooth communication is determined based on the signal strength. The signal strength is obtained in real time by the Bluetooth communication module, and different signal strengths correspond to different distance ranges. For example, when the signal strength is between -30 dBm and -50 dBm, the corresponding communication distance interval is 0 to 5 meters; when the signal strength is between -50 dBm and -70 dBm, the communication distance interval is 5 to 10 meters; when the signal strength is lower than -70 dBm, the communication distance interval is more than 10 meters.

[0114] The distribution characteristics of the interference sources in the communication distance interval are analyzed. In the near distance interval of 0 to 5 meters, the interference sources are mostly other Bluetooth devices around, such as Bluetooth earphones, Bluetooth speakers and the like, which work in a similar frequency band to the remote control and are prone to signal overlap. In the medium distance interval of 5 to 10 meters, the interference sources may be not only Bluetooth devices but also WiFi signals, especially the 2.4 GHz frequency band WiFi signals, which have a high overlap with the interval. In the long distance interval of more than 10 meters, the interference sources may include microwave ovens, cordless phones and other household appliances, which release electromagnetic waves when working and thus interfere with the Bluetooth signals. As the distance increases, the signal itself attenuates more, and the influence of the interference is more obvious.

[0115] According to the interference type and the distribution characteristics of the interference sources, the vulnerable interference frequency band of the Bluetooth communication is identified, and the interference intensity of the vulnerable interference frequency band is identified. If the interference type is co-frequency interference, and the interference source is a nearby Bluetooth earphone, the vulnerable interference frequency band is concentrated near the channel commonly used by the Bluetooth earphone. If the interference source is a WiFi signal, the vulnerable interference frequency band overlaps with the channel occupied by the WiFi. The interference intensity is determined by monitoring the signal-to-noise ratio in the frequency band. The lower the noise ratio, the greater the interference intensity. For example, when the signal-to-noise ratio of a certain frequency band is lower than 10 dB, it can be determined that the frequency band is strongly interfered. When the signal-to-noise ratio is between 10 dB and 20 dB, it is moderately interfered. When the signal-to-noise ratio is higher than 20 dB, it is weakly interfered.

[0116] According to the signal intensity and the interference intensity, the frequency hopping strategy of the Bluetooth communication is set. When the signal intensity is strong and the interference intensity is weak, the hopping strategy is low-frequency hopping, that is, staying in the same channel for a long time to reduce the power consumption caused by hopping. When the signal intensity is moderate and the interference intensity is moderate, the medium-frequency hopping strategy is adopted, and the hopping interval is moderate to ensure the stability of the communication while taking into account the efficiency. When the signal intensity is weak and the interference intensity is strong, the high-frequency hopping strategy is enabled to shorten the residence time in each channel and quickly switch to a channel with less interference. The channel selection for hopping is based on a preset channel list, and the channel with weak interference intensity is preferentially selected for switching.

[0117] In combination with the interference intensity and the frequency hopping strategy, the anti-interference mode of the Bluetooth communication is set. For strong interference, on the basis of the high-frequency hopping strategy, a signal enhancement mechanism is enabled to offset the signal attenuation caused by the interference by increasing the transmission power. For moderate interference, the medium-frequency hopping strategy is adopted, and the channel filtering is used to filter out the signals in the interference frequency band. For weak interference, the low-frequency hopping strategy is adopted, and the energy-saving mode is turned on to reduce the power consumption of the device. After the anti-interference mode is set, the Bluetooth communication module will automatically switch to the corresponding mode according to the current interference situation to maintain the stability of the communication.

[0118] In identifying the user's gesture habit preference, historical gesture instruction data of the user is collected based on the anti-interference mode. In different anti-interference modes, the stability and response speed of Bluetooth communication are different, and the user's operation habit may change accordingly. For example, in a strong interference mode, the user may repeatedly perform a gesture to ensure that the instruction is correctly received, and these repeated gesture data will be recorded; in a weak interference mode, the user's operation is more fluent, and the continuity of the gesture data is better. The historical gesture instruction data includes the hand action feature vector of each operation, the corresponding control instruction, the execution time, the anti-interference mode, and other information.

[0119] According to the historical gesture instruction data, the high-frequency gesture type of the user is counted. All historical gesture instruction data is clustered according to the similarity of the feature vector, each cluster group represents a gesture type, the proportion of the number of instructions contained in each group to the total number of instructions is calculated, and the gesture type corresponding to the group with a higher proportion is the high-frequency gesture type. For example, in the play control, the number of times of the forward sliding gesture accounts for 30% of the total number of operations, and the number of times of the backward sliding gesture accounts for 25%. These two gestures are identified as high-frequency gesture types.

[0120] The action amplitude and execution speed of the high-frequency gesture type are identified. The action amplitude is measured by the maximum displacement of the key feature point in the hand action, for example, the displacement distance of the fingertip feature point in the horizontal direction in the forward sliding gesture is the action amplitude of the gesture; the execution speed is the ratio of the action amplitude to the time required to complete the action. For the high-frequency gesture of forward sliding, if the action amplitude is mostly between 5cm to 10cm and the execution speed is mostly between 2cm / s to 5cm / s in multiple operations, then these numerical ranges are the typical action amplitude and execution speed of the gesture.

[0121] Combining the action amplitude and execution speed, the gesture characteristic law of the user is analyzed. If the user's action amplitude gradually increases over time when performing the forward sliding gesture, while the execution speed remains relatively stable, this law will be recorded; if the action amplitude of the backward sliding gesture is small, but the execution speed is fast, and the speed change of each operation is not large, this will also become one of the gesture characteristic laws of the user.

[0122] Based on the high-frequency gesture type, the recognition threshold of the gesture habit preference is set. For the high-frequency gesture of forward sliding, the recognition threshold of the action amplitude is set to 3cm to 12cm, allowing a certain deviation of the actual action amplitude from the typical amplitude; the recognition threshold of the execution speed is set to 1cm / s to 6cm / s, covering the speed range of most operations. The setting of the recognition threshold will be adjusted according to the distribution range of the high-frequency gesture, to ensure that it can cover the normal operation deviation of the user.

[0123] According to the recognition threshold and the gesture feature rule, the gesture habit preference of the user is recognized. In the real-time recognition process, when the action amplitude and execution speed of the detected hand action fall within the recognition threshold range and conform to the gesture feature rule, it is determined that the action conforms to the gesture habit preference of the user. For example, the user is accustomed to first slow and then fast when sliding forward, if the speed change of the current gesture conforms to this rule and the amplitude is within the threshold, it is considered that the gesture conforms to the user's preference, and in subsequent instruction matching, this kind of gesture will be preferentially responded.

[0124] Embodiment 5: refer to Figure 5 In the optimization of the distribution strategy of feature weights, the user sensitive features in the effective feature set are determined according to the gesture habit preference. In the gesture habit preference, some features in the high-frequency execution gesture actions of the user will show higher stability and repeatability, and these features are the user sensitive features. For example, the user is accustomed to issuing a switching instruction by quickly waving the lower arm, so the speed parameter of the hand action and the trajectory feature of the lower arm movement will become the user sensitive features in the effective feature set.

[0125] The recognition accuracy of the user sensitive features is analyzed, and the weight adjustment coefficient of the user sensitive features is set based on the recognition accuracy. The recognition accuracy is obtained by comparing the actual value and the recognition value of the user sensitive features in the historical gesture instruction data, and the proportion of the number of coincidences of the two in the total number of recognitions in multiple recognitions is counted. When the recognition accuracy is high, a larger weight adjustment coefficient is set to enhance the influence of the feature in the feature vector; when the recognition accuracy is low, a smaller weight adjustment coefficient is set to temporarily reduce its influence, and it will be re-evaluated and adjusted after subsequent data accumulation.

[0126] According to the instruction matching model, the relevance of the user sensitive features and other features is recognized. The internal relationship between various features is stored in the instruction matching model, and by analyzing the feature association matrix in the model, the association degree of the user sensitive features and other features can be determined. If the user sensitive features and a certain other feature appear at the same time and have consistent change trends in multiple gesture recognitions, it means that the relevance is strong; if the timing and change trend of the two have no obvious rules, the relevance is weak.

[0127] Based on the relevance, the weight compensation coefficient of the other features is set. For the other features with strong relevance to the user sensitive features, a higher weight compensation coefficient is set to make these features play an auxiliary role when the recognition accuracy of the user sensitive features fluctuates; for the other features with weak relevance, a lower weight compensation coefficient is set to reduce the interference of these features on the overall feature weight distribution.

[0128] The distribution strategy of feature weights is optimized by combining the weight adjustment coefficient and the weight compensation coefficient. The basis weight of the user-sensitive feature is multiplied by the weight adjustment coefficient to obtain the adjusted weight value; the basis weight of the other features is multiplied by the corresponding weight compensation coefficient to obtain the compensated weight value. The proportion of each feature in the feature vector is redistributed according to the adjusted and compensated weight values to form a new feature weight distribution strategy.

[0129] When the hand action is subjected to instruction conversion processing and the control instruction is optimized, the signal stability of Bluetooth communication is determined according to the anti-interference mode. The anti-interference mode includes parameters such as signal fluctuation range and packet loss rate. By monitoring the real-time values of these parameters, it is determined whether the signal is stable. If the signal fluctuation is within the preset range and the packet loss rate is below a certain value, the signal is determined to be stable; if the signal fluctuation is severe or the packet loss rate exceeds the preset value, the signal is determined to be unstable.

[0130] Based on the signal stability, the trigger condition of the instruction mapping rule is set, and the instruction anti-shake mechanism of the hand action is set. When the signal is stable, the trigger condition is set to single recognition of the hand action conforming to the feature vector to start the instruction mapping; when the signal is unstable, the trigger condition is set to continuous multiple recognition of the same hand action to start the instruction mapping. The instruction anti-shake mechanism is realized by setting a time window. Within the time window, if the same or similar hand action is recognized multiple times, only the first recognized action is treated as an effective action to avoid repeated instructions caused by signal interference.

[0131] According to the trigger condition and the instruction anti-shake mechanism, the effectiveness of the feature vector is verified to obtain a verification result. In the verification process, it is checked whether the feature vector meets the requirements of the number of action recognitions in the trigger condition, and it is judged whether the feature vector is repeatedly recognized within the time window of the instruction anti-shake mechanism. If the feature vector meets the trigger condition and is not filtered by the anti-shake mechanism, the verification result is valid; if it does not meet the trigger condition or is filtered by the anti-shake mechanism, the verification result is invalid.

[0132] Based on the verification result and the distribution strategy, the hand action is subjected to instruction conversion processing to obtain the control instruction. For the feature vector with a valid verification result, the feature components with a high weight proportion are extracted according to the optimized feature weight distribution strategy, and these feature components are converted into corresponding control instruction codes combined with the instruction mapping rule. For the feature vector with an invalid verification result, no instruction conversion is performed and it is directly discarded. The converted control instruction will be formatted according to the requirements of the Bluetooth communication module to ensure that it can be correctly transmitted and executed.

[0133] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and illustrative figures, it should be apparent that the scope of the present application is not limited to these specific embodiments.

[0134] While the embodiments of the application have been shown and described herein, it is to be understood that the scope of the application, jointly pointed out in the appended claims, is not to be limited to the above-described embodiments but can be otherwise variously changed, modified, replaced, and altered within the principles and spirit of the present application.

Claims

1. A Bluetooth remote control based on gesture recognition, characterized in that, The method comprises the following steps: A gesture acquisition module is used to acquire user hand action data, determine the acquisition frequency and data dimension of the hand action data, acquire the space-time characteristics of the hand action data, and identify the motion trajectory and posture change of the hand action according to the space-time characteristics; A feature extraction module is used to extract key feature points of the hand action based on the motion trajectory and the posture change, locate the relative coordinates of the key feature points in the motion trajectory, and determine the feature vector of the hand action based on the space-time characteristics and the relative coordinates; An instruction mapping module is used to identify an effective feature set in the feature vector and a feature weight of the effective feature set, construct an instruction matching model of the effective feature set based on the feature weight, and set an instruction mapping rule of the hand action according to the instruction matching model and the feature weight; A Bluetooth communication module is used to combine the feature vector and the instruction mapping rule, encode the control instruction of the hand action in real time, select a transmission protocol of Bluetooth communication based on the control instruction, and set the data packet format of the Bluetooth communication in combination with the transmission protocol and the control instruction; An interference detection module is used to monitor the signal strength of the Bluetooth communication based on the data packet format, analyze the interference type of the Bluetooth communication according to the signal strength and the transmission protocol, and set an anti-interference mode of the Bluetooth communication in combination with the interference type and the signal strength; An adaptive learning module is used to identify the gesture habit preference of the user based on the anti-interference mode, optimize the distribution strategy of the feature weight according to the gesture habit preference and the instruction matching model, perform instruction conversion processing on the hand action according to the anti-interference mode, the distribution strategy and the instruction mapping rule, and optimize the control instruction; The method of optimizing the distribution strategy of the feature weight according to the gesture habit preference and the instruction matching model comprises the following steps: Determine the user sensitive feature in the effective feature set according to the gesture habit preference; Analyze the recognition accuracy of the user sensitive feature, and set a weight adjustment coefficient of the user sensitive feature based on the recognition accuracy; Identify the relevance of the user sensitive feature and other features according to the instruction matching model; Set a weight compensation coefficient of the other features based on the relevance; Optimize the distribution strategy of the feature weight in combination with the weight adjustment coefficient and the weight compensation coefficient.

2. The gesture recognition based Bluetooth remote control of claim 1, wherein, The method of determining the feature vector of the hand action based on the space-time characteristics and the relative coordinates comprises the following steps: Separate the dynamic characteristics and static characteristics of the hand action based on the space-time characteristics; Extract the speed parameter and acceleration parameter of the hand action according to the dynamic characteristics; Calculate the spatial distribution density of the key feature points based on the static characteristics and the relative coordinates; Determine the feature vector of the hand action in combination with the speed parameter, the acceleration parameter and the spatial distribution density.

3. The gesture recognition based Bluetooth remote control of claim 1, wherein, The method of constructing the instruction matching model of the effective feature set based on the feature weight comprises the following steps: Screening core feature components in the effective feature set based on the feature weights; Identifying standard instruction templates corresponding to the core feature components; Analyzing a similarity threshold of the standard instruction templates and the core feature components; Extracting feature variation trends of the effective feature set, and analyzing an influence degree of the feature variation trends on instruction matching according to the similarity threshold; According to the similarity threshold and the influence degree, constructing an instruction matching model of the effective feature set.

4. The gesture recognition based Bluetooth remote control of claim 1, wherein, According to the instruction matching model and the feature weights, setting instruction mapping rules of the hand action, including: Real-time comparing the feature vector with standard feature vectors in the instruction matching model; According to the instruction matching model, calculating a matching degree of the feature vector and the standard feature vectors; Scheduling an association relationship data of the feature weights and the matching degree, and determining an effectiveness threshold of the matching degree based on the association relationship data; According to the matching degree and the effectiveness threshold, setting an instruction mapping priority of the hand action; Identifying a duration of the hand action, and setting instruction mapping rules of the hand action based on the duration and the instruction mapping priority.

5. The gesture recognition based Bluetooth remote control of claim 1, wherein, According to the control instruction, selecting a transmission protocol of the Bluetooth communication, including: Classifying instruction types of the control instruction to obtain classified instructions; Extracting real-time requirements of the classified instructions, collecting protocol performance parameters of the Bluetooth communication based on the real-time requirements; Identifying transmission rates and delay indicators in the protocol performance parameters; According to the transmission rates, analyzing bandwidth occupancy rates of the Bluetooth communication; According to the delay indicators, analyzing response time ranges of the Bluetooth communication; Combining the bandwidth occupancy rates, the response time ranges, and the real-time requirements, selecting the transmission protocol of the Bluetooth communication.

6. The gesture recognition based Bluetooth remote control of claim 1, wherein, Combining the interference types and the signal strengths, setting an anti-interference mode of the Bluetooth communication, including: Based on the signal strengths, determining a communication distance interval of the Bluetooth communication; Analyzing interference source distribution characteristics in the communication distance interval; According to the interference types and the interference source distribution characteristics, identifying an interference-prone frequency band of the Bluetooth communication, and identifying an interference strength of the interference-prone frequency band; According to the signal strengths and the interference strength, setting a frequency hopping strategy of the Bluetooth communication; Combining the interference strength and the frequency hopping strategy, setting the anti-interference mode of the Bluetooth communication.

7. The gesture recognition based Bluetooth remote control of claim 1, wherein, Based on the anti-interference mode, identifying gesture habit preferences of the user, including: Based on the anti-interference mode, collecting historical gesture instruction data of the user; According to the historical gesture instruction data, counting high-frequency gesture types of the user; Identifying action amplitudes and execution speeds of the high-frequency gesture types, and analyzing gesture feature rules of the user by combining the action amplitudes and the execution speeds; Based on the high-frequency gesture types, setting an identification threshold of the gesture habit preferences; According to the identification threshold and the gesture feature rules, identifying the gesture habit preferences of the user.

8. The gesture recognition based Bluetooth remote control of claim 1, wherein, The instruction conversion processing of the hand action according to the anti-interference mode, the distribution strategy and the instruction mapping rule optimizes the control instruction, comprising: According to the anti-interference mode, the signal stability of the Bluetooth communication is confirmed; Based on the signal stability, the trigger condition of the instruction mapping rule is set, and the instruction anti-shake mechanism of the hand action is set; According to the trigger condition and the instruction anti-shake mechanism, the effectiveness of the feature vector is verified, and the verification result is obtained; Based on the verification result and the distribution strategy, the instruction conversion processing of the hand action is carried out, and the control instruction is obtained.

Images