Gesture recognition method and device, skirting line type warmer and medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a gesture recognition method and device, a skirting line type warmer and a medium. The infrared thermal imaging acquisition device and the ultrasonic acquisition device are controlled by using the gesture recognition result to acquire infrared thermal imaging data and ultrasonic data respectively, gesture recognition verification is carried out, and finally the gesture recognition result is obtained by integrating the radar gesture recognition result, the infrared thermal imaging verification result and the ultrasonic verification result. Gesture misjudgment caused by a single radar sensor is avoided, the complementarity and redundancy of multi-modal data are used for cross validation of gestures recognized by the radar, interference of the complex environment around the skirting line type warmer and other moving objects is avoided, the reliability and accuracy of gesture recognition results are improved, and the user experience is improved. Therefore, efficient and accurate interaction between the user and the skirting line type warmer is achieved, and the use experience of the user is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a gesture recognition method, device, baseboard heater, and medium. Background Art

[0002] With the rapid development of smart homes, more and more home appliances have begun to integrate human-computer interaction functions. At present, more and more home appliances can be intelligently controlled through user gestures, such as users controlling fans through gestures, which greatly improves the user's interactive experience.

[0003] In recent years, baseboard heaters have become increasingly popular, integrating more features such as smart displays and environmental monitoring. However, existing baseboard heaters are limited by their installation location and are often obstructed by children, pets, and other furniture. This makes accurate gesture recognition difficult, hindering efficient user interaction and impacting the user experience. Summary of the Invention

[0004] In view of this, the present invention provides a gesture recognition method, device, skirting board heater and medium to solve the problem in the related art that the skirting board heater is limited by the installation position and is difficult to accurately recognize user gestures, resulting in a poor user interaction experience.

[0005] In a first aspect, the present invention provides a gesture recognition method applied to a baseboard heater, wherein the baseboard heater is provided with: a radar acquisition device, an infrared thermal imaging acquisition device, and an ultrasonic acquisition device, the method comprising:

[0006] When the gesture recognition function of the baseboard heater is turned on, the radar collection device is controlled to collect radar data within the set area;

[0007] Performing gesture recognition based on the radar data to obtain a first recognition result of the gesture action, the first recognition result at least including: a recognized gesture action and a corresponding gesture action type;

[0008] Based on the number of gesture actions recognized in the first recognition result, controlling the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data respectively;

[0009] Based on the gesture action type, performing gesture recognition on the infrared thermal imaging data and the ultrasonic data respectively to obtain a first verification result and a second verification result of the gesture action;

[0010] A gesture recognition result is determined based on the first recognition result, the first verification result, and the second verification result.

[0011] The present invention sets a radar acquisition device in the skirting-board heater, and uses the radar data collected by the radar acquisition device to perform gesture recognition when the gesture recognition function is turned on, and uses the gesture recognition results to control the infrared thermal imaging acquisition device and the ultrasonic acquisition device set in the skirting-board heater to collect infrared thermal imaging data and ultrasonic data respectively, and then uses the infrared thermal imaging data and ultrasonic data to perform gesture recognition verification on the gesture action type identified by the radar data, and finally integrates the radar gesture recognition results, infrared thermal imaging verification results and ultrasonic verification results to obtain the final gesture recognition result, thereby avoiding gesture misjudgment caused by a single radar sensor, and cross-verifying the gesture recognized by the radar by using the complementarity and redundancy of multimodal data through infrared thermal imaging data and ultrasonic data, avoiding interference from the complex environment and other moving objects around the skirting-board heater, improving the reliability and accuracy of the gesture recognition results, and thus realizing efficient and accurate interaction between the user and the skirting-board heater, and improving the user experience.

[0012] In an optional implementation, performing gesture recognition based on the radar data includes:

[0013] Performing frequency domain conversion on the radar data to obtain frequency domain data;

[0014] Feature extraction is performed on the frequency domain data, and the extracted features are matched with a radar feature library to obtain a gesture action type of the gesture action, wherein the radar feature library stores radar feature data corresponding to different preset gesture action types.

[0015] The present invention converts radar data from the time domain to the frequency domain, utilizes the operating frequency characteristics of different gesture actions in the frequency domain, matches the extracted features with the standard radar feature library corresponding to different gesture action types, and obtains the gesture action type of the gesture action, thereby achieving efficient recognition of gesture action targets and improving the accuracy of radar recognition of gesture action targets.

[0016] In an optional embodiment, based on the number of gesture actions recognized in the first recognition result, controlling the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data respectively includes:

[0017] When the gesture action recognized in the first recognition result is a single one, the infrared thermal imaging acquisition device and the ultrasonic acquisition device are controlled to respectively acquire infrared thermal imaging data and ultrasonic data within the set area.

[0018] The present invention distinguishes the characteristic of radar that can track multiple targets simultaneously. When the radar identifies a single gesture action target, it indicates that there are no other suspected gesture actions in the set area. Directly controlling the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data in the set area can cover the single gesture action target identified by the radar, thereby improving data acquisition efficiency and accuracy.

[0019] In an optional embodiment, the method further includes:

[0020] When multiple gesture actions are recognized in the first recognition result, respectively obtaining a target position of each gesture action in the radar data;

[0021] Performing coordinate transformation on each target position to obtain a first position corresponding to the infrared thermal imaging acquisition device and a second position corresponding to the ultrasonic acquisition device;

[0022] The infrared thermal imaging acquisition device is controlled to collect data from the first position to obtain infrared thermal imaging data corresponding to each gesture action, and the ultrasonic acquisition device is controlled to collect data from the second position to obtain ultrasonic data corresponding to each gesture action.

[0023] When the radar identifies multiple gesture targets, the present invention indicates that there are multiple suspected gestures in the set area. By utilizing the characteristic of radar that can locate each target position, the target position of each target is determined separately, and then the infrared thermal imaging acquisition device and the ultrasonic acquisition device are controlled to directly collect data at the corresponding target position through coordinate conversion, so as to avoid the collected data being interfered with by other suspected gestures and affecting the accuracy of subsequent gesture verification, thereby further improving the accuracy of the final gesture recognition result.

[0024] In an optional embodiment, based on the gesture action type, gesture recognition is performed on the infrared thermal imaging data and the ultrasonic data respectively to obtain a first verification result and a second verification result of the gesture action, including:

[0025] Obtaining an infrared thermal imaging feature library and an ultrasonic feature library, wherein the infrared thermal imaging feature library stores infrared thermal imaging feature data corresponding to different preset gesture action types, and the ultrasonic feature library stores ultrasonic feature data corresponding to different preset gesture action types;

[0026] Extracting first feature data corresponding to the current gesture action type of the current gesture action from the infrared thermal imaging feature library, and extracting second feature data corresponding to the current gesture action type from the ultrasonic feature library;

[0027] Extracting features from the infrared thermal imaging data corresponding to the current gesture action type, and matching the extracted features with the first feature data to obtain a first verification result of the current gesture action;

[0028] Feature extraction is performed on the ultrasonic data corresponding to the current gesture action type, and the extracted features are matched with the second feature data to obtain a second verification result of the current gesture action.

[0029] The present invention establishes a standard infrared thermal imaging feature library and a standard ultrasonic feature library corresponding to different gesture action types, and then directly uses the standard library to perform feature matching with the real-time collected infrared thermal imaging data and ultrasonic data, thereby achieving concise and rapid verification of gesture actions, improving the efficiency of gesture recognition and verification, and further improving the efficiency of overall gesture recognition, thereby enhancing the user experience.

[0030] In an optional embodiment, the first recognition result, the first verification result, and the second verification result all include: a confidence level that the gesture action belongs to a gesture action type; and determining the gesture recognition result based on the first recognition result, the first verification result, and the second verification result includes:

[0031] Calculating a comprehensive confidence level that the current gesture action belongs to the current gesture action type based on the confidence level that the current gesture action belongs to the current gesture action type in the first recognition result, the first verification result, and the second verification result;

[0032] Determining whether the comprehensive confidence level is greater than a preset confidence threshold;

[0033] When the comprehensive confidence is greater than a preset confidence threshold, the gesture recognition result is determined to be the current gesture action type.

[0034] The present invention performs gesture recognition on multimodal data to obtain the confidence that the current gesture action belongs to the current gesture action type, calculates the comprehensive confidence that the current gesture action belongs to the current gesture action type, and determines the final gesture recognition result by means of comprehensive confidence value judgment. It fully utilizes the different characteristics of gesture actions reflected in different types of data, makes a comprehensive judgment of gesture actions, and further improves the accuracy of gesture recognition.

[0035] In an optional embodiment, the method further includes:

[0036] The baseboard heater is controlled to operate based on the gesture recognition result.

[0037] The present invention uses gesture recognition results to control the operation of the skirting board heater, thereby realizing the function of the user controlling the skirting board heater through gestures, improving the efficient and accurate interaction between the user and the skirting board heater, and enhancing the user's usage experience.

[0038] In an optional embodiment, before performing gesture recognition on the infrared thermal imaging data and the ultrasonic data based on the gesture action type, the method further includes:

[0039] Matching the infrared thermal imaging data and the ultrasonic data with a preset interference source feature library respectively, wherein the preset interference source feature library stores interference features corresponding to different preset interference sources;

[0040] When an interference feature matching any preset interference source exists in the infrared thermal imaging data or the ultrasonic data, the gesture recognition result is determined to be an invalid gesture.

[0041] The present invention establishes an interference library with corresponding features of different preset interference sources in advance, and directly matches the features of infrared thermal imaging data and ultrasonic data with the interference library to achieve rapid elimination of common interferences, thereby improving the efficiency of gesture recognition and further improving the accuracy of gesture recognition results.

[0042] In a second aspect, the present invention provides a gesture recognition system for a baseboard heater, wherein the baseboard heater is provided with: a radar acquisition device, an infrared thermal imaging acquisition device, and an ultrasonic acquisition device, and the system includes:

[0043] The first processing module is used to control the radar collection device to collect radar data within a set area when the gesture recognition function of the baseboard heater is turned on;

[0044] a second processing module, configured to perform gesture recognition based on the radar data to obtain a first recognition result of the gesture action, wherein the first recognition result at least includes: a recognized gesture action and a corresponding gesture action type;

[0045] a third processing module, configured to control the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data, respectively, based on the number of gesture actions recognized in the first recognition result;

[0046] A fourth processing module is configured to perform gesture recognition on the infrared thermal imaging data and the ultrasonic data based on the gesture action type, to obtain a first verification result and a second verification result of the gesture action;

[0047] A fifth processing module is configured to determine a gesture recognition result based on the first recognition result, the first verification result, and the second verification result.

[0048] In a third aspect, the present invention provides a baseboard heater, comprising: a radar acquisition device, an infrared thermal imaging acquisition device, and an ultrasonic acquisition device; the baseboard heater further comprises: a controller, the controller comprising:

[0049] A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 is a flowchart of a gesture recognition method according to an embodiment of the present invention;

[0053] Figure 2 is a flowchart of another gesture recognition method according to an embodiment of the present invention;

[0054] Figure 3 is a structural diagram of a baseboard heater according to an embodiment of the present invention;

[0055] Figure 4 is a structural block diagram of a gesture recognition system according to an embodiment of the present invention;

[0056] Figure 5 1 is a schematic diagram of the hardware structure of a controller of a baseboard heater according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0058] In recent years, with the rapid development of smart homes, baseboard heaters (hereinafter referred to as baseboard devices) have begun to integrate display functions, such as smart displays and environmental monitoring. However, existing baseboard devices still have many shortcomings in terms of human-computer interaction, such as the lack of intelligent gesture recognition, which prevents efficient interaction between users and devices.

[0059] In related technologies, although some skirting board devices use gesture recognition solutions based on cameras or radar sensors, these solutions generally have the following problems:

[0060] 1. The existing skirting board device has a single function and cannot realize the combination of intelligent control and environmental regulation.

[0061] 2. Limitations of complex scene recognition: A single radar is prone to misjudging gesture trajectories in environments with multiple people in motion and furniture obstruction. This is especially true in areas where children or pets are active, with a false touch rate as high as 23%.

[0062] Therefore, there is an urgent need for a method that can achieve intelligent control through advanced gesture recognition technology and reduce the misrecognition rate.

[0063] The gesture recognition solution provided by this embodiment utilizes data acquisition and algorithm fusion from multiple sensor models to accurately identify user gestures, even in extreme scenarios with multiple targets at the same distance and complex background motion. This significantly improves the accuracy of single-sensor recognition. Furthermore, by integrating and aligning the data across three sensor models, even when temporal and spatial alignment is difficult, the solution ultimately outputs a final gesture category based on a comprehensive confidence score, addressing the complexities of multi-sensor collaboration and data fusion.

[0064] According to an embodiment of the present invention, an embodiment of a gesture recognition method is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system, such as a set of computer-executable instructions. Moreover, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown.

[0065] This embodiment provides a gesture recognition method for a baseboard heater controller, such as an MCU or single-chip microcomputer. The baseboard heater is equipped with a radar acquisition device, an infrared thermal imaging acquisition device, and an ultrasonic acquisition device. The specific placement of each acquisition device can be flexibly determined based on the gesture recognition area requirements and product structure of the baseboard heater, but the present invention is not limited to this. Figure 1 is a flow chart of a gesture recognition method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0066] Step S101 : When the gesture recognition function of the baseboard heater is turned on, the radar collection device is controlled to collect radar data within a set area.

[0067] The set area is the area of ​​the baseboard heater that the user can control through gestures. This area can be flexibly set based on product requirements. For example, the set area is within 3 meters of the baseboard heater. This is only an example, and the present invention is not limited to this. The gesture recognition function is triggered when the user selects gesture control of the baseboard heater or activates the interactive function with the baseboard heater.

[0068] Specifically, in this embodiment of the present invention, the radar acquisition device is a 77GHz millimeter-wave radar array, used to collect radar data on gesture micro-Doppler characteristics and three-dimensional trajectory. It transmits 77GHz millimeter waves and receives the time-domain waveform of the target's reflected echo (e.g., sampling rate 10kHz, duration 200ms). In practical applications, the time-domain signal can be converted to the frequency domain using a fast Fourier transform (FFT) to extract micro-Doppler characteristics (e.g., the 50-150Hz frequency shift generated by arm movement). Based on the phase difference between three groups of transceiver units, the target's range (r, with an accuracy of ±5cm), azimuth (θ, with an accuracy of ±5°), and altitude (z, with an accuracy of ±10cm) are calculated. The collected radar data consists of: a time-frequency matrix: a 256×64 two-dimensional matrix (time axis × frequency axis) that records the frequency of the target's motion over time; and point cloud data: a set of three-dimensional coordinate points {(r1,θ1,z1),(r2,θ2,z2)...}, each accompanied by radial velocity (v, in m / s) and reflection intensity (dB).

[0069] Step S102: performing gesture recognition based on the radar data to obtain a first recognition result of the gesture action.

[0070] The first recognition result includes at least: a recognized gesture action and its corresponding gesture action type. For example, if the recognized gesture action is a hand-raising gesture, the corresponding gesture action type is a hand-raising gesture type; and if the recognized gesture action is a double-click gesture, the corresponding gesture action type is a double-click gesture type.

[0071] Specifically, gesture recognition can be performed directly on the radar data using an existing gesture recognition algorithm, or gesture recognition results can be obtained by extracting feature vectors from the original radar data and matching them with standard feature vectors corresponding to standard gesture actions. The present invention is not limited to this.

[0072] Step S103 : Based on the number of gestures recognized in the first recognition result, the infrared thermal imaging acquisition device and the ultrasonic acquisition device are controlled to respectively acquire infrared thermal imaging data and ultrasonic data.

[0073] Specifically, since radar can track multiple moving targets, the present invention uses radar data to identify the number of gesture movements to control the infrared thermal imaging acquisition device and the ultrasonic acquisition device to perform targeted data collection, thereby providing an accurate data basis for subsequent verification of each identified gesture target.

[0074] For example, in an embodiment of the present invention, the infrared thermal imaging acquisition device is an infrared thermal imaging array used to detect the thermal radiation coordinates of human joints. The raw data collected includes: thermal map data, specifically a 16×16 pixel temperature matrix (range: 20-40°C, accuracy: ±0.5°C), which records the distribution of human thermal radiation. In practical applications, a lightweight CNN network (such as MobileNetV2) is used to detect the locations (u, v) of key points such as the wrist, elbow, and shoulder, and their confidence levels (0-1) from the thermal map data to obtain the key point coordinates. The specific data formats include: a temperature matrix: a 16×16 floating-point matrix whose values ​​represent the temperature of the corresponding pixel; and a joint point sequence: {(u1, v1, c1), (u2, v2, c2)...}, where c is the confidence level (e.g., if the wrist joint coordinates move from (32, 48) to (32, 32), the confidence level increases from 0.8 to 0.95).

[0075] For example, in an embodiment of the present invention, the ultrasonic data acquisition device is an ultrasonic ranging sensor, which collects distance data by emitting 40kHz ultrasonic pulses and measuring the echo time to calculate the target distance (d, range 0.3-1.5m, accuracy ±2mm). It can also calculate the speed (v=d / t, unit m / s) through continuous sampling to obtain the distance change rate. The specific data format includes: time series: 100 distance sampling points (sampling rate 1kHz), such as {d1=0.8m, d2=0.78m...d 100 =0.82m}; Speed ​​curve: Speed ​​value calculated based on distance difference, such as {V1=0m / s, V2=0.1m / s...V 100 =-0.05m / s}.

[0076] Step S104 : Based on the gesture action type, gesture recognition is performed on the infrared thermal imaging data and the ultrasonic data respectively to obtain a first verification result and a second verification result of the gesture action.

[0077] Specifically, gesture recognition can be performed directly on the above-mentioned infrared thermal imaging data and ultrasonic data using an existing gesture recognition algorithm, or the corresponding gesture verification result can be obtained by extracting feature vectors from the infrared thermal imaging data and ultrasonic data and matching them with standard feature vectors corresponding to standard gesture actions. The present invention is not limited to this.

[0078] Step S105 : determining a gesture recognition result based on the first recognition result, the first verification result, and the second verification result.

[0079] Specifically, because the first recognition result, the first verification result, and the second verification result are the results of gesture recognition using three different types of data, the gesture identified in the first recognition result of radar data is used as the basis for verification of the gesture using infrared thermal imaging data and ultrasonic data, thereby improving the accuracy of the final gesture recognition result. For example, if all three results match the same gesture, the final gesture recognition result is that gesture. If the three results are not completely consistent or completely inconsistent, the final gesture recognition result is an invalid gesture, etc.

[0080] The embodiment of the present invention sets a radar acquisition device in the skirting-board heater, and uses the radar data collected by the radar acquisition device to perform gesture recognition when the gesture recognition function is turned on, and uses the gesture recognition results to control the infrared thermal imaging acquisition device and the ultrasonic acquisition device set in the skirting-board heater to collect infrared thermal imaging data and ultrasonic data respectively, and then uses the infrared thermal imaging data and ultrasonic data to perform gesture recognition verification on the gesture action type identified by the radar data, and finally integrates the radar gesture recognition results, infrared thermal imaging verification results and ultrasonic verification results to obtain the final gesture recognition result, thereby avoiding gesture misjudgment caused by a single radar sensor, and cross-verifying the gesture recognized by the radar by using the complementarity and redundancy of multimodal data through infrared thermal imaging data and ultrasonic data, avoiding interference from the complex environment and other moving objects around the skirting-board heater, and improving the reliability and accuracy of the gesture recognition results, thereby realizing efficient and accurate interaction between the user and the skirting-board heater, and improving the user experience.

[0081] In this embodiment, a gesture recognition method is also provided, which is applied to a controller of a skirting-board heater, such as an MCU, a single-chip microcomputer, or other control chip. The skirting-board heater is provided with a radar acquisition device, an infrared thermal imaging acquisition device, and an ultrasonic acquisition device. Figure 2 is a flow chart of a gesture recognition method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0082] Step S201: When the baseboard heater turns on the gesture recognition function, the radar collection device is controlled to collect radar data within the set area. Figure 1 The description of step S101 is omitted here.

[0083] Step S202: Perform gesture recognition based on the radar data to obtain a first recognition result of the gesture. The first recognition result includes at least: the recognized gesture and its corresponding gesture type.

[0084] Specifically, the above step S202 specifically includes:

[0085] Step S2021: Perform frequency domain conversion on the radar data to obtain frequency domain data.

[0086] In practical applications, the time domain signal can be converted into the frequency domain through the Fast Fourier Transform (FFT).

[0087] Step S2022: extract features from the frequency domain data, and match the extracted features with the radar feature library to obtain the gesture action type of the gesture action.

[0088] The radar feature library stores radar feature data corresponding to different preset gesture action types.

[0089] Exemplarily, by performing feature extraction on frequency domain data, a 16-dimensional feature vector is extracted, including: frequency shift mean, variance, peak frequency, energy distribution, velocity change rate, etc., and the radar feature is formed by the 16-dimensional feature vector. Then, in the radar feature library, 16-dimensional standard feature vectors corresponding to different preset gesture action types are stored. Then, by matching the above-mentioned radar feature with the 16-dimensional standard feature vector, if the radar feature has a matching 16-dimensional standard feature vector in the radar feature library, the preset gesture action type corresponding to the 16-dimensional standard feature vector is used as the gesture action identified in the first recognition result. In actual applications, there may be one or more features corresponding to the preset gesture action types in the radar feature library that match, and the corresponding gesture action identified in the first recognition result may also be one or more.

[0090] The embodiment of the present invention converts radar data from the time domain to the frequency domain, utilizes the operating frequency characteristics of different gesture actions in the frequency domain, matches the extracted features with the standard radar feature library corresponding to different gesture action types, and obtains the gesture action type of the gesture action, thereby achieving efficient recognition of gesture action targets and improving the accuracy of radar recognition of gesture action targets.

[0091] Step S203 : Based on the number of gestures recognized in the first recognition result, the infrared thermal imaging acquisition device and the ultrasonic acquisition device are controlled to respectively acquire infrared thermal imaging data and ultrasonic data.

[0092] Specifically, the above step S203 includes:

[0093] In step a1, when the gesture action recognized in the first recognition result is a single one, the infrared thermal imaging acquisition device and the ultrasonic acquisition device are controlled to respectively acquire infrared thermal imaging data and ultrasonic data within a set area.

[0094] The embodiment of the present invention distinguishes the characteristic of radar that can track multiple targets simultaneously. When the radar recognizes a single gesture action target, it indicates that there are no other suspected gesture actions in the set area. Directly controlling the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data in the set area can cover the single gesture action target recognized by the radar, thereby improving data acquisition efficiency and accuracy.

[0095] Step a2: When multiple gestures are recognized in the first recognition result, the target position of each gesture in the radar data is obtained respectively.

[0096] Specifically, since the radar can track multiple targets, the target position corresponding to each recognized gesture action in the radar data can be realized through existing technology. For example, the target position is the radar polar coordinate, which will not be repeated here.

[0097] Step a3: performing coordinate transformation on each target position to obtain a first position corresponding to the infrared thermal imaging acquisition device and a second position corresponding to the ultrasonic acquisition device.

[0098] Specifically, the radar polar coordinates can be converted to Cartesian coordinates, and then converted from Cartesian coordinates to the corresponding acquisition position coordinates of the infrared thermal imaging acquisition device and the ultrasonic acquisition device. The specific coordinate conversion method is existing technology and will not be detailed here. This achieves the unification of spatial coordinates, ensuring that the data collected by the three devices is from the same physical space, providing an accurate data foundation for subsequent gesture recognition verification.

[0099] Step a4: Control the infrared thermal imaging acquisition device to collect data at the first position to obtain infrared thermal imaging data corresponding to each gesture action, and control the ultrasonic acquisition device to collect data at the second position to obtain ultrasonic data corresponding to each gesture action.

[0100] In practical applications, when controlling the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect data, it is also necessary to time-align the infrared thermal imaging acquisition device, the ultrasonic acquisition device, and the radar acquisition device to ensure that the data collected by the three are data of the same time period, so as to avoid the target moving over time and affecting the recognition results. For example, the three acquisition devices can share the same high-precision clock source (such as the RTC module of the MCU, with an accuracy of ±1μs) to ensure that the sampling trigger time is consistent. Alternatively, the radar and ultrasonic waves use a pulse trigger mechanism, and the infrared is sampled at a fixed frame rate (30fps), allowing a time difference of ±5ms. In addition, software compensation can also be used: using the dynamic time warping (DTW) algorithm to process asynchronous data: establishing a time mapping function τ(t), aligning the radar's 200ms time-frequency matrix with the infrared's 6-frame thermal map (200ms / 33ms≈6 frames), etc.;

[0101] In an embodiment of the present invention, when the radar identifies multiple gesture targets, it indicates that there are multiple suspected gestures in the set area. By utilizing the characteristic of the radar that can locate the position of each target, the target position of each target is determined separately. Then, through coordinate conversion, the infrared thermal imaging acquisition device and the ultrasonic acquisition device are controlled to directly collect data at the corresponding target position, so as to avoid the collected data being interfered with by other suspected gestures and affecting the accuracy of subsequent gesture verification, thereby further improving the accuracy of the final gesture recognition result.

[0102] Step S204 : Based on the gesture action type, gesture recognition is performed on the infrared thermal imaging data and the ultrasonic data respectively to obtain a first verification result and a second verification result of the gesture action.

[0103] Specifically, the above step S204 includes:

[0104] Step S2041, obtaining an infrared thermal imaging feature library and an ultrasonic feature library.

[0105] The infrared thermal imaging feature library stores infrared thermal imaging feature data corresponding to different preset gesture action types, and the ultrasonic feature library stores ultrasonic feature data corresponding to different preset gesture action types.

[0106] For example, infrared thermal imaging feature data includes wrist vertical displacement, elbow angle change, Euclidean distance between key points, and temperature gradient. These features form a feature vector as the infrared thermal imaging feature data. Furthermore, ultrasonic feature data includes average distance, maximum velocity, and acceleration rate of change. Ultrasonic feature data is formed from a three-dimensional feature vector.

[0107] Step S2042: extracting first feature data corresponding to the current gesture action type of the current gesture action from the infrared thermal imaging feature library, and extracting second feature data corresponding to the current gesture action type from the ultrasonic feature library.

[0108] Step S2043 , extracting features from the infrared thermal imaging data corresponding to the current gesture action type, and matching the extracted features with the first feature data to obtain a first verification result of the current gesture action.

[0109] Specifically, the feature type extracted from the infrared thermal imaging data is the same as the infrared thermal imaging feature data type in the infrared thermal imaging feature library. The specific feature extraction method is an existing technology and will not be described in detail here.

[0110] Step S2044: extract features from the ultrasonic data corresponding to the current gesture action type, and match the extracted features with the second feature data to obtain a second verification result of the current gesture action.

[0111] Specifically, the feature type extracted from the ultrasonic data is the same as the ultrasonic feature data type in the ultrasonic feature library. The specific feature extraction method is an existing technology and will not be described in detail here.

[0112] The embodiment of the present invention establishes a standard infrared thermal imaging feature library and a standard ultrasonic feature library corresponding to different gesture action types, thereby directly using the standard library to perform feature matching with real-time collected infrared thermal imaging data and ultrasonic data, thereby achieving simple and rapid verification of gesture actions, improving the efficiency of gesture recognition and verification, and further improving the efficiency of overall gesture recognition, thereby enhancing the user experience.

[0113] Step S205 : determining a gesture recognition result based on the first recognition result, the first verification result, and the second verification result.

[0114] The first recognition result, the first verification result, and the second verification result all include: the confidence level that the gesture action belongs to the gesture action type. Specifically, the above step S105 includes:

[0115] Step S2051 : Calculate the comprehensive confidence that the current gesture belongs to the current gesture type based on the confidence that the current gesture belongs to the current gesture type in the first recognition result, the first verification result, and the second verification result.

[0116] Specifically, the comprehensive confidence can be obtained by multiplying the confidence that the current gesture action belongs to the current gesture action type in the first recognition result, the first verification result and the second verification result. It can also be obtained by setting weights corresponding to the first recognition result, the first verification result and the second verification result, and calculating the comprehensive confidence by taking a weighted average of the confidences of each result. The present invention only takes this as an example and is not limited to this.

[0117] Step S2052: determine whether the comprehensive confidence is greater than a preset confidence threshold.

[0118] The preset confidence threshold is flexibly set according to the gesture recognition accuracy set by the product and actual needs. For example, the preset confidence threshold is 0.75. This is only an example and the present invention is not limited thereto.

[0119] Step S2053: When the comprehensive confidence is greater than the preset confidence threshold, the gesture recognition result is determined to be the current gesture action type.

[0120] For example, if the current gesture action type is hand raising, when the comprehensive confidence corresponding to the hand raising gesture action is 0.9, it means that the user is indeed performing a hand raising action, and the gesture recognition result is determined to be hand raising.

[0121] Specifically, in actual applications, when the comprehensive confidence is not greater than the preset confidence threshold, the gesture recognition result is determined to be an invalid gesture. It may be due to a moving animal or child, or furniture of a special shape blocking the radar recognition as a suspected gesture. Since the comprehensive confidence is lower than 0.75, the gesture recognition result is considered to be an invalid gesture, thereby eliminating interference and improving gesture recognition accuracy.

[0122] The embodiment of the present invention performs gesture recognition on multimodal data to obtain the confidence that the current gesture action belongs to the current gesture action type, calculates the comprehensive confidence that the current gesture action belongs to the current gesture action type, and determines the final gesture recognition result by means of a numerical judgment of the comprehensive confidence level. This fully utilizes the different characteristics of gesture actions reflected in different types of data, performs a comprehensive judgment of gesture actions, and further improves the accuracy of gesture recognition.

[0123] In some optional implementations, the gesture recognition method provided by the embodiment of the present invention further includes the following steps:

[0124] Step S206: Control the operation of the baseboard heater based on the gesture recognition result.

[0125] Specifically, the skirting board heater is designed with different control methods corresponding to different gestures. For example, the hand-raising action indicates controlling the skirting board heater to increase the gear, and the hand-clapping action indicates controlling the skirting board heater to shut down, etc. This is just an example, and the present invention is not limited to this.

[0126] The embodiment of the present invention uses gesture recognition results to control the operation of the skirting board heater, thereby realizing the function of the user controlling the skirting board heater through gestures, improving the efficient and accurate interaction between the user and the skirting board heater, and enhancing the user's usage experience.

[0127] In some optional implementations, before executing step S204, the gesture recognition method provided by the embodiment of the present invention further includes the following steps:

[0128] Step b1: Match the infrared thermal imaging data and ultrasonic data with a preset interference source feature library respectively.

[0129] The preset interference source feature library stores interference features corresponding to different preset interference sources. The interference features can be infrared thermal imaging features or ultrasonic features. For example, the infrared thermal imaging temperature of a pet is different from that of a human body, so the infrared thermal imaging temperature of the pet can be used as an interference feature. This is merely an example and is not intended to be limiting.

[0130] The preset interference source can be other interferences in the environment that are easily recognized as gestures, such as pets, children, furniture, etc., or it can be interference movements that are mistakenly recognized as gestures during exercise by the user, such as: the user waving his arms while dancing, raising his hands when picking up an object, etc. The present invention is not limited to this.

[0131] Step b2: When there is an interference feature matching any preset interference source in the infrared thermal imaging data or the ultrasonic data, determining that the gesture recognition result is an invalid gesture.

[0132] If the infrared thermal imaging data or CC wave data has interference characteristics consistent with the interference source, it means that the gesture recognized by the radar is the interference source. The gesture is directly determined to be an invalid gesture to improve the overall gesture recognition efficiency.

[0133] The embodiments of the present invention establish an interference library with corresponding features of different preset interference sources in advance, and directly match the features of infrared thermal imaging data and ultrasonic data with the interference library to achieve rapid elimination of common interferences, thereby improving the efficiency of gesture recognition and further improving the accuracy of gesture recognition results.

[0134] The gesture recognition solution provided by the embodiment of the present invention will be described in detail below with reference to specific application examples.

[0135] For example, Figure 3 As shown, the skirting board heater provided by the embodiment of the present invention includes: a skirting board body 1 and a display panel 2 arranged on the skirting board body, the display panel 2 is integrated with the above-mentioned radar acquisition device, infrared thermal imaging acquisition device and ultrasonic acquisition device, and a lifting motor 11 is arranged inside the skirting board body 1 and is connected to the air outlet assembly 12 on the skirting board body 1 to support the up and down movement of the air outlet assembly 12, and they are uniformly controlled by the control unit.

[0136] It should be noted that the final judgment in the embodiment of the present invention is based on "confidence" (ranging from 0 to 1). Using the data returned by the three acquisition devices and the aforementioned pre-set feature databases, the confidence level of each acquisition device's corresponding recognition or verification result is determined. Ultimately, the reliability of the gesture is determined based on the overall confidence level (which must be > 0.75).

[0137] For example, if the radar data matches the data feature template for the "hand-raising" action in the preset standard gesture library, the output confidence level is 0.85 (this is a preset value; for example, hand-raising is 0.85, double-clicking is 0.91, and waving is 0.95. The easier the action is to recognize, the higher the confidence level). If the infrared temperature is 20-23°C, below the human detection threshold of 35°C, the output confidence level is 0. If the ultrasonic detection target distance is 1 meter, within the preset distance range, and the speed is 0, the output confidence level is 0.75. The overall confidence level is 0.53 < 0.75. Therefore, this action is judged as interference and an invalid gesture. In fact, this action is produced by a lucky cat toy, and in experiments using only radar detection, the misidentification rate is as high as 40%.

[0138] It should be noted that in actual applications, an interference source database can be set up, and the returned confidence is also divided into standard gesture confidence and interference source confidence. If the interference source confidence is greater than a certain value, it is directly determined to be interference without gesture recognition.

[0139] The following three cases are introduced, ranging from simple to complex, to illustrate the advantages of the embodiments of the present invention:

[0140] Case 1: A single user raises their hand within the detection range.

[0141] 1. Data collection:

[0142] ① Radar: Detects the hand moving from (0.8m, 0°, 0.5m / s) to (1.2m, 30°, 0.8m / s), generating a 100ms trajectory sequence;

[0143] ② Infrared thermal imaging: The coordinates of the wrist joint point were identified to move from (20px, 35px) to (45px, 15px), and the confidence level increased from 0.6 to 0.9;

[0144] ③ Ultrasonic: The distance increases from 0.7m to 1.1m, and the peak value of the speed curve is 0.9m / s.

[0145] 2. Fusion decision-making process:

[0146] Radar evidence: Hand-raising confidence level 0.85 (micro-Doppler feature matching standard gesture library);

[0147] Infrared evidence: The joint motion trajectory matches the "hand-raising" gesture, with a confidence level of 0.92;

[0148] Ultrasonic evidence: distance changes consistent with vertical motion pattern, confidence level 0.78;

[0149] 3. Fusion results:

[0150] The comprehensive confidence level is 0.94>0.75, and it is determined to be a “hand-raising” gesture.

[0151] Execution Control:

[0152] The command is output to the lifting motor 11 of the baseboard heater to drive the air outlet of the baseboard heater to rise.

[0153] Case 2: The user stands 0.8 m in front of the skirting board device and performs the "make a fist → quickly double-click twice" gesture. At the same time, the robot vacuum passes by 1.5 m to the right (moving speed 0.3 m / s, closest to the device 1.2 m).

[0154] The system needs to accurately recognize gestures under the following interference conditions:

[0155] Radar interference: The metal shell of the robot vacuum cleaner generates strong radar echoes, which may mask the micro-Doppler signature of the hand;

[0156] Infrared interference: The heating of the robot motor (about 35°C) may be mistakenly identified as a heat source from the human body;

[0157] Ultrasonic interference: Robot movement increases the complexity of environmental echoes, which may trigger false touches.

[0158] 1. Data collection:

[0159] ① Radar:

[0160] Signal 1: When making a fist, the hand is still (Doppler shift ≈ 0 Hz). When double-clicking, the hand opens and closes rapidly (frequency shift ±150 Hz, duration < 200 ms), forming a characteristic waveform of "0 Hz → positive and negative pulses → 0 Hz". Three-dimensional trajectory: When double-clicking, the hand vibrates rapidly in the XY plane (displacement < 10 cm), and the Z-axis height is maintained at 0.9-1.1 m.

[0161] Signal 2: Uniform movement (frequency shift ≈ 30 Hz), no pulse-like changes; 3D trajectory: translation along the Y axis (speed 0.3 m / s), Z axis height 0.3 m (below hand height).

[0162] Preprocessing: Filter the robot's low-frequency continuous echoes through sliding window denoising (window 50ms); extract trajectory confidence: Signal 1 trajectory conforms to the "fixed height + high-frequency vibration" pattern (confidence 0.92), and Signal 2 trajectory is "low height + linear movement" (confidence 0.21).

[0163] ②Infrared thermal imaging:

[0164] Signal 1: Joint point detection: The wrist joint coordinates when making a fist are (45px, 28px). When double-clicking, the wrist joint quickly opens to (52px, 22px) and then closes, forming a coordinate sequence of "gathering → separating → gathering"; thermal image features: The hand temperature is 32°C, which is higher than the ambient temperature (24°C). The thermal imaging confidence map shows continuous activation of the wrist area (confidence 0.89 → 0.95 → 0.89).

[0165] Signal 2: Heat source location: The motor area temperature is 35°C, but the shape is a regular rectangle, irrelevant to the node features; dynamic mask filtering is used.

[0166] ③Ultrasound:

[0167] Signal 1: During a double-click, the distance suddenly decreases from 0.8m to 0.75m (first clap), then returns to 0.8m, and repeats after a 50ms interval (second clap), forming a pulse sequence of "0.8→0.75→0.8→0.75→0.8m"; speed calculation: peak speed 1.0m / s (higher than the robot's 0.3m / s).

[0168] Signal 2: Smooth distance change: from 1.5m → 1.2m → 1.5m, stable speed 0.3m / s, no pulse-like fluctuations; threshold filtering: The robot signal is excluded through the distance-speed joint threshold (the preset distance > 1.0m and speed < 0.5m / s are considered as background).

[0169] 2. Fusion decision-making process:

[0170] ① Space-time alignment processing.

[0171] Time alignment: The timestamp error of radar, infrared, and ultrasonic signals is less than 1μs. Double-click events are synchronized through dynamic time warping (DTW) (allowing a ±5ms tolerance) to confirm that there is no time overlap between hand movements and robot movements.

[0172] Spatial Calibration:

[0173] Signal 1 coordinates: radar (0.8m, 30°, 1.0m) → Cartesian coordinates (0.69m, 0.40m, 1.0m);

[0174] Signal 2 coordinates: radar (1.2m, 60°, 0.3m) → Cartesian coordinates (0.60m, 1.04m, 0.3m). Z-axis height separation from the hand is greater than 0.5m, eliminating spatial overlap interference.

[0175] Decision-level fusion:

[0176] Signal 1 evidence:

[0177] Radar: "Double click" confidence level 0.91, "Robot interference" confidence level 0.05;

[0178] Infrared: "Hand motion" confidence level 0.88, "Ambient heat source" confidence level 0.07;

[0179] Ultrasonic: "Rapid distance change" confidence level 0.93, "background motion" confidence level 0.04;

[0180] Fusion result: The total confidence of the "double-click gesture" is 0.97> the threshold value 0.75, and it is judged to be valid.

[0181] Signal 2 evidence: Radar, infrared, and ultrasonic waves do not support the "gesture" category, with an overall confidence score of <0.2, and are considered interference and filtered out.

[0182] Execution control: Close the air outlet of the baseboard heater.

[0183] Case 3: A user stands 1 meter in front of the device and performs a hand-raising motion (arm raised from waist to shoulder for 1.2 seconds). One meter to the right of the device, another person dances (high-frequency, complex movements, including waving and turning). Three meters from the device, curtains sway in the wind (low-frequency swinging, approximately 20 cm).

[0184] Key challenges:

[0185] 1. The radar needs to distinguish between two moving targets at the same distance but different directions;

[0186] 2. Infrared thermal imaging needs to recognize the user's arm movements to avoid confusion with the multi-joint movements of a dancer;

[0187] 3. Filter out weak radar echoes and possible false triggers from distant curtains.

[0188] 1. Interference filtering mechanism of multimodal sensors:

[0189] 1. Millimeter-wave radar: spatial resolution and micro-Doppler feature separation;

[0190] Three-dimensional positioning and azimuth differentiation: The radar array uses phase difference triangulation to calculate the azimuth difference between the user and the dancer:

[0191] Signal 1: azimuth θ = 0° (straight ahead), distance r = 1.0 m, height z = 1.1 m;

[0192] Signal 2: azimuth angle θ = 45° (right side), distance r = 1.0 m, height z = 1.6 m (including vertical displacement due to turning);

[0193] Signal 3: azimuth angle θ = 30° (left side), distance r = 3.0 m, height z = 1.5 m (weak signal, low frequency < 5 Hz).

[0194] Micro-Doppler feature separation:

[0195] User raises hand: linear motion in a single direction (Doppler shift increases from 0 Hz to 50 Hz, corresponding to a speed of 0.5 m / s);

[0196] Dance movements: multi-directional compound movements (frequency shift includes multiple components such as 20Hz and 80Hz, corresponding to movements such as waving hands and stomping feet);

[0197] The low-frequency linear motion features of the user are extracted through a bandpass filter (passband 20-60 Hz), and the high-frequency components of the dancer and the extremely low-frequency components (<5 Hz) of the curtain are suppressed.

[0198] 2. Infrared thermal imaging: human body region segmentation and motion pattern matching;

[0199] Heat source area division: The infrared array detected two independent heat sources:

[0200] Signal 1: Wrist joint coordinates (32px, 48px) → (32px, 32px) (vertically moved up 16px), the elbow joint remains stable;

[0201] Signal 2: Dynamic changes in multiple joints (significant fluctuations in wrist, elbow, and hip coordinates);

[0202] Signal 3: The temperature is lower than the human body detection threshold and is judged as an interference item.

[0203] Motion pattern recognition: Signal 1 motion matches the "single-joint vertical motion" model (confidence 0.91), and Signal 2 matches the "multi-joint compound motion" model (confidence 0.85), triggering the interference source mark; applying a spatial mask: only retaining the ROI directly in front of Signal 1 (azimuth ±15°, height 0.8-1.3m), and shielding the Signal 2 area on the right.

[0204] 3. Ultrasonic module:

[0205] Distance threshold and dynamic environment modeling.

[0206] Effective interaction area limitation: Ultrasonic wave is set to have an effective detection range of 0.3-1.5m. If the signal 3 at 3 meters is out of range, its distance data will be directly filtered out.

[0207] For dual targets at 1 meter:

[0208] Signal 1: Echo phase is stable (single motion on a stationary basis);

[0209] Signal 2: The echo phase changes rapidly (high-frequency motion causes multipath effect);

[0210] Output dynamic mask: only signal 1 1 meter in front is marked as a valid target (mask weight 1.0), the mask weight of signal 2 on the right is reduced to 0.3, and the weight of signal 3 is 0.0.

[0211] 2. Data fusion decision-making process:

[0212] 1. Spatiotemporal alignment and interference feature suppression.

[0213] Time synchronization:

[0214] The sensor synchronization clock error is less than 1μs. The time windows of signal 1, hand raising (t = 0-1.2s), signal 2, action (t = 0-3s), and curtain swing (continuous) partially overlap, but the event trigger mechanism only captures the start frame of signal 1 action (t = 0.1s).

[0215] Spatial calibration: Convert the radar polar coordinates (r, θ, z), infrared pixels (u, v), and ultrasonic distance d to Cartesian coordinates (X, Y, Z). Ensure that Signal 1 and Signal 2 are separated by 0.7 m on the Y axis (Signal 1 Y = 0 m, Signal 2 Y = 0.7 m) to avoid spatial confusion.

[0216] 2. Hierarchical feature fusion and weight allocation.

[0217] Decision-level fusion:

[0218] Signal 1 evidence: Radar: "Single target linear motion" confidence level 0.88; Infrared: "Single joint hand raise" confidence level 0.92; Ultrasonic: "Stable target within effective range" confidence level 0.95; Overall confidence level: 0.96 > threshold 0.75, determined to be a valid gesture;

[0219] Interference source evidence: Signal 2: Feature inconsistency results in an overall confidence level of 0.45 less than the threshold; Signal 3: Radar evidence is single and there is no infrared response, with an overall confidence level of 0.12 less than the threshold.

[0220] Final execution: Raise your hand to drive the air outlet of the skirting heater to rise.

[0221] Additional explanation:

[0222] Differentiate multiple targets at the same distance:

[0223] 1. Improved azimuth resolution of the radar array: By calculating the phase difference between three sets of transceiver units, the azimuth resolution reaches 5°. This allows the system to distinguish between two targets 0.17m apart (1m × sin 5°) at a distance of 1 meter, meeting the spatial separation requirements between the user and the dancer (actual distance 0.7m > 0.17m).

[0224] 2. Dynamic tracking with infrared thermal imaging: A mean-shift algorithm is used to track the user's wrist joint and establish a dynamic region of interest. Even if a dancer enters the same distance zone, tracking stability can be maintained using the initially calibrated user position (directly in front).

[0225] 3. Logical thresholds for multimodal evidence: Joint detection rules are set: gesture recognition is triggered only when radar (correct azimuth), infrared (matched joint points), and ultrasonic (stable distance) conditions are simultaneously met, preventing misjudgment by a single sensor. Gesture recognition accurately outputs the "raise your hand" command, triggering the air vent to rise. Interference suppression results: 1. The dancer's movements did not trigger any false operations; 2. The curtain swing was not misjudged as human movement by the radar.

[0226] 4. Performance: In complex interference scenarios, this solution achieved a recognition accuracy of 98.3% and a response delay of 120ms (from the start of the action to the output of the command). The misjudgment rate was as high as 53% when using a single radar.

[0227] Summary: The present invention utilizes radar's spatial resolution (three-dimensional filtering of azimuth, range, and altitude), infrared thermal imaging's semantic understanding of the human body (modeling joint motion), and ultrasonic interaction zone definition to construct a three-level anti-interference system: "physical space segmentation → feature semantic filtering → evidence logic verification." Even in extreme scenarios with multiple targets at the same distance and complex background motion, the complementarity and redundancy of multimodal data enable high-reliability gesture recognition, overcoming the performance bottlenecks of traditional single-sensor solutions faced by "co-channel interference" and "background confusion," thereby improving gesture recognition accuracy.

[0228] This embodiment also provides a gesture recognition system, which is applied to a skirting-type heater. The skirting-type heater is provided with: a radar acquisition device, an infrared thermal imaging acquisition device and an ultrasonic acquisition device, such as Figure 4 As shown, the gesture recognition system includes:

[0229] The first processing module 401 is used to control the radar collection device to collect radar data within a set area when the gesture recognition function of the baseboard heater is turned on;

[0230] The second processing module 402 is configured to perform gesture recognition based on the radar data to obtain a first recognition result of the gesture action, wherein the first recognition result at least includes: the recognized gesture action and its corresponding gesture action type;

[0231] The third processing module 403 is used to control the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data respectively based on the number of gesture actions recognized in the first recognition result;

[0232] A fourth processing module 404 is configured to perform gesture recognition on the infrared thermal imaging data and the ultrasonic data based on the gesture action type, and obtain a first verification result and a second verification result of the gesture action;

[0233] The fifth processing module 405 is configured to determine a gesture recognition result based on the first recognition result, the first verification result, and the second verification result.

[0234] In some optional implementations, the second processing module 402 includes:

[0235] A first processing unit is used to perform frequency domain conversion on the radar data to obtain frequency domain data;

[0236] The second processing unit is used to extract features from the frequency domain data and match the extracted features with the radar feature library to obtain the gesture action type of the gesture action. The radar feature library stores radar feature data corresponding to different preset gesture action types.

[0237] In some optional implementations, the third processing module 403 includes:

[0238] The fourth processing unit is configured to control the infrared thermal imaging acquisition device and the ultrasonic acquisition device to respectively acquire infrared thermal imaging data and ultrasonic data within a set area when the gesture action recognized in the first recognition result is a single one.

[0239] In some optional implementations, the third processing module 403 further includes:

[0240] a fifth processing unit, configured to obtain a target position of each gesture in the radar data when multiple gestures are recognized in the first recognition result;

[0241] A sixth processing unit, configured to perform coordinate conversion on each target position to obtain a first position corresponding to the infrared thermal imaging acquisition device and a second position corresponding to the ultrasonic acquisition device;

[0242] The seventh processing unit is used to control the infrared thermal imaging acquisition device to collect data at the first position to obtain infrared thermal imaging data corresponding to each gesture action, and control the ultrasonic acquisition device to collect data at the second position to obtain ultrasonic data corresponding to each gesture action.

[0243] In some optional implementations, the fourth processing module 404 includes:

[0244] an eighth processing unit, configured to obtain an infrared thermal imaging feature library and an ultrasonic feature library, wherein the infrared thermal imaging feature library stores infrared thermal imaging feature data corresponding to different preset gesture action types, and the ultrasonic feature library stores ultrasonic feature data corresponding to different preset gesture action types;

[0245] a ninth processing unit, configured to extract first feature data corresponding to a current gesture action type of a current gesture action from the infrared thermal imaging feature library, and to extract second feature data corresponding to the current gesture action type from the ultrasonic feature library;

[0246] Extracting features from the infrared thermal imaging data corresponding to the current gesture action type, and matching the extracted features with the first feature data to obtain a first verification result of the current gesture action;

[0247] The tenth processing unit is configured to extract features from the ultrasonic data corresponding to the current gesture action type, and match the extracted features with the second feature data to obtain a second verification result of the current gesture action.

[0248] In some optional implementations, the fifth processing module 405 includes:

[0249] an eleventh processing unit, configured to calculate a comprehensive confidence level that the current gesture action belongs to the current gesture action type based on the confidence level that the current gesture action belongs to the current gesture action type in the first recognition result, the first verification result, and the second verification result;

[0250] a twelfth processing unit, configured to determine whether the comprehensive confidence level is greater than a preset confidence threshold;

[0251] The thirteenth processing unit is configured to determine that the gesture recognition result is the current gesture action type when the comprehensive confidence level is greater than a preset confidence threshold.

[0252] In some optional implementations, the gesture recognition system further includes:

[0253] The sixth processing module is used to control the operation of the baseboard heater based on the gesture recognition result.

[0254] In some optional implementations, the gesture recognition system further includes:

[0255] a seventh processing module, configured to match the infrared thermal imaging data and the ultrasonic data with a preset interference source feature library, wherein the preset interference source feature library stores interference features corresponding to different preset interference sources;

[0256] The eighth processing module is configured to determine that the gesture recognition result is an invalid gesture when an interference feature matching any preset interference source exists in the infrared thermal imaging data or the ultrasonic data.

[0257] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0258] The gesture recognition system in this embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0259] The embodiment of the present invention provides a baseboard heater, comprising: a radar acquisition device, an infrared thermal imaging acquisition device and an ultrasonic acquisition device, and the baseboard heater also comprises: a controller. Figure 5 , Figure 5 Schematic diagram of the structure of the controller of the skirting heater provided by the optional embodiment of the present invention. Figure 5 As shown, the controller includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0260] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0261] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0262] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0263] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0264] The controller further includes a communication interface 30 for the controller to communicate with other devices or a communication network.

[0265] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0266] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A gesture recognition method is applied to a baseboard heater, wherein the baseboard heater is provided with: a radar acquisition device, an infrared thermal imaging acquisition device, and an ultrasonic acquisition device, characterized in that: The method comprises: When the gesture recognition function of the baseboard heater is turned on, the radar collection device is controlled to collect radar data within the set area; Performing gesture recognition based on the radar data to obtain a first recognition result of the gesture action, the first recognition result at least including: a recognized gesture action and a corresponding gesture action type; Based on the number of gestures recognized in the first recognition result, controlling the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data respectively; Based on the gesture action type, performing gesture recognition on the infrared thermal imaging data and the ultrasonic data respectively to obtain a first verification result and a second verification result of the gesture action; A gesture recognition result is determined based on the first recognition result, the first verification result, and the second verification result.

2. The method according to claim 1, characterized in that The performing gesture recognition based on the radar data includes: Performing frequency domain conversion on the radar data to obtain frequency domain data; Feature extraction is performed on the frequency domain data, and the extracted features are matched with a radar feature library to obtain a gesture action type of the gesture action, wherein the radar feature library stores radar feature data corresponding to different preset gesture action types.

3. The method according to claim 1, characterized in that Based on the number of gestures recognized in the first recognition result, controlling the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data respectively includes: When the gesture action recognized in the first recognition result is a single one, the infrared thermal imaging acquisition device and the ultrasonic acquisition device are controlled to respectively acquire infrared thermal imaging data and ultrasonic data within the set area.

4. The method according to claim 3, characterized in that The method further comprises: When multiple gesture actions are recognized in the first recognition result, respectively obtaining a target position of each gesture action in the radar data; Performing coordinate transformation on each target position to obtain a first position corresponding to the infrared thermal imaging acquisition device and a second position corresponding to the ultrasonic acquisition device; The infrared thermal imaging acquisition device is controlled to collect data from the first position to obtain infrared thermal imaging data corresponding to each gesture action, and the ultrasonic acquisition device is controlled to collect data from the second position to obtain ultrasonic data corresponding to each gesture action.

5. The method according to any one of claims 1 to 4, characterized in that Based on the gesture action type, gesture recognition is performed on the infrared thermal imaging data and the ultrasonic data respectively to obtain a first verification result and a second verification result of the gesture action, including: Obtaining an infrared thermal imaging feature library and an ultrasonic feature library, wherein the infrared thermal imaging feature library stores infrared thermal imaging feature data corresponding to different preset gesture action types, and the ultrasonic feature library stores ultrasonic feature data corresponding to different preset gesture action types; Extracting first feature data corresponding to the current gesture action type of the current gesture action from the infrared thermal imaging feature library, and extracting second feature data corresponding to the current gesture action type from the ultrasonic feature library; Extracting features from the infrared thermal imaging data corresponding to the current gesture action type, and matching the extracted features with the first feature data to obtain a first verification result of the current gesture action; Feature extraction is performed on the ultrasonic data corresponding to the current gesture action type, and the extracted features are matched with the second feature data to obtain a second verification result of the current gesture action.

6. The method according to claim 1, characterized in that The first recognition result, the first verification result, and the second verification result all include: a confidence level that the gesture action belongs to a gesture action type; and determining the gesture recognition result based on the first recognition result, the first verification result, and the second verification result includes: Calculating a comprehensive confidence level that the current gesture action belongs to the current gesture action type based on the confidence level that the current gesture action belongs to the current gesture action type in the first recognition result, the first verification result, and the second verification result; Determining whether the comprehensive confidence level is greater than a preset confidence threshold; When the comprehensive confidence is greater than a preset confidence threshold, the gesture recognition result is determined to be the current gesture action type.

7. The method according to claim 1, characterized in that The method further comprises: The baseboard heater is controlled to operate based on the gesture recognition result.

8. The method according to claim 1, characterized in that Before performing gesture recognition on the infrared thermal imaging data and the ultrasonic data based on the gesture action type, the method further includes: Matching the infrared thermal imaging data and the ultrasonic data with a preset interference source feature library respectively, wherein the preset interference source feature library stores interference features corresponding to different preset interference sources; When an interference feature matching any preset interference source exists in the infrared thermal imaging data or the ultrasonic data, the gesture recognition result is determined to be an invalid gesture.

9. A gesture recognition system, applied to a baseboard heater, wherein the baseboard heater is provided with: a radar acquisition device, an infrared thermal imaging acquisition device, and an ultrasonic acquisition device, characterized in that: The system comprises: The first processing module is used to control the radar collection device to collect radar data within a set area when the gesture recognition function of the baseboard heater is turned on; a second processing module, configured to perform gesture recognition based on the radar data to obtain a first recognition result of the gesture action, wherein the first recognition result at least includes: a recognized gesture action and a corresponding gesture action type; a third processing module, configured to control the infrared thermal imaging acquisition device and the ultrasonic acquisition device to collect infrared thermal imaging data and ultrasonic data, respectively, based on the number of gesture actions recognized in the first recognition result; A fourth processing module is configured to perform gesture recognition on the infrared thermal imaging data and the ultrasonic data based on the gesture action type, to obtain a first verification result and a second verification result of the gesture action; A fifth processing module is configured to determine a gesture recognition result based on the first recognition result, the first verification result, and the second verification result.

10. A baseboard heater, comprising: The radar acquisition device, the infrared thermal imaging acquisition device and the ultrasonic acquisition device are characterized in that the baseboard heater also includes: a controller, and the controller includes: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 8 by executing the computer instructions.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 8.

Images