Gesture recognition method and smart panel based on millimeter-wave radar

Abstract

This invention proposes a gesture recognition method and intelligent panel based on millimeter-wave radar. It belongs to the field of sensor and gesture recognition technology. The method includes: transmitting a frequency-modulated continuous wave radar signal to a millimeter-wave radar module integrated within the panel; performing range and Doppler transformation processing on the received echo signal to generate time-frequency feature data containing micro-motion information of the hand; performing clutter suppression and noise filtering processing on the time-frequency feature data to generate clean echo feature data; by combining the FMCW radar echo signal with micro-Doppler feature extraction, the system can accurately capture micro-motion information of the hand within a range of up to 3 meters, and simultaneously penetrate non-metallic obstacles such as thin wooden boards to complete gesture recognition, significantly expanding the applicable scenarios for gesture control.

Description

Technical Field

[0001] This invention proposes a gesture recognition method and intelligent panel based on millimeter-wave radar, belonging to the field of sensor and gesture recognition technology. Background Technology

[0002] With the rapid development of smart home and human-computer interaction technologies, gesture recognition, as a natural and intuitive interaction method, has received widespread attention. Currently, mainstream gesture recognition solutions mainly include camera-based visual solutions and sensor-based non-visual solutions. While visual solutions offer high recognition accuracy, they pose a risk of privacy breaches and are significantly limited by lighting conditions, with recognition rates dropping drastically in low light or strong light environments. Infrared or ultrasonic sensor solutions can mitigate lighting issues to some extent, but their detection range is limited, typically not exceeding one meter, and they cannot penetrate obstacles, significantly restricting their application scenarios.

[0003] Millimeter-wave radar boasts advantages such as all-weather operation, penetration of non-metallic materials, and long detection range, leading to its recent introduction into the field of gesture recognition. However, most existing millimeter-wave radar gesture recognition solutions rely on high-performance computing platforms, resulting in high algorithm complexity and power consumption, making them difficult to deploy in embedded panels. Furthermore, most solutions focus solely on gesture recognition, failing to integrate gesture control with functions such as presence detection and vital sign monitoring, resulting in limited system functionality. In addition, existing panels are often exposed, disrupting the overall aesthetics of the home environment and hindering the widespread application of millimeter-wave radar gesture recognition technology in smart home scenarios. Summary of the Invention

[0004] This invention provides a gesture recognition method and smart panel based on millimeter-wave radar to solve the problems mentioned in the background section above: The gesture recognition method based on millimeter-wave radar proposed in this invention includes: S1. Transmit frequency-modulated continuous wave radar signals to the millimeter-wave radar module integrated inside the panel, perform range and Doppler transformation processing on the received echo signals to generate time-frequency characteristic data containing hand micro-movement information; perform clutter suppression and noise filtering processing on the time-frequency characteristic data to generate clean echo characteristic data. S2. Extract three-dimensional features of velocity, direction and trajectory from the clean echo feature data, use the micro-Doppler effect to analyze the movement patterns of each joint of the hand, and generate hand movement feature vector data; perform lightweight neural network forward inference processing on the hand movement feature vector data to generate gesture classification probability data. S3. Perform threshold judgment and temporal continuity verification on the gesture classification probability data to generate gesture command data; perform vital sign signal separation processing on the gesture command data, extract respiratory and heart rate features from the same set of echo signals, and generate vital sign monitoring data. S4. Perform multimodal fusion processing on gesture command data and vital sign monitoring data to generate comprehensive control command data; perform application scenario matching processing on comprehensive control command data to generate scenario adaptive control signals; S5. Perform execution priority sorting processing on the scene adaptive control signal to generate multi-task scheduling sequence data; perform panel hidden installation adaptation processing on the multi-task scheduling sequence data to generate non-visual interactive control output.

[0005] The present invention proposes a smart panel based on millimeter-wave radar, the smart panel comprising: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

[0006] The beneficial effects of this invention are as follows: By combining FMCW radar echo signals with micro-Doppler feature extraction, the system can accurately capture micro-hand movements within a range of up to 3 meters, and simultaneously penetrate non-metallic obstacles such as thin wooden boards to complete gesture recognition, significantly expanding the applicable scenarios for gesture control. The introduction of a lightweight neural network significantly reduces computational overhead while maintaining recognition accuracy, enabling the algorithm to run smoothly in an embedded panel. It can respond to gesture commands and simultaneously extract vital sign data such as respiration and heart rate, avoiding the limitations of traditional single-function solutions. The concealed installation of the panel eliminates the interference of exposed devices on the aesthetics of the home, while also incorporating presence detection functionality. Automatic control of lights turning on when someone enters and turning off when someone leaves is achieved, enhancing the overall coordination and ease of use of smart homes. Furthermore, the multi-dimensional analysis of speed, direction, and trajectory using distance-Doppler images effectively reduces misidentification caused by single-feature judgments, avoiding the problem of incorrect execution of gesture commands in complex backgrounds, and overall improving the robustness of the system and user experience in real-world home environments. Attached Figure Description

[0007] Figure 1 This is a diagram illustrating the steps of the method described in this invention; Figure 2 This is a detailed flowchart of step S3 in this invention. Detailed Implementation

[0008] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0009] One embodiment of the present invention, such as Figure 1As shown, a gesture recognition method based on millimeter-wave radar is described, the method comprising: S1. Transmit frequency-modulated continuous wave radar signals to the millimeter-wave radar module integrated inside the panel, perform range and Doppler transformation processing on the received echo signals to generate time-frequency characteristic data containing hand micro-movement information; perform clutter suppression and noise filtering processing on the time-frequency characteristic data to generate clean echo characteristic data. S2. Extract three-dimensional features of velocity, direction and trajectory from the clean echo feature data, use the micro-Doppler effect to analyze the movement patterns of each joint of the hand, and generate hand movement feature vector data; perform lightweight neural network forward inference processing on the hand movement feature vector data to generate gesture classification probability data. S3. Perform threshold judgment and temporal continuity verification on the gesture classification probability data to generate gesture command data; perform vital sign signal separation processing on the gesture command data, extract respiratory and heart rate features from the same set of echo signals, and generate vital sign monitoring data. S4. Perform multimodal fusion processing on gesture command data and vital sign monitoring data to generate comprehensive control command data; perform application scenario matching processing on comprehensive control command data to generate scenario adaptive control signals; S5. Perform execution priority sorting processing on the scene adaptive control signal to generate multi-task scheduling sequence data; perform panel hidden installation adaptation processing on the multi-task scheduling sequence data to generate non-visual interactive control output.

[0010] The working principle and effects of the above technical solution are as follows: By synchronously acquiring hand movements and physiological micro-motion signals through millimeter-wave radar, the recognition sensitivity of the panel without button interaction can be improved, and even subtle gestures can be stably recognized. The radar signal undergoes multi-layer clutter and noise filtering, reducing recognition deviations caused by environmental debris and electromagnetic interference, and enhancing signal acquisition stability under different lighting and obstruction scenarios. Multimodal fusion scheduling commands can reduce the frequency of false triggering of single gesture commands, and the regularized scheduling sequence can reduce hardware load fluctuations. Layered timing verification and weight adaptation processing can prevent instantaneous clutter signals from being judged as valid gestures, preventing accidental manipulation of device operation. The entire signal analysis process can achieve precise gesture control while simultaneously capturing respiratory and heart rate status, balancing the needs of device interaction and human status monitoring. The hidden signal output structure can also reduce wear on exposed panel components and extend the service life of the overall interactive structure.

[0011] In one embodiment of the present invention, S1 includes: S11. The control panel has a built-in millimeter-wave radar module that radiates frequency-modulated continuous wave signals outward with a fixed frequency sweep period. The radar receiver continuously collects the raw sampled data of the echoes scattered and reflected back by the human hand. It completes the target distance dimension calculation by relying on the fast Fourier transform of the distance dimension and performs the Doppler dimension spectrum conversion simultaneously to obtain the raw time-frequency sampled dataset that binds the distance information and the instantaneous radial velocity information. S12. Split the original time-frequency sampling dataset into a static clutter slice and a dynamic target slice. The static clutter slice stores fixed frequency data generated by the cabinet, panel structural components, and fixed obstacles in the environment. The dynamic target slice retains variable frequency domain data caused by hand movements. The time-domain mean cancellation algorithm is used to remove static clutter components and obtain the first noise reduction time-frequency intermediate data. S13. For the intermediate time-frequency data of the initial noise reduction, conduct random noise screening in the frequency domain. By comparing the amplitude correlation of adjacent frequency points, filter out discrete noise data caused by environmental electromagnetic interference. Divide the high and low frequency noise intervals and configure corresponding attenuation coefficients to complete the layered filtering, and obtain the transitional time-frequency characteristic data with random noise removed. S14. Extract the signal segments corresponding to the amplitude change intervals in the transitional time-frequency feature data. These segments correspond to subtle hand movements such as finger flexion and extension, and palm swing. Remove the stationary redundant frequency domain segments corresponding to no movements, and generate refined time-frequency feature data that focuses on the micro-motion information of the hand after cropping the invalid spectrum intervals. S15. Input the refined time-frequency characteristic data into the adaptive filtering link to complete the secondary filtering of residual clutter, statistically analyze the signal-to-noise ratio change index of the spectrum throughout the time period, perform segmented smoothing correction on data segments with a signal-to-noise ratio lower than the preset threshold, and generate pure echo characteristic data without redundant interference components after the correction is completed.

[0012] The working principle and effects of the above technical solution are as follows: By using multi-dimensional spectrum calculation to fully capture the details of the echo signal corresponding to hand movements, the radar can improve the completeness of acquiring hand micro-motion signals. By splitting static clutter and dynamic target data into layers to cancel clutter, signal interference caused by equipment structure and fixed environment can be significantly reduced, effectively improving the purity of the original signal. By using partitioned filtering to process various frequency domain noises, it can adapt to electromagnetic interference in different environments, enhancing the environmental adaptability of signal processing. By pruning invalid stationary spectrum segments, it can reduce the computational burden of invalid data and improve the overall speed of subsequent feature processing. Through secondary adaptive filtering and signal-to-noise ratio correction, it can compensate for the processing blind spots of single noise reduction, avoid residual interference signals causing feature data distortion, and not only comprehensively remove signal defects caused by various environmental interferences, but also completely retain the effective signal characteristics of hand micro-movements, significantly improving the accuracy and stability of the initial echo data processing.

[0013] In one embodiment of the present invention, S2 includes: S21. The pure echo feature data is split into continuous frame slices along the time axis. The radial instantaneous motion velocity value, the motion orientation parameter represented by the signal phase offset, and the spatial displacement trajectory parameter formed by the stacking of multiple frame coordinates are extracted from the single frame data respectively. The three types of parameters are stored independently to form a three-dimensional original feature dataset. S22. Based on the sideband fluctuation pattern of the micro-Doppler spectrum, the spectrum sub-bands corresponding to three different scattering sources are divided into palm, finger root and fingertip. The frequency fluctuation period of each sub-band is mapped to the joint flexion, extension and rotation movement change pattern. The spectrum change pattern is converted into joint dynamic change quantification parameters and added to the three-dimensional original feature dataset. S23. Perform dimension normalization and scaling on the original 3D feature dataset after supplementing joint quantization parameters, unify the data units and value ranges of the three types of features, remove outlier feature values ​​with abnormal jumps in a single frame, and generate normalized basic feature data of hand movement after the normalization process is completed. S24. Based on the regularized hand motion basic feature data, feature splicing is completed according to the fixed vector arrangement rules. Each group of continuous temporal features is combined into a multi-dimensional feature vector. All temporal vectors are summarized in batches to obtain a complete hand motion feature vector dataset. S25. Input the hand motion feature vector dataset into a lightweight convolutional neural network with solidified parameters in batches. The network performs convolution and pooling operations layer by layer to complete feature depth mapping. The fully connected layer outputs the probability values ​​corresponding to various preset gestures. All output values ​​are summarized to generate complete gesture classification probability data.

[0014] The working principle and effects of the above technical solution are as follows: By extracting multi-dimensional hand motion parameters through temporal frame segmentation, various basic information of hand movements can be comprehensively collected, improving the comprehensiveness of hand action feature extraction. By splitting the spectral sub-bands corresponding to different hand structures and converting joint motion parameters, the feature information of subtle hand movements can be refined, compensating for the lack of detail in the overall feature acquisition. By normalizing the feature data scale and removing outliers, the negative impact of messy and abnormal data on subsequent calculations can be reduced, improving the regularity of feature data. By concatenating temporal features to form a complete feature vector set, the temporal correlation characteristics of continuous hand movements can be strengthened. Relying on lightweight neural networks to complete hierarchical feature operation and classification can reduce the consumption of device computing resources and avoid the gesture recognition lag problem caused by the computational delay of complex models. This ensures the recognition accuracy of subtle gesture movements and improves the overall speed and stability of gesture classification.

[0015] In one embodiment of the present invention, step S21 includes: The clean echo feature data is subjected to full-domain temporal decomposition processing. The continuous signal is uniformly divided into frames according to the fixed sampling interval of the radar to obtain independent frame unit data with uniform duration and continuous temporal sequence. Dynamic parameter analysis and calculation are performed on the data of each independent frame unit. The radial instantaneous motion velocity value of the hand target relative to the radar is extracted frame by frame. The spatial motion orientation parameter corresponding to the signal phase offset is extracted. The coordinate points of multiple frames are stacked to generate spatial displacement trajectory parameters. The three types of motion parameters obtained from the analysis—velocity, orientation, and trajectory—are classified and stored separately. Data storage intervals for parameters of different dimensions are distinguished to prevent parameter crossover and confusion, thereby generating preliminary three-dimensional motion parameter data. The preliminary three-dimensional motion parameter data is processed to remove invalid information. Stable invalid parameters generated in a static, motionless state are eliminated, while valid dynamic parameters corresponding to hand micro-movements are retained to generate simplified three-dimensional parameter data. The simplified 3D parameter data is processed by time-series normalization and structural encapsulation. All effective parameter units are reorganized according to the time-series progressive logic, the data layout format is unified, and a complete 3D original feature dataset of hand motion is generated.

[0016] The working principle and effects of the above technical solution are as follows: By uniformly segmenting the radar echo signal into frames, continuous hand movement signals can be broken down into regular temporal units, improving the stability of motion parameter analysis. Frame-by-frame, multi-angle analysis of hand movement speed, orientation, and trajectory information can completely capture the details of dynamic hand changes, improving the completeness of basic feature acquisition. Classifying and isolating multi-dimensional motion parameters eliminates data interference between different parameters, improving the purity of the original parameters. Removing static and invalid parameter information reduces the computational space occupied by redundant data, improving the overall efficiency of subsequent data processing. Temporal regularization and unified encapsulation of valid parameters clarify the temporal logic of hand movements, avoiding data disorder that could lead to deviations in subsequent feature extraction. This processing method retains all valid features of subtle hand movements while reducing the volume of invalid data, continuously ensuring the accuracy and efficiency of subsequent gesture recognition work.

[0017] One embodiment of the present invention, such as Figure 2 As shown, S3 includes: S31. Set a fixed decision threshold for gesture confidence, iterate through the gesture classification probability data row by row, mark the single-class gesture probability value that exceeds the threshold as a valid candidate gesture, and directly remove the entries with a probability lower than the threshold as invalid. After filtering, the initial screening of valid gesture candidate dataset is obtained. S32. Perform continuous frame coherence statistics on the initial screening of valid gesture candidate datasets along the time sequence dimension, count the frequency of the same gesture mark in consecutive sampling frames, remove candidate gestures whose consecutive frame frequency does not reach the time sequence benchmark value, and combine the remaining entries to form standardized gesture command data. S33. Backtrack to generate the original echo baseband data corresponding to the pure echo feature data, and separate the high-frequency hand movement signal component and the low-frequency body micro-movement signal component from the baseband data. The low-frequency component is stored separately as the original sampling data of vital signs. S34. Perform bandpass frequency division on the raw vital signs sampling data. The low-frequency narrowband signal corresponds to the respiratory cycle fluctuation formed by the chest wall rise and fall, and the sub-narrowband pulse signal corresponds to the heart rate fluctuation caused by the pulse on the body surface. Extract the period and amplitude change parameters of the two types of signals respectively. S35. Summarize respiratory cycle parameters and heart rate pulse parameters and perform full-time data smoothing to eliminate parameter distortion caused by slight limb swaying. The smoothed parameters are then uniformly packaged to generate standardized vital sign monitoring data.

[0018] The working principle and effects of the above technical solution are as follows: Threshold filtering performs preliminary screening of gesture classification probability data, eliminating low-reliability gesture recognition results and reducing invalid gesture judgments. Temporal continuity verification of gesture frame data filters out misrecognition results caused by transient clutter, improving the stability of gesture command output. Signal component splitting of the original echo baseband data enables multiplexing of the same set of radar signals, reducing acquisition losses from additional sensing equipment. Separating the signal bands corresponding to respiration and heart rate through bandpass frequency division accurately distinguishes between the two physiological characteristics, improving the precision of vital sign data extraction. Full-time smoothing correction of vital sign parameters offsets data distortion caused by slight limb movements, preventing large fluctuations in vital sign monitoring results. This processing flow can output accurate and stable interactive gesture commands while simultaneously performing precise monitoring of human vital signs, effectively broadening the functional application range of radar signals.

[0019] In one embodiment of the present invention, S34 includes: S341. Perform full-domain spectrum decomposition processing on the original vital signs sampling data, remove residual limb movement clutter components, retain the effective spectrum signal corresponding to human physiological micro-movements, and generate pure vital signs spectrum basic data. S342. Perform multi-band bandpass separation processing on the pure vital signs spectrum basic data, and extract low-frequency narrowband continuous fluctuation signals. These signals conform to the movement pattern of the human chest cavity and generate the original respiratory fluctuation signal data. S343. Perform sub-narrowband pulse signal filtering processing on the remaining spectrum signal to capture the regular pulse fluctuation signal generated by the pulse on the human body surface, separate the redundant signals of irrelevant frequency bands, and generate the original heart rate pulse signal data. S344. Perform time-series waveform analysis on the raw respiratory wave signal data, continuously collect waveform cycle interval and dynamic amplitude change information, batch statistically analyze the effective feature information of the whole time period, and generate respiratory cycle and amplitude feature parameter data. S345. Perform pulse feature analysis processing on the raw heart rate pulse signal data, continuously capture pulse jump interval and peak amplitude fluctuation information, collect all effective feature information, and integrate to generate heart rate cycle and amplitude change parameter data.

[0020] The working principle and effects of the above technical solution are as follows: By decomposing the entire spectrum to remove clutter components caused by limb movement, interference information in the original vital sign data can be cleared, improving the purity of the vital sign spectrum signal. By separating signal data into different bands through multi-band bandpass separation, continuous fluctuation signals corresponding to respiration can be accurately distinguished, making respiratory feature extraction more accurate and stable. By selectively filtering sub-narrowband pulse signals, effective fluctuation information corresponding to heart rate can be extracted separately, reducing interference from redundant data in irrelevant frequency bands on heart rate recognition. By analyzing and statistically analyzing the period and amplitude changes of respiratory waveforms through time-series waveform analysis, the dynamic characteristics of respiratory operation can be fully recorded, improving the completeness of respiratory parameter acquisition. By continuously capturing pulse intervals and peak value changes to obtain heart rate characteristics, the dynamic fluctuation state of heart rate can be meticulously recorded, avoiding information loss caused by single parameter acquisition. The entire processing method can independently and accurately analyze two types of human physiological parameters while maintaining the continuity and authenticity of vital sign data, effectively improving the overall accuracy of radar-based non-contact vital sign monitoring.

[0021] In one embodiment of the present invention, S345 includes: The heart rate cycle and amplitude change parameter data are processed for time-series continuity verification. Instantaneous jump parameters and invalid window parameters are screened in the parameter segments of the whole time period. Abnormal and distorted parameter segments are removed to generate preliminary verified heart rate characteristic parameter data. The preliminary verification heart rate characteristic parameter data is subjected to global time alignment processing to unify the sampling time sequence nodes of each parameter segment, eliminate the time sequence offset deviation caused by the parsing of different frame signals, and generate time-normalized heart rate parameter data. Multi-dimensional parameter fusion processing is performed on time-series regularized heart rate parameter data, and heart rate cycle change parameters and peak amplitude fluctuation parameters are superimposed to form a multi-feature fused heart rate basic dataset. The heart rate baseline dataset is stabilized to smooth out parameter shifts caused by small local fluctuations and reduce data interference from slight limb movements, thereby generating a high-precision heart rate feature dataset. The high-precision heart rate feature dataset is collected and encapsulated as a whole, the data arrangement structure of all parameters is standardized, the heart rate feature information of the whole time period is summarized, and the complete heart rate cycle and amplitude change parameter data are generated.

[0022] The working principle and effects of the above technical solution are as follows: By screening for jumps and gaps in heart rate parameters through temporal continuity verification, distorted data generated during the acquisition process can be eliminated, reducing the adverse impact of abnormal values ​​on heart rate detection results. By calibrating the time nodes of various parameters through global temporal alignment, temporal deviations caused by frame parsing can be eliminated, improving the temporal consistency of heart rate data. By integrating periodic and amplitude information through multi-dimensional parameter fusion, the feature dimensions of heart rate data can be enriched, making the feedback of heart rate status more comprehensive. By smoothing out local fluctuations in data through stabilization correction, signal interference caused by slight limb shaking can be weakened, avoiding irregular fluctuations in heart rate values. By standardizing and encapsulating data through unified regularization, aggregation, and encapsulation, the regularity and completeness of the final heart rate data can be improved. The entire processing flow can filter out data errors caused by various acquisition interferences while retaining the true characteristics of dynamic heart rate changes, significantly improving the accuracy and stability of non-contact heart rate monitoring.

[0023] In one embodiment of the present invention, step S4 includes: S41. Unify the sampling time axis of gesture command data and vital sign monitoring data, use timestamp as alignment benchmark to complete the temporal pairing of the two types of data, and bind gesture entries and vital sign entries at the same time to form paired multi-source raw fusion data. S42. The feature weighted fusion algorithm is used to assign fusion weights to the two types of data. The weight of gesture command data fluctuates dynamically with the magnitude of the movement, and the weight of vital sign monitoring data is adjusted synchronously with the fluctuation of physiological parameters. After the weight assignment is completed, the initial data fusion is completed, and an intermediate fusion dataset is generated. S43. Verify the logical correlation of the data within the fusion intermediate dataset. When abnormal fluctuations in vital signs are combined with specific gestures, correct the corresponding instruction weight coefficients to eliminate the fusion deviation caused by mutual interference between the two types of data. After correction, generate comprehensive control instruction data. S44. Retrieve the local pre-stored multi-application scenario benchmark parameter library, extract the instruction features from the integrated control instruction data segment by segment, compare the extracted features with the scenario benchmark parameters, and match the scenario classification label with the highest similarity. S45. Based on the scene classification tags obtained by matching, retrieve the corresponding control parameter configuration template, rewrite the output amplitude and response delay parameters in the comprehensive control instruction according to the template parameters, and generate a scene adaptive control signal that is adapted to the current use environment after the parameters are rewritten.

[0024] The working principle and effects of the above technical solution are as follows: Synchronizing gesture data and vital sign data through time-series alignment eliminates information misalignment caused by asynchronous multi-source data acquisition, improving the fundamental reliability of multimodal data fusion. Dynamic weight adjustment adapts to fluctuations in actions and physiological parameters, aligning with actual usage scenarios and reducing fusion distortion caused by fixed weight calculations. Logical correlation verification corrects fusion data deviations, avoiding command errors caused by interference between the two types of data and improving the output quality of comprehensive control commands. Feature comparison matches corresponding application scenarios, allowing control commands to adapt to different usage environments and enhancing the adaptability of device interaction. Adjusting command output parameters through scene parameter templates optimizes device response. The entire processing flow balances the accuracy of gesture interaction with optimizing control logic based on human vital sign status, effectively improving the precision and scene adaptability of intelligent panel interactive control.

[0025] In one embodiment of the present invention, S42 includes: Collect all motion amplitude quantization values ​​within the gesture command data, statistically analyze the fluctuation range of amplitude values ​​within a continuous time interval, and generate full-domain gesture amplitude measurement data. Collect all physiological parameter fluctuation differences within the vital signs monitoring data, statistically analyze the fluctuation range of the differences within the corresponding time interval, and generate full-domain measurement data of vital signs fluctuations. Based on the global measurement data of gesture amplitude, set the floating gradient range of gesture weight. When the amplitude value increases, the corresponding gradient range switches upward to generate dynamic gesture weight assignment data. By comparing the full-domain measurement data of vital sign fluctuations, a floating gradient range of vital sign weights is set. When the fluctuation difference increases, the corresponding gradient range is switched synchronously to generate dynamic weight assignment data of vital signs. The gesture dynamic weight assignment data and vital sign dynamic weight assignment data are applied to the paired multi-source original fusion data to complete the numerical weight superposition. After the superposition operation is completed, the fusion intermediate dataset is generated.

[0026] The working principle and effects of the above technical solution are as follows: By statistically analyzing the fluctuation range of gesture amplitude and physiological parameters across the entire domain, the dynamic change patterns of both types of data can be fully grasped, providing real data support for weight allocation. Adjusting the corresponding weight ratio by adapting to changes in gesture amplitude within gradient intervals allows gesture interaction information to function appropriately in scenarios with varying gesture amplitudes. Synchronously switching the weight gradient of vital signs based on fluctuations in vital signs ensures that the proportion of physiological monitoring data aligns with real-time changes in the human body's state. Dynamically adjusting the fusion weights of the two types of data overcomes the adaptation limitations of fixed ratios and reduces deviations in fusion results caused by a single dominant data source. Weighted superposition achieves multi-source data fusion, balancing the output proportions of gesture control information and vital sign monitoring information, avoiding distortion of single-dimensional information that could negatively impact overall control performance. This processing method not only aligns with the dynamic changes in human-computer interaction but also continuously improves the rationality and accuracy of multimodal data fusion, making the device control logic more suitable for actual usage scenarios.

[0027] In one embodiment of the present invention, step S5 includes: S51. Extract the task type identifier field carried by the scene adaptive control signal, and classify all control signals according to the preset task urgency level division rules. High urgency tasks are assigned to the first-level queue, routine adjustment tasks are assigned to the second-level queue, and delayed execution tasks are assigned to the third-level queue. S52. Within the same level queue, a secondary sorting is performed according to the order of signal generation. Across levels, the arrangement logic of prioritizing scheduling of higher-level queues is followed. After the entire queue is arranged, the scheduling order of all signals is summarized to generate structured multi-task scheduling sequence data. S53. Combining the panel's hidden hardware wiring parameters and the electrical characteristics of the internal driver chip, the output level range of each instruction in the multi-task scheduling sequence data is corrected to adapt the drive load parameters corresponding to the narrow installation space inside the panel, and the hardware adaptation scheduling intermediate data is obtained. S54. Split the hardware adaptation scheduling intermediate data into multiple parallel output sub-instructions. The sub-instruction arrangement format is adapted to the data transmission protocol of the panel's built-in signal interface. Remove excessively long instruction fragments that exceed the interface's transmission bandwidth. After trimming and optimization, generate pre-processed output data. S55. Convert the preprocessed output data into electrical output codes that can be directly parsed by the panel driver module. After encoding, bypass the exposed physical buttons on the panel to send invisible signals, and finally generate non-visual interactive control outputs for the whole machine.

[0028] The working principle and effects of the above technical solution are as follows: By dividing various control signals into execution queues according to emergency levels, priority can be given to handling equipment actions requiring rapid response, improving the response speed of critical instructions. By using both hierarchical and temporal sorting to regulate the scheduling order, the execution order when multiple tasks are issued simultaneously can be streamlined, reducing operational chaos caused by concurrent instructions. Combining hardware electrical parameters to correct the instruction level range can match the load operating conditions in confined spaces, avoiding chip overload caused by electrical parameter mismatch. Splitting parallel sub-instructions and trimming excessive segments can adapt to the interface transmission limit, reducing data transmission stuttering or packet loss. Converting signals into dedicated electrical codes to bypass physical buttons reduces wear and tear caused by prolonged button pressing. The entire scheduling output process ensures stable and orderly operation of multiple tasks while adapting to the confined hardware environment of hidden panels, relying entirely on radar gestures for buttonless operation, extending the overall lifespan of the panel.

[0029] In one embodiment of the present invention, S53 includes: Retrieve the wiring layout parameters and line impedance parameters corresponding to the concealed installation structure of the panel, batch collect the electrical constraints of the internal hardware of the panel, and generate panel hardware adaptation reference data. The panel collects the rated operating level range, load tolerance value, and signal response parameters of the built-in driver chip, summarizes the chip's electrical operating limits, and generates electrical characteristic data of the driver chip. The structured multi-task scheduling sequence data is compared and verified with hardware adaptation benchmark data and driver chip electrical characteristic data to screen out level parameters in the scheduling instructions that exceed the hardware adaptation range and generate abnormal parameter scheduling data. The level range iterative correction is carried out on the abnormal parameter scheduling data, and the output level amplitude and fluctuation range of each scheduling command are fine-tuned so that all command parameters match the load operating conditions of the narrow space of the panel, and the parameter-corrected scheduling transition data is generated. Perform full-domain parameter verification and normalization on the scheduling transition data, unify the electrical output standards of all scheduling instructions, adapt to the hardware operating environment of the hidden panel, and generate hardware-adapted scheduling intermediate data.

[0030] The working principle and effects of the above technical solution are as follows: By aggregating wiring impedance and structural parameters to form a hardware adaptation benchmark, the operating boundaries of the internal circuitry of the hidden panel can be completely locked, providing a complete reference for instruction correction. Summarizing various electrical limits of the driver chip allows for precise understanding of the parameter range for stable chip operation, reducing the probability of parameter deviations. Comparing the scheduling sequence and hardware parameters item by item to screen for abnormal levels enables rapid identification of out-of-specification instructions, preventing non-compliant levels from causing operational failures. Iteratively adjusting the amplitude and fluctuation range of abnormal instructions can match the load-bearing capacity of the confined space circuitry, reducing the risk of circuit overheating and overload. A unified electrical output standard across the entire domain standardizes scheduling data, ensuring all instructions maintain consistent output specifications. The entire calibration process not only matches the operational limitations of the hidden, compact hardware layout but also maintains stable transmission of multi-task scheduling signals, continuously reducing hardware losses and extending the service life of the internal circuitry and driver chip of the panel.

[0031] One embodiment of the present invention provides a smart panel based on millimeter-wave radar, the smart panel comprising: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

[0032] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A gesture recognition method based on millimeter-wave radar, characterized in that, The method includes: S1. Transmit frequency-modulated continuous wave radar signals to the millimeter-wave radar module integrated inside the panel, perform range and Doppler transformation processing on the received echo signals to generate time-frequency characteristic data containing hand micro-movement information; perform clutter suppression and noise filtering processing on the time-frequency characteristic data to generate clean echo characteristic data. S2. Extract three-dimensional features of velocity, direction and trajectory from the clean echo feature data, use the micro-Doppler effect to analyze the movement patterns of each joint of the hand, and generate hand movement feature vector data; perform lightweight neural network forward inference processing on the hand movement feature vector data to generate gesture classification probability data. S3. Perform threshold judgment and temporal continuity verification on the gesture classification probability data to generate gesture command data; perform vital sign signal separation processing on the gesture command data, extract respiratory and heart rate features from the same set of echo signals, and generate vital sign monitoring data. S4. Perform multimodal fusion processing on gesture command data and vital sign monitoring data to generate comprehensive control command data; perform application scenario matching processing on comprehensive control command data to generate scenario adaptive control signals; S5. Perform execution priority sorting processing on the scene adaptive control signal to generate multi-task scheduling sequence data; perform panel hidden installation adaptation processing on the multi-task scheduling sequence data to generate non-visual interactive control output.

2. The gesture recognition method and smart panel based on millimeter-wave radar according to claim 1, characterized in that, S1 includes: S11. The control panel has a built-in millimeter-wave radar module that radiates frequency-modulated continuous wave signals outward with a fixed frequency sweep period. The radar receiver continuously collects the raw sampled data of the echoes scattered and reflected back by the human hand. It completes the target range dimension calculation by relying on the range dimension fast Fourier transform and simultaneously performs the Doppler dimension spectrum conversion to obtain the raw time-frequency sampled dataset. S12. Split the original time-frequency sampling dataset into static clutter slices and dynamic target slices. Use the time-domain mean cancellation algorithm to remove static clutter components and obtain the first noise reduction time-frequency intermediate data. S13. For the intermediate time-frequency data of the initial noise reduction, conduct random noise screening in the frequency domain. By comparing the amplitude correlation of adjacent frequency points, filter out discrete noise data caused by environmental electromagnetic interference. Divide the high and low frequency noise intervals and configure corresponding attenuation coefficients to complete the layered filtering, and obtain the transitional time-frequency characteristic data with random noise removed. S14. Extract the signal segments corresponding to the amplitude change intervals in the transitional time-frequency feature data, remove the stationary redundant frequency domain segments corresponding to no movement, and generate refined time-frequency feature data focusing on hand micro-movement information after trimming the invalid spectrum intervals. S15. Input the refined time-frequency characteristic data into the adaptive filtering link to complete the secondary filtering of residual clutter, statistically analyze the signal-to-noise ratio change index of the spectrum throughout the time period, perform segmented smoothing correction on data segments with a signal-to-noise ratio lower than the preset threshold, and generate pure echo characteristic data without redundant interference components after the correction is completed.

3. The gesture recognition method and smart panel based on millimeter-wave radar according to claim 1, characterized in that, S2 includes: S21. The pure echo feature data is split into continuous frame slices along the time axis. The radial instantaneous motion velocity value, the motion orientation parameter represented by the signal phase offset, and the spatial displacement trajectory parameter formed by the stacking of multiple frame coordinates are extracted from the single frame data respectively. The three types of parameters are stored independently to form a three-dimensional original feature dataset. S22. Based on the sideband fluctuation pattern of the micro-Doppler spectrum, the spectrum sub-bands corresponding to three different scattering sources are divided into palm, finger root and fingertip. The frequency fluctuation period of each sub-band is mapped to the joint flexion, extension and rotation movement change pattern. The spectrum change pattern is converted into joint dynamic change quantification parameters and added to the three-dimensional original feature dataset. S23. Perform dimension normalization and scaling on the original 3D feature dataset after supplementing joint quantization parameters, unify the data units and value ranges of the three types of features, remove outlier feature values ​​with abnormal jumps in a single frame, and generate normalized basic feature data of hand movement after the normalization process is completed. S24. Based on the regularized hand motion basic feature data, feature splicing is completed according to the fixed vector arrangement rules. Each group of continuous temporal features is combined into a multi-dimensional feature vector. All temporal vectors are summarized in batches to obtain a complete hand motion feature vector dataset. S25. Input the hand motion feature vector dataset into a lightweight convolutional neural network with solidified parameters in batches. The network performs convolution and pooling operations layer by layer to complete feature depth mapping. The fully connected layer outputs the probability values ​​corresponding to various preset gestures. All output values ​​are summarized to generate complete gesture classification probability data.

4. The gesture recognition method and smart panel based on millimeter-wave radar according to claim 1, characterized in that, The S3 includes: S31. Set a fixed decision threshold for gesture confidence, iterate through the gesture classification probability data row by row, mark the single-class gesture probability value that exceeds the threshold as a valid candidate gesture, and directly remove the entries with a probability lower than the threshold as invalid. After filtering, the initial screening of valid gesture candidate dataset is obtained. S32. Perform continuous frame coherence statistics on the initial screening of valid gesture candidate datasets along the time sequence dimension, count the frequency of the same gesture mark in consecutive sampling frames, remove candidate gestures whose consecutive frame frequency does not reach the time sequence benchmark value, and combine the remaining entries to form standardized gesture command data. S33. Backtrack to generate the original echo baseband data corresponding to the pure echo feature data, and separate the high-frequency hand movement signal component and the low-frequency body micro-movement signal component from the baseband data. The low-frequency component is stored separately as the original sampling data of vital signs. S34. Perform bandpass frequency division on the raw vital signs sampling data. The low-frequency narrowband signal corresponds to the respiratory cycle fluctuation formed by the chest wall rise and fall, and the sub-narrowband pulse signal corresponds to the heart rate fluctuation caused by the pulse on the body surface. Extract the period and amplitude change parameters of the two types of signals respectively. S35. Summarize respiratory cycle parameters and heart rate pulse parameters and perform full-time data smoothing. The smoothed parameters are then uniformly packaged to generate standardized vital sign monitoring data.

5. The gesture recognition method and smart panel based on millimeter-wave radar according to claim 4, characterized in that, S34 includes: S341. Perform full-domain spectrum decomposition processing on the original vital signs sampling data, remove residual limb movement clutter components, retain the effective spectrum signal corresponding to human physiological micro-movements, and generate pure vital signs spectrum basic data. S342. Perform multi-band bandpass separation processing on the pure vital signs spectrum basic data, and extract low-frequency narrowband continuous fluctuation signals. These signals conform to the movement pattern of the human chest cavity and generate the original respiratory fluctuation signal data. S343. Perform sub-narrowband pulse signal filtering processing on the remaining spectrum signal to capture the regular pulse fluctuation signal generated by the pulse on the human body surface, separate the redundant signals of irrelevant frequency bands, and generate the original heart rate pulse signal data. S344. Perform time-series waveform analysis on the raw respiratory wave signal data, continuously collect waveform cycle interval and dynamic amplitude change information, batch statistically analyze the effective feature information of the whole time period, and generate respiratory cycle and amplitude feature parameter data. S345. Perform pulse feature analysis processing on the raw heart rate pulse signal data, continuously capture pulse jump interval and peak amplitude fluctuation information, collect all effective feature information, and integrate to generate heart rate cycle and amplitude change parameter data.

6. The gesture recognition method and smart panel based on millimeter-wave radar according to claim 5, characterized in that, The S345 includes: The heart rate cycle and amplitude change parameter data are subjected to time-series continuity verification processing to screen for instantaneous jump parameters and invalid window parameters in the parameter segments of the whole time period, and generate preliminary verification heart rate characteristic parameter data; The preliminary verified heart rate characteristic parameter data is subjected to global time-series alignment processing to unify the sampling time-series nodes of each parameter segment and generate time-ordered heart rate parameter data. Multi-dimensional parameter fusion processing is performed on time-series regularized heart rate parameter data, and heart rate cycle change parameters and peak amplitude fluctuation parameters are superimposed to form a multi-feature fused heart rate basic dataset. The heart rate baseline dataset is stabilized to smooth out parameter shifts caused by small local fluctuations and reduce data interference from slight limb movements, thereby generating a high-precision heart rate feature dataset. The high-precision heart rate feature dataset is collected and encapsulated as a whole, the data arrangement structure of all parameters is standardized, the heart rate feature information of the whole time period is summarized, and the complete heart rate cycle and amplitude change parameter data are generated.

7. The gesture recognition method and smart panel based on millimeter-wave radar according to claim 1, characterized in that, The S4 includes: S41. Unify the sampling time axis of gesture command data and vital sign monitoring data, use timestamp as alignment benchmark to complete the temporal pairing of the two types of data, and bind gesture entries and vital sign entries at the same time to form paired multi-source raw fusion data. S42. The feature weighted fusion algorithm is used to assign fusion weights to the two types of data. The weight of gesture command data fluctuates dynamically with the magnitude of the movement, and the weight of vital sign monitoring data is adjusted synchronously with the fluctuation of physiological parameters. After the weight assignment is completed, the initial data fusion is completed, and an intermediate fusion dataset is generated. S43. Verify the logical correlation of the data within the fusion intermediate dataset. When abnormal fluctuations in vital signs are combined with specific gestures, adjust the corresponding instruction weight coefficients. After correction, generate comprehensive control instruction data. S44. Retrieve the local pre-stored multi-application scenario benchmark parameter library, extract the instruction features from the integrated control instruction data segment by segment, compare the extracted features with the scenario benchmark parameters, and match the scenario classification label with the highest similarity. S45. Based on the scene classification tags obtained by matching, retrieve the corresponding control parameter configuration template, rewrite the output amplitude and response delay parameters in the comprehensive control instruction according to the template parameters, and generate a scene adaptive control signal that is adapted to the current use environment after the parameters are rewritten.

8. The gesture recognition method and smart panel based on millimeter-wave radar according to claim 7, characterized in that, S42 includes: Collect all motion amplitude quantization values ​​within the gesture command data, statistically analyze the fluctuation range of amplitude values ​​within a continuous time interval, and generate full-domain gesture amplitude measurement data. Collect all physiological parameter fluctuation differences within the vital signs monitoring data, statistically analyze the fluctuation range of the differences within the corresponding time interval, and generate full-domain measurement data of vital signs fluctuations. Based on the global measurement data of gesture amplitude, set the floating gradient range of gesture weight. When the amplitude value increases, the corresponding gradient range switches upward to generate dynamic gesture weight assignment data. By comparing the full-domain measurement data of vital sign fluctuations, a floating gradient range of vital sign weights is set. When the fluctuation difference increases, the corresponding gradient range is switched synchronously to generate dynamic weight assignment data of vital signs. The gesture dynamic weight assignment data and vital sign dynamic weight assignment data are applied to the paired multi-source original fusion data to complete the numerical weight superposition. After the superposition operation is completed, the fusion intermediate dataset is generated.

9. The gesture recognition method and smart panel based on millimeter-wave radar according to claim 1, characterized in that, The S5 includes: S51. Extract the task type identifier field carried by the scene adaptive control signal, and classify all control signals according to the preset task urgency level division rules. High urgency tasks are assigned to the first-level queue, routine adjustment tasks are assigned to the second-level queue, and delayed execution tasks are assigned to the third-level queue. S52. Within the same level queue, a secondary sorting is performed according to the order of signal generation. Across levels, the arrangement logic of prioritizing scheduling of higher-level queues is followed. After the entire queue is arranged, the scheduling order of all signals is summarized to generate structured multi-task scheduling sequence data. S53. Combining the panel's hidden hardware wiring parameters and the electrical characteristics of the internal driver chip, the output level range of each instruction in the multi-task scheduling sequence data is corrected to adapt the drive load parameters corresponding to the narrow installation space inside the panel, and the hardware adaptation scheduling intermediate data is obtained. S54. Split the hardware adaptation scheduling intermediate data into multiple parallel output sub-instructions. The sub-instruction arrangement format is adapted to the data transmission protocol of the panel's built-in signal interface. Remove excessively long instruction fragments that exceed the interface's transmission bandwidth. After trimming and optimization, generate pre-processed output data. S55. Convert the preprocessed output data into electrical output codes that can be directly parsed by the panel driver module. After encoding, bypass the exposed physical buttons on the panel to send invisible signals, and finally generate non-visual interactive control outputs for the whole machine.

10. A smart panel based on millimeter-wave radar, characterized in that, The smart panel includes: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 9.

Images