Instrument gesture control method based on millimeter wave radar and related equipment

Abstract

The invention provides an instrument gesture control method based on a millimeter wave radar and related equipment, and relates to the technical field of instrument intelligent control. The method comprises the steps of firstly obtaining a target reflection signal through a built-in millimeter wave radar, then identifying the signal to determine whether a target is a palm, and then identifying a target action according to a preset palm template library to obtain palm action information; then, a control instruction is determined by combining the information with an instrument control mapping library (storing a palm action and control instruction mapping relation), and finally, corresponding parts of the electric bicycle are controlled according to the instruction. According to the scheme, non-contact gesture control is achieved by means of the millimeter wave radar technology and a multi-layer feature recognition mechanism, the convenience and the intelligent level of interaction of the electric bicycle can be improved, and a more flexible control mode is provided for a user.

Description

Technical Field

[0001] The present application relates to the technical field of instrument intelligent control, and in particular to an instrument gesture control method and related equipment based on millimeter-wave radar. Background Art

[0002] In today's era of rapid technological advancement, intelligent interactive technology is increasingly permeating various fields, bringing unprecedented convenience to people's lives. In the electric bicycle sector, as people's demand for riding convenience and intelligent technology continues to rise, optimizing instrument control methods has become a focus of the industry.

[0003] Traditional electric bicycles have long relied on buttons to control instrument functions. Riders need to press these buttons to perform operations such as switching display pages, adjusting riding gears, and turning lights on and off.

[0004] However, existing button control technology has serious drawbacks. Frequently operating buttons while riding can distract riders and affect safety. Buttons are also prone to malfunction after prolonged use. Their relatively fixed functions make it difficult to meet user demands for personalized and diverse functions, limiting the advancement of e-bike intelligence. Therefore, a contactless instrument control method is urgently needed. Summary of the Invention

[0005] The present application provides a millimeter-wave radar-based instrument gesture control method and related equipment, which are used to solve the problems of the existing electric bicycle control method being single, lacking flexibility and personalization.

[0006] In the first aspect, the present application provides an instrument gesture control method based on millimeter-wave radar, which is applied to the controller of the instrument equipment. The method includes: obtaining the reflected signal of the target through the millimeter-wave radar built into the instrument; identifying the reflected signal to determine that the target is a palm; identifying the movement of the target according to a preset palm template library to obtain palm movement information; combining the palm movement information, determining the corresponding control instruction according to the instrument control mapping library, which pre-stores the mapping relationship between different palm movements and instrument control instructions; and controlling the corresponding part of the electric bicycle according to the control instruction.

[0007] By implementing this technical solution, the high-resolution characteristics of millimeter-wave radar accurately capture palm reflection signals and, combined with a pre-set palm template library, accurately identify palm movements. The construction of multidimensional feature vectors effectively distinguishes palms from other interfering targets, while the introduction of an environmental interference feature library further reduces the false recognition rate. The instrument control mapping library converts recognition results into specific commands, enabling contactless control and addressing the vulnerability of traditional physical buttons to environmental influences. Ultimately, through high-precision motion recognition and command mapping, this solution makes instrument control more consistent with natural human interaction habits, meeting user needs for personalized and diverse operation methods.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the step of identifying the reflected signal to determine that the target is a palm specifically includes: constructing an original feature data set containing time-series target distance, velocity information, horizontal angle and pitch angle; preprocessing the original feature data set to obtain a preprocessed feature data set; constructing a multidimensional feature vector containing Doppler features, instantaneous energy features, distance weighted features and Doppler azimuth features; combining the multidimensional feature vector and an environmental interference feature library to calculate a similarity score, wherein the environmental interference feature library pre-stores feature vector templates corresponding to common environmental interference sources; if the similarity score is lower than a set feature similarity threshold, the target is determined to be a palm.

[0009] By employing this technical solution, the original feature dataset constructed comprehensively records relevant information about the target, and preprocessing removes noise interference for more accurate data. A multidimensional feature vector integrates multiple features and, combined with the environmental interference feature library, calculates a similarity score, effectively distinguishing between palms and common environmental interference sources. If the similarity score falls below a threshold, it is identified as a palm. This significantly improves palm recognition accuracy and reduces false positives, ensuring that subsequent control commands based on palm movements are accurate. This allows users to use gesture control more stably and reliably, meeting diverse user needs.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the step of constructing an original feature data set including target distance, speed information, horizontal angle and pitch angle specifically includes: performing two-dimensional fast Fourier transform processing on the reflected signal to obtain two-dimensional grid data including the target distance and the speed information; combining the two-dimensional grid data, calculating the horizontal angle and the pitch angle of the target through angle fast Fourier transform; and constructing the original feature data set based on the target distance, the speed information, the horizontal angle and the pitch angle.

[0011] By employing this technical solution, a two-dimensional fast Fourier transform (FFT) converts the time-domain signal into a two-dimensional range-velocity grid, enabling precise analysis of the target's spatial position and motion state. An angular FFT further supplements the pan / tilt angle information, providing three-dimensional spatial positioning capabilities. This signal processing approach leverages the frequency-domain feature extraction advantages of millimeter-wave radar and more effectively separates multi-target reflection signals compared to traditional time-domain analysis methods. The construction of the original feature dataset utilizes multi-dimensional data fusion to establish a high-precision input baseline for subsequent palm recognition, ensuring the system can accurately extract valid features from complex echoes.

[0012] In combination with some embodiments of the first aspect, in some embodiments, the step of constructing a multidimensional feature vector including Doppler characteristics, instantaneous energy characteristics, distance-weighted characteristics and Doppler azimuth characteristics specifically includes: obtaining energy data of the reflected signal; calculating the Doppler characteristics based on the target distance and the velocity information; calculating the instantaneous energy characteristics based on the energy data; weighting the velocity information, the horizontal angle and the pitch angle according to the target distance to obtain a distance-weighted characteristic; obtaining a Doppler azimuth characteristic based on the velocity information, the horizontal angle and the pitch angle; and constructing the multidimensional feature vector based on the Doppler characteristics, instantaneous energy characteristics, distance-weighted characteristics and Doppler azimuth characteristics.

[0013] By employing these technical solutions, Doppler features reflect subtle changes in palm movement speed, instantaneous energy features capture the intensity of gestures, distance-weighted features optimize the recognition weight of near-field operations using the distance attenuation coefficient, and Doppler azimuth features correlate movement direction with spatial position. This multidimensional feature vector construction comprehensively describes gesture characteristics from both kinematic and spatial perspectives, enabling the system to identify complex movement differences and laying the foundation for refined control command mapping.

[0014] In combination with some embodiments of the first aspect, in some embodiments, after the step of performing two-dimensional fast Fourier transform processing on the reflected signal to obtain two-dimensional grid data containing target distance and speed information, it also includes: if the target distance is greater than the set effective distance threshold, terminating the recognition operation; determining the change speed of the target based on the speed information; if the change speed is not within the set effective change speed range, terminating the recognition operation.

[0015] By employing this technical solution, the effective distance threshold setting eliminates interfering targets outside the control range, improving system response efficiency. Speed range filtering specifically selects targets that match human hand motion characteristics to avoid false triggering. This two-stage rapid screening mechanism performs coarse filtering before feature extraction, significantly reducing the subsequent computational load. This design is particularly suitable for mobile scenarios such as electric bicycles, effectively addressing environmental interference caused by changing road conditions and ensuring that system resources are focused on processing valid gesture signals, ensuring real-time performance while reducing the probability of false operation.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of calculating the horizontal angle and pitch angle of the target by angle fast Fourier transform in combination with the two-dimensional grid data, it also includes: calculating the horizontal angle change and the pitch angle change respectively according to the horizontal angle and the pitch angle; combining the horizontal angle change and the pitch angle change, weighting them according to preset weights to obtain a weighted angle change; if the weighted angle change is not within the preset angle change valid range, terminating the recognition operation.

[0017] By adopting the above technical solution, the horizontal angle change and pitch angle change are calculated and weighted to obtain the weighted angle change. If this change is not within the preset valid angle change range, it means that the target angle change does not conform to the range of normal palm movements, which may be external interference or unexpected movement. Terminating the recognition operation at this time can effectively eliminate interference and improve the accuracy of palm movement recognition. Recognition is only continued when the angle change is within a reasonable range, which ensures that the system only responds to valid palm movements, providing users with a more accurate and reliable gesture control experience and meeting personalized usage requirements.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the step of identifying the target's movement according to a preset palm template library to obtain palm movement information specifically includes: extracting the current movement features of the target from the preprocessed feature data set to obtain a movement feature vector; performing similarity calculation on the movement feature vector and multiple palm template feature vectors in the palm template library to obtain a palm movement similarity score, wherein the palm template library stores feature vectors corresponding to different standard palm movements in advance; selecting the palm movement corresponding to the template feature vector with the highest palm movement similarity score as the preliminary recognition result; if the palm movement similarity score is higher than the set movement recognition similarity threshold, determining that the preliminary recognition result is the final palm movement information; if the palm movement similarity score is lower than the set movement recognition similarity threshold, determining that this movement recognition has failed, and terminating the recognition operation.

[0019] By employing this technical solution, motion feature vectors are extracted from the preprocessed feature dataset and their similarity is calculated against the template feature vectors in the palm template library. The one with the highest similarity score is selected as the preliminary recognition result. If the score exceeds a threshold, it is determined as the final palm motion information; if it falls below the threshold, recognition is deemed a failure and the operation is terminated. This rigorous recognition process ensures the accuracy and reliability of the recognition results, avoiding misjudgments and incorrect control. Accurate palm motion recognition ensures the execution of correct control commands based on the instrument control mapping library, providing users with stable and precise gesture control capabilities and meeting their needs for personalized and precise operation.

[0020] In a second aspect, the present application provides a controller comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code comprising computer instructions, the one or more processors invoking the computer instructions to cause the controller to execute the method described in the first aspect and any possible implementation of the first aspect. In a third aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on the controller, cause the controller to execute the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a fourth aspect, the present application provides a computer program product, which, when executed on a controller, enables the controller to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. The technology uses millimeter-wave radar to obtain reflected signals, identifies the target as a palm, combines the palm template library to identify motion information, and determines the control instructions based on the instrument control mapping library to control the electric bicycle. Therefore, it effectively solves the technical problems of the existing electric bicycles with a single control method and inconvenient and inflexible operation. It also breaks away from the limitations of traditional physical buttons, allowing users to conveniently control the vehicle through gestures, meeting personalized and diversified functional needs and improving the user experience.

[0023] 2. By acquiring reflected signal energy data and combining it with target distance and velocity information to calculate Doppler characteristics, instantaneous energy characteristics, distance-weighted characteristics, and Doppler azimuth characteristics, and constructing a multi-dimensional feature vector, the technology effectively solves the technical problem of insufficient accuracy in existing palm recognition and motion judgment, thereby achieving a comprehensive and detailed depiction of target characteristics from different angles, improving recognition accuracy, and providing strong support for diversified and personalized gesture control.

[0024] 3. By calculating the changes in horizontal and pitch angles and weighting them to obtain a weighted angle change, and judging whether it is within the preset effective range of angle changes, and terminating the recognition operation if it is not, the technical means is adopted. Therefore, the technical problem that existing gesture recognition is susceptible to external interference and has low accuracy is effectively solved. It then eliminates interference, ensures that the system only responds to valid palm movements, provides a precise and reliable gesture control experience, and meets the technical effect of personalized use requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a millimeter-wave radar-based instrument gesture control method according to an embodiment of the present application; Figure 2 This is a schematic diagram of an application scenario of the instrument gesture control method based on millimeter wave radar in an embodiment of the present application; Figure 3 This is a schematic diagram of the physical device structure of the controller in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The terms used in the following examples of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular expressions "a," "an," "said," "above," "the," and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and encompasses any or all possible combinations of one or more of the listed items.

[0027] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0028] For ease of understanding, the following describes the process of the method provided by this implementation. Figure 1 , which is a flow chart of the instrument gesture control method based on millimeter-wave radar in an embodiment of the present application.

[0029] S101, acquiring a reflected signal of a target through a millimeter-wave radar built into the instrument; In the e-bike's instrumentation, the controller uses a built-in millimeter-wave radar to collect reflected signals from the surrounding environment in real time. Millimeter-wave radar has high directivity, strong penetration, and anti-interference capabilities, allowing it to operate stably in complex environments. Before setting this, the radar's transmit power must be configured. This power level affects the radar's detection range and can be adjusted based on the actual application scenario. For example, in open environments, the power can be increased to expand the detection range. Additionally, the scanning period must be configured, which determines how frequently the radar acquires data.

[0030] After initialization is complete, the controller triggers the millimeter-wave radar to emit high-frequency electromagnetic waves. These waves propagate outward in a specific waveform. When encountering a target, some of the waves are reflected back. The radar's receiving antenna captures these reflected signals and converts them into electrical signals. Because reflected signals are typically weak, the controller amplifies them to enhance signal strength for subsequent analysis. The controller continuously collects data on the reflected signals according to the set scanning cycle. This data includes information such as the target's distance, speed, and angle. To ensure data integrity and continuity, the controller temporarily stores the collected data in an internal cache.

[0031] S102: Identify the reflected signal to determine that the target is a palm; First, an original feature data set containing the time series of target distance, speed information, horizontal angle and pitch angle is constructed. Specifically, the echo signal returned after the RF signal emitted by the radar is reflected by the target can be collected. The echo signal is subjected to fast Fourier transform in the time domain and frequency domain to obtain a two-dimensional data grid. One dimension represents the distance. By analyzing the signal strength at different distances, the distance of the target can be estimated; the other dimension represents the speed. By analyzing the Doppler frequency shift at different speeds, the speed of the target can be estimated. In this way, the distance and speed information of the target can be extracted from the data of the two-dimensional grid.

[0032] Combined with the two-dimensional grid data, the horizontal angle and pitch angle of the target are calculated through the angle fast Fourier transform. Specifically, a radar antenna array containing multiple antenna elements can be used. There is a certain phase difference in the echo signals received from different antennas. The two-dimensional grid data is fast Fourier transformed in the angle domain to obtain the angle information of the target in the horizontal and pitch directions. The horizontal angle and pitch angle of the target can be calculated based on the positive and negative sign and size of the phase difference. Repeating the above operation, the distance, speed, horizontal angle and pitch angle of the target can be obtained at different times. The data at different times are organized into an original feature data set containing time series information, where each frame contains the distance, speed, horizontal angle and pitch angle of the target at that moment, thus constructing an original radar feature containing rich dynamic information.

[0033] Next, the original dataset can be preprocessed by removing noise, filling missing values, and normalizing it to obtain a more standardized and clean dataset. This reduces the impact of noise in the original data on subsequent feature extraction and standardizes the range of feature values, facilitating computational combinations of different features. Power calculations can be performed on the preprocessed feature dataset to obtain the energy of the reflected signal at each moment, which reflects the overall reflection intensity of the target.

[0034] Using the target distance and velocity information in the preprocessed data set, the Doppler shift feature can be calculated. The Doppler shift is related to the target's radial velocity and can provide target motion information. The instantaneous energy feature is then calculated based on the energy data. Specifically, the controller obtains the energy data of the reflected signal; calculates the Doppler feature based on the target distance and velocity information; calculates the instantaneous energy feature based on the energy data; weights the velocity information, the horizontal angle, and the pitch angle according to the target distance to obtain a distance-weighted feature; obtains the Doppler azimuth feature based on the velocity information, the horizontal angle, and the pitch angle; and constructs the multidimensional feature vector based on the Doppler feature, the instantaneous energy feature, the distance-weighted feature, and the Doppler azimuth feature. The energy data of the reflected signal represents the intensity of the echo signal received by the millimeter-wave radar. The Doppler signature refers to the frequency offset caused by the target's motion relative to the radar. The instantaneous energy signature represents the energy distribution of the target's reflected wave at a specific moment. The distance-weighted signature represents the signature after weighting the motion parameters based on the target's distance. The Doppler azimuth signature is a comprehensive angular signature derived from the target's velocity and spatial position. The multidimensional feature vector represents the feature space representation formed by combining multiple features. More specifically, the controller first calculates the power spectral density of the received signal to obtain an energy data sequence reflecting the target's scattering characteristics. To extract the Doppler signature, the controller uses the target's range R and radial velocity v to calculate the frequency offset according to the Doppler effect formula fd = 2v / λ, where λ is the radar's operating wavelength, to obtain the target's Doppler signature. The instantaneous energy signature extracts the energy distribution within each time window by performing a short-time Fourier transform on the energy data sequence. The distance-weighted feature is calculated using e^(-αR) as the weight function, where α is the attenuation coefficient, to weight the velocity and angle information. The Doppler azimuth feature is calculated through the projection relationship between the velocity vector and the spatial azimuth. Finally, these features are normalized and concatenated into a feature vector.

[0035] In practical applications, the diversity of gestures and the complexity of environmental interference can lead to unstable feature extraction and reduced recognition accuracy. To address this issue, this solution employs an adaptive feature fusion strategy: First, each feature's recognition contribution is evaluated, calculating a feature importance score (si = wi * pi), where wi is the feature weight and pi is the feature's recognition accuracy. The weight of the feature in the vector is then dynamically adjusted based on the score: vi = si * fi, where fi is the original feature value. Finally, the adjusted features are combined to form the final feature vector. This approach emphasizes the role of key features, suppresses interference from unstable features, and improves recognition robustness.

[0036] It's also necessary to collect radar echo signals from common environmental interference sources (such as leaves and rain) in advance and extract their multidimensional feature vectors to construct an environmental interference signature library. Each vector corresponds to a specific interference source. For the target signal to be detected, its multidimensional feature vector is similarly extracted. The similarity score between this vector and all vectors in the environmental interference signature library is then calculated. Similarity algorithms can use cosine similarity or correlation coefficients to reflect the directional proximity of two vectors. A similarity score is then selected as a threshold to determine whether the target is an environmental interference source. This threshold needs to be adjusted based on the actual situation. A lower value increases the accuracy of the palm detection, but also increases the probability of false positives for environmental interference.

[0037] The calculated highest similarity score is compared with the preset threshold. If the highest similarity is lower than the threshold, it means that the target signal is very different from all environmental interferences and cannot match any interference. Therefore, it can be determined that the target is a palm.

[0038] In some embodiments, after determining the horizontal angle and pitch angle of the target, the following three scenarios can be triggered to terminate the gesture recognition operation. The first is to terminate the recognition operation if the target distance is greater than the set effective distance threshold; the second is to determine the target's change speed based on the speed information; if the change speed is not within the set effective change speed range, the recognition operation is terminated. The third is to calculate the horizontal angle change and pitch angle change based on the horizontal angle and pitch angle respectively; combine the horizontal angle change and pitch angle change, and weight them according to the preset weights to obtain the weighted angle change; if the weighted angle change is not within the preset angle change effective range, the recognition operation is terminated.

[0039] Among them, the target distance represents the vertical distance between the target and the plane where the millimeter-wave radar receiving and transmitting antennas are located; the effective distance threshold refers to the maximum effective detection distance pre-set according to the actual application scenario; the change speed refers to the target position change per unit time; the effective change speed range represents the pre-set reasonable hand movement speed range; the horizontal angle represents the angle of the target relative to the radar in the horizontal plane; the pitch angle refers to the elevation angle of the target relative to the radar in the vertical plane; the angle change represents the amplitude of the target's angle change during the movement; the weighted angle change refers to the comprehensive change after weighted fusion of the angle changes in the horizontal and pitch directions.

[0040] After completing the preliminary detection of the target, the controller needs to verify the validity of the detected target to ensure that it is a possible gesture action. Specifically, the controller first checks whether the target distance exceeds the valid range. If the target distance is greater than the preset threshold (such as 50cm), it means that the target distance is too far and may not be a gesture for control. At this time, the recognition process is terminated. Then, the speed of change is calculated based on the target's speed information. If the speed is not within a reasonable range (such as 0.1-1.0m / s), it is determined that it does not meet the normal gesture movement characteristics and the recognition is terminated. Finally, the controller calculates the angular change of the target in the horizontal and pitch directions respectively, and sets different weights according to the actual application scenario (such as horizontal: pitch = 6:4) for weighted fusion to obtain the comprehensive angle change. If the change exceeds the preset range (such as ±45°), it means that the movement amplitude is abnormal and the recognition process is also terminated.

[0041] In some embodiments, after completing basic palm motion recognition, the system needs to perform timing verification to ensure the validity and continuity of the gesture. Specifically, when the system first recognizes a palm approach motion, it starts a timer and records the current system timestamp. The system continuously monitors subsequent gesture recognition results. If a palm approach motion is detected again within a preset time window (e.g., 1 second), it is considered a duplicate recognition due to jitter or false touch, and the timer and approach record are cleared. If the recognition result is another gesture, the system re-evaluates whether the gesture is a palm approach. When the system recognizes a specific gesture that triggers the alarm, it directly clears the timer and begins executing the alarm command. This process is designed to identify double-confirmed gesture operations. By detecting whether there are two palm approach motions within a set time, false triggers are avoided and the reliability of the system is improved. If two palm approach motions are completed within the specified time, it is considered a valid operation instruction; if the timeout period expires or there is only a single motion, the recognition result is discarded.

[0042] S103, identifying the target's motion according to a preset palm template library to obtain palm motion information; The controller first needs to extract valid motion feature information from the preprocessed feature dataset. This process primarily involves the following steps: First, the controller segments the time series data. Since gestures typically last 0.3-1 seconds, the controller segments the continuous feature data into fixed time windows, analyzing the data within each time window as a complete motion segment.

[0043] Next, the controller extracts key feature parameters within each time window: spatial features, which include the palm's trajectory in three-dimensional space—the sequence of changes in (x, y, z) coordinates over time; velocity features, which calculate the instantaneous and average velocity of the palm's movement; acceleration features, which calculate the acceleration change during the palm's movement; direction features, which extract information about the palm's directional changes; and posture features, which analyze the palm's tilt angle, rotation angle, and other parameters. The controller organizes these extracted features into a fixed-dimensional feature vector. For example, if 20 dimensions of features are extracted, the feature vector might look like [x1, x2, ..., x20], where each component represents a specific feature parameter. The controller normalizes these features to a uniform range of [0, 1] to facilitate subsequent similarity calculations.

[0044] The controller of the instrument device calculates the similarity between the action feature vector and multiple palm template feature vectors in the palm template library. Specifically, the controller loads the palm template library, which stores the standard feature vectors of various predefined gestures. For example: the standard feature vector of the right swipe gesture; the standard feature vector of the left swipe gesture; the standard feature vector of the up and down swipe gesture; the standard feature vector of the clockwise / counterclockwise rotation gesture; the standard feature vector of the push and pull gesture, etc. For each template feature vector, the controller calculates its similarity score with the current action feature vector. The similarity calculation usually uses the Euclidean distance method: score = (v1·v2) / (||v1||·||v2||), where v1 and v2 are the current feature vector and the template feature vector, respectively.

[0045] The controller then selects the best match from all calculated similarity scores. First, it sorts all similarity scores in descending order. For example, the right swipe gesture template has a similarity score of 0.92; the left swipe gesture template has a similarity score of 0.45; the up swipe gesture template has a similarity score of 0.38; and the down swipe gesture template has a similarity score of 0.35. The controller selects the gesture corresponding to the template with the highest score as the preliminary recognition result and compares this maximum similarity score with a predefined threshold for motion recognition, which is determined through extensive experimental statistical analysis. If the maximum similarity score is greater than or equal to the threshold, the current template match is confirmed to be successful, meaning the preliminary recognition result is correct. This result is then output as the final palm motion recognition result.

[0046] If the maximum similarity score is less than the threshold, it means that the current action does not match any standard action in the template library sufficiently, and a definite recognition result cannot be obtained. If the similarity score is lower than the threshold, the palm action recognition is determined to have failed, the recognition result is not output, and the subsequent generation and execution of control instructions based on the recognition result are terminated to avoid erroneous control due to misidentification. At the same time, the currently extracted action feature vector is cached for subsequent use when the template library is upgraded, and the number of recognition failures is recorded. If the number of failures is too high, the template library needs to be recalibrated. The controller waits for the next radar data to be collected, extracts the action feature vector again, and repeats the above palm action recognition process until it successfully matches a standard gesture in the template library.

[0047] The above steps describe in detail how the palm gesture similarity score is processed. If the score exceeds the threshold, recognition is considered successful; if it falls below the threshold, recognition is terminated to avoid misoperation. This mechanism can improve the accuracy and reliability of gesture recognition and provide users with a more precise contactless interactive control experience.

[0048] S104: Determine a corresponding control instruction based on the palm motion information and an instrument control mapping library, wherein the instrument control mapping library stores mapping relationships between different palm motions and instrument control instructions in advance; After successfully identifying the palm movement, the controller uses the acquired palm movement information as input and prepares to generate control instructions based on the information. The controller accesses the instrument control mapping library stored in the system, which has previously stored the mapping relationship between palm movements and control instructions in a structured manner such as tables. For example, the mapping relationship can be set as follows: the palm flipping action is mapped to the control instruction of "switch to the next display interface", the palm movement to the left and right is mapped to the control instruction of "adjusting the brightness of the car lights", etc. The controller will query the mapping library to find the action item that matches the input palm movement information. The controller can then directly read the control instruction content mapped to the action, such as "switch to the next display interface". The control instruction is saved in the controller in the form of a readable code or signal for subsequent calls.

[0049] S105: Control corresponding parts of the electric bicycle according to the control instruction.

[0050] After receiving the corresponding gesture control command, the controller needs to locate the part of the e-bike where the command is intended to act. Different control commands correspond to different components in the vehicle control system, such as the display, motor, and lighting. The controller pre-stores device information templates and queries them based on the command type to determine the target device. It then obtains the device's communication interface parameters and control protocol.

[0051] The controller sends a control command data frame to the target device via the CAN bus or other wired connection, using its dedicated communication protocol. After receiving the control command, the device parses the data frame content and performs the corresponding action in its own drive system, such as motor rotation, light switching, etc. The device will package the execution status and feedback information into a data frame and send it back to the controller to confirm that the command has been executed. The controller parses the feedback frame to determine whether the device executed the action correctly. If it fails, the control command can be resent. When the device feedback confirms that the command has been successfully executed, the controller completes the entire control process for the device. Repeating the above steps for different control commands can control different devices on the electric vehicle in turn, realizing contactless vehicle operation.

[0052] In the embodiment of the present application, a millimeter-wave radar is used to obtain the target reflection signal, and the palm is identified through multi-dimensional feature analysis. The gesture is then accurately judged based on the preset palm template library, and finally mapped to the corresponding control instruction to realize the technical means of contactless control of the electric vehicle. Therefore, the limitations of the existing physical button control method, the inconvenient and inflexible operation, and the single control function are effectively solved. It is realized that the vehicle can be controlled through different gestures without manual contact, which meets the user's personalized and diversified control needs, and greatly improves the user experience and the technical effect of the intelligent level of electric vehicles.

[0053] The following is a schematic diagram of the application scenario of the method provided by this implementation, combined with the above content. Figure 2 , is a schematic diagram of an application scenario of the instrument gesture control method based on millimeter wave radar in an embodiment of the present application.

[0054] Figure 2 The diagram shows a schematic of an electric bicycle dashboard, featuring a simple design layout. From top to bottom, the diagram includes the following main components: A semicircular area labeled "Headlight" at the top indicates the headlight status (on / off) and may include a brightness level display. This is typically linked to the vehicle's lighting system to provide illumination at night or in low light conditions. In the center is a square "display" area, which displays riding information. This may include speed, battery status, mileage, gear position, time display, and fault warnings. It may support multiple pages of information, providing the primary interface for user interface interaction. At the bottom is the millimeter-wave radar module area, which includes a receiving antenna (left) that receives millimeter-wave signals reflected from gestures, collects gesture motion data, and processes environmental reflection information. The fingerprint recognition area (center circle) can be used for identity verification, provide additional security authentication features, or integrate fingerprint unlocking. The transmitting antenna (right) transmits millimeter-wave signals, generates radar waves for gesture detection, and provides signal coverage for the sensing area. The entire module constitutes a complete human-computer interaction system, which realizes non-contact control through millimeter-wave radar technology. Users can operate various functions of the vehicle through gestures, improving the convenience and safety of operation.

[0055] The controller in the embodiment of the present invention is described below from the perspective of hardware processing. Figure 3 , is a schematic diagram of a physical device structure of the controller in an embodiment of the present application.

[0056] It should be noted that Figure 3 The structure of the controller shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0057] like Figure 3As shown, the controller includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes, such as the methods described in the above embodiments, based on programs stored in a read-only memory (ROM) 302 or programs loaded from a storage unit 308 into a random access memory (RAM) 303. RAM 303 also stores various programs and data required for system operation. CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to bus 304.

[0058] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, push button switches, and the like; an output section 307 including a liquid crystal display (LCD), an audio output device, indicator lights, and the like; a storage section 308 including a hard disk and the like; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. Removable media 311, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 310 as needed, so that computer programs read from the removable media can be installed in the storage section 308 as needed.

[0059] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 309 and / or installed from removable media 311. When executed by the central processing unit (CPU) 301, the computer program performs the various functions defined in the present invention.

[0060] It should be noted that specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0061] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings.

[0062] Specifically, the controller of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the instrument gesture control method based on millimeter-wave radar provided in the above embodiment is implemented.

[0063] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the controller described in the above embodiments, or may exist independently and not incorporated into the controller. The storage medium carries one or more computer programs, which, when executed by a processor of the controller, enable the controller to implement the millimeter-wave radar-based instrument gesture control method provided in the above embodiments.

[0064] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0065] As used in the above embodiments, the term “when” may be interpreted to mean “if” or “after” or “in response to determining that” or “in response to detecting that”, depending on the context. Similarly, the phrases “upon determining that” or “if (stated condition or event) is detected” may be interpreted to mean “if determining that” or “in response to determining that” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0066] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A millimeter-wave radar-based instrument gesture control method, applied to a controller of an instrument device, characterized in that: The method comprises: Acquire the target's reflected signal through the millimeter-wave radar built into the instrument; Identifying the reflected signal to determine that the target is a palm; Recognize the target's motion according to a preset palm template library to obtain palm motion information; In combination with the palm motion information, a corresponding control instruction is determined according to an instrument control mapping library, wherein the instrument control mapping library stores mapping relationships between different palm motions and instrument control instructions in advance; The corresponding parts of the electric bicycle are controlled according to the control instructions.

2. The method according to claim 1, characterized in that The step of identifying the reflected signal to determine that the target is a palm specifically includes: Construct an original feature dataset containing time-series target distance, speed information, horizontal angle, and pitch angle; Preprocessing the original feature data set to obtain a preprocessed feature data set; Constructing a multidimensional feature vector including Doppler feature, instantaneous energy feature, distance weighted feature and Doppler azimuth feature; Calculating a similarity score by combining the multidimensional feature vector and an environmental interference feature library, wherein the environmental interference feature library stores feature vector templates corresponding to common environmental interference sources in advance; If the similarity score is lower than a set feature similarity threshold, the target is determined to be a palm.

3. The method according to claim 2, characterized in that The steps for constructing the original feature dataset containing the time series target distance, speed information, horizontal angle, and pitch angle include: Performing a two-dimensional fast Fourier transform on the reflected signal to obtain two-dimensional grid data containing the target distance and the speed information; Calculating the horizontal angle and the pitch angle of the target by angular fast Fourier transform in combination with the two-dimensional grid data; The original feature data set is constructed according to the target distance, the speed information, the horizontal angle and the pitch angle at different time sequences.

4. The method according to claim 2, characterized in that The steps of constructing a multidimensional feature vector including Doppler features, instantaneous energy features, distance weighted features and Doppler azimuth features specifically include: Acquiring energy data of the reflected signal; Calculating a Doppler characteristic according to the target distance and the velocity information; calculating instantaneous energy characteristics based on the energy data; weighting the speed information, the horizontal angle, and the pitch angle according to the target distance to obtain a distance-weighted feature; Obtaining a Doppler azimuth angle feature according to the velocity information, the horizontal angle, and the pitch angle; The multidimensional feature vector is constructed according to the Doppler feature, instantaneous energy feature, distance weighted feature and Doppler azimuth feature.

5. The method according to claim 1 or 3, characterized in that After the step of performing a two-dimensional fast Fourier transform on the reflected signal to obtain two-dimensional grid data containing target distance and speed information, the method further includes: If the target distance is greater than the set effective distance threshold, the recognition operation is terminated; determining a changing speed of the target according to the speed information; If the change speed is not within the set effective change speed range, the recognition operation is terminated.

6. The method according to claim 1 or 3, characterized in that After the step of calculating the horizontal angle and the pitch angle of the target by using the angle fast Fourier transform in combination with the two-dimensional grid data, the method further includes: Calculating a horizontal angle change and a pitch angle change according to the horizontal angle and the pitch angle respectively; Combining the horizontal angle change and the pitch angle change, weighting them according to a preset weight to obtain a weighted angle change; If the weighted angle change is not within the preset angle change valid range, the recognition operation is terminated.

7. The method according to claim 1, characterized in that The step of identifying the target's motion according to a preset palm template library to obtain palm motion information specifically includes: Extracting the motion features of the current target from the preprocessed feature data set to obtain a motion feature vector; performing similarity calculation on the motion feature vector and multiple palm template feature vectors in the palm template library to obtain a palm motion similarity score, wherein the palm template library stores feature vectors corresponding to different standard palm motions in advance; Selecting the palm movement corresponding to the template feature vector with the highest palm movement similarity score as the preliminary recognition result; If the palm motion similarity score is higher than a set motion recognition similarity threshold, determining the preliminary recognition result as the final palm motion information; If the palm motion similarity score is lower than the set motion recognition similarity threshold, it is determined that the current motion recognition has failed, and the recognition operation is terminated.

8. A controller, characterized in that: The controller includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the controller to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a controller, the controller is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a controller, the controller is caused to perform the method according to any one of claims 1 to 7.

Images