A smart vehicle control method, device and equipment based on gesture recognition and a medium

Abstract

This application discloses a method, device, equipment, and medium for intelligent vehicle control based on gesture recognition, applied to intelligent vehicle control systems and relating to the field of vehicle control technology. The method includes: processing current gesture timing information obtained through a gesture sensor using a hybrid deep learning model to obtain dynamic gesture information; the dynamic gesture information includes a target gesture trajectory determined from preset gesture trajectories that corresponds to the current gesture spatial trajectory; determining, through a vehicle control module, whether the current gesture spatial trajectory is an intentional gesture trajectory; if so, determining, based on a preset function-working-state mapping relationship, whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled under the current vehicle working state; if allowed, sending a function command to the corresponding execution device based on the target vehicle function, so that the execution device can enable the target vehicle function based on the function command to complete vehicle control; this improves the accuracy of gesture recognition, thereby improving the precision of vehicle control.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and in particular to an intelligent vehicle control method, device, equipment and medium based on gesture recognition. Background Technology

[0002] Currently, vehicle (e.g., motorcycle) function control mainly relies on physical buttons, remote controls, or mobile apps. These methods all depend on signal input from third-party components, and the user experience is relatively limited. Especially in scenarios such as wearing gloves, rainy days, or nighttime, traditional control methods are slow to respond and prone to accidental touches. In recent years, some high-end models have attempted to introduce voice control or simple gesture recognition, but the problem of poor recognition accuracy still exists. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a method, device, equipment, and medium for intelligent vehicle control based on gesture recognition, which can improve the accuracy of gesture recognition and thus improve the precision of vehicle control. The specific solution is as follows:

[0004] In a first aspect, this application discloses an intelligent vehicle control method based on gesture recognition, applied to an intelligent vehicle control system, comprising:

[0005] The gesture sensor acquires the current gesture timing information, and the gesture recognition microcontroller built into the gesture sensor processes the current gesture timing information based on a hybrid deep learning model to identify dynamic gesture information; the dynamic gesture information includes a target gesture trajectory determined from preset gesture trajectories that corresponds to the current gesture spatial trajectory.

[0006] The vehicle control module determines whether the current gesture spatial trajectory is an intentional gesture trajectory.

[0007] If so, the vehicle control module determines whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled under the current vehicle working state based on the preset function and working state mapping relationship; the vehicle working state includes the start state, the power-off state, and the power-on but not started state;

[0008] If permitted, the vehicle body control module sends function commands to the corresponding execution device based on the target vehicle function, so that the execution device can activate the target vehicle function based on the function commands and complete vehicle control.

[0009] Optionally, the dynamic gesture information includes gesture-related time information;

[0010] Accordingly, the step of determining whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module includes:

[0011] The vehicle control module determines whether the current gesture spatial trajectory is continuous within a preset time window based on the gesture-related time information.

[0012] If not, then the current gesture space trajectory is determined to be an intentional gesture trajectory; if yes, then the current gesture space trajectory is determined to be an intentional gesture trajectory.

[0013] Optionally, determining whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module includes:

[0014] The vehicle control module determines, based on a preset function and operating status prohibition mapping relationship, whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled under the current vehicle operating status; the vehicle operating status includes vehicle-related speed and vehicle posture.

[0015] If not allowed, then it is determined that the current gesture space trajectory is not an intentional gesture trajectory; if allowed, then it is determined that the current gesture space trajectory is an intentional gesture trajectory.

[0016] Optionally, before acquiring the current gesture timing information through the gesture sensor, the method further includes:

[0017] When a valid key within a preset distance range is obtained through the vehicle control module, the gesture sensor, which is in a dormant state, is woken up by a low-power wake-up circuit in the vehicle control module, so as to obtain the current gesture timing information through the gesture sensor; the gesture sensor is a millimeter-wave radar sensor and / or a time-of-flight infrared sensor; the valid key is a Bluetooth key;

[0018] Accordingly, after waking up the gesture sensor from its dormant state via the low-power wake-up circuit in the vehicle control module, the method further includes:

[0019] If the gesture sensor does not acquire the current gesture timing information within a preset time range, the gesture sensor will automatically enter a sleep state.

[0020] Optionally, the hybrid deep learning model sequentially includes a convolutional neural network, a long short-term memory network, a fully connected layer, and a softmax layer;

[0021] Accordingly, the step of processing the current gesture timing information through the gesture recognition microcontroller built into the gesture sensor and based on a hybrid deep learning model to identify dynamic gesture information includes:

[0022] The gesture recognition microcontroller built into the gesture sensor extracts the local spatial information of each time step in the current gesture temporal information in parallel based on the multi-scale convolution kernel of the convolutional neural network, so as to obtain the static morphological features corresponding to the current gesture temporal information.

[0023] Based on the Long Short-Term Memory network, the temporal relationship of the static morphological features is learned to obtain the dynamic change process corresponding to the current gesture temporal information;

[0024] The dynamic change process is passed sequentially through the fully connected layer and the Softmax layer to identify dynamic gesture information.

[0025] Optionally, the gesture recognition-based intelligent vehicle control method further includes:

[0026] Obtain user-defined gesture trajectories and corresponding vehicle functions to update the preset gesture trajectories and corresponding vehicle functions.

[0027] Secondly, this application discloses an intelligent vehicle control device based on gesture recognition, applied to an intelligent vehicle control system, comprising:

[0028] The gesture information recognition module is used to acquire the current gesture timing information through a gesture sensor, and process the current gesture timing information through a gesture recognition microcontroller unit built into the gesture sensor and based on a hybrid deep learning model to identify dynamic gesture information; the dynamic gesture information includes a target gesture trajectory determined from preset gesture trajectories that corresponds to the current gesture spatial trajectory;

[0029] The first judgment module is used to determine whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module;

[0030] The second judgment module is used to determine, if yes, whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled in the current vehicle working state by means of the vehicle body control module and based on the preset function and working state mapping relationship; the vehicle working state includes the start state, the power-off state, and the power-on but not started state.

[0031] The vehicle control module is configured to, if permitted, send function commands to the corresponding execution device based on the target vehicle function via the body control module, so that the execution device can enable the target vehicle function based on the function commands and complete vehicle control.

[0032] Optionally, the gesture recognition-based intelligent vehicle control device further includes:

[0033] The sensor wake-up module is used to wake up the gesture sensor, which is in a dormant state, through a low-power wake-up circuit in the body control module when a valid key within a preset distance range is obtained by the body control module, so as to obtain the current gesture timing information through the gesture sensor; the gesture sensor is a millimeter-wave radar sensor and / or a time-of-flight infrared sensor; the valid key is a Bluetooth key.

[0034] The sensor sleep module is used to automatically put the gesture sensor into sleep mode if the gesture sensor does not acquire the current gesture timing information within a preset time range after being woken up by the low-power wake-up circuit in the body control module.

[0035] Thirdly, this application discloses an electronic device, including:

[0036] Memory, used to store computer programs;

[0037] A processor is used to execute the computer program to implement the aforementioned gesture recognition-based intelligent vehicle control method.

[0038] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned gesture recognition-based intelligent vehicle control method.

[0039] As can be seen, this application acquires the current gesture timing information through a gesture sensor, processes the current gesture timing information through a gesture recognition microcontroller built into the gesture sensor and based on a hybrid deep learning model to identify dynamic gesture information; the dynamic gesture information includes a target gesture trajectory determined from preset gesture trajectories that corresponds to the current gesture spatial trajectory; the vehicle control module determines whether the current gesture spatial trajectory is an intentional gesture trajectory; if so, the vehicle control module, based on a preset function and working state mapping relationship, determines whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled in the current vehicle working state; the vehicle working state includes a start state, a power-off state, and a power-on but not started state; if allowed, the vehicle control module sends a function command to the corresponding execution device based on the target vehicle function, so that the execution device enables the target vehicle function based on the function command and completes vehicle control. Therefore, this application obtains the target gesture trajectory corresponding to the current gesture space trajectory from the preset gesture trajectory through a hybrid deep learning model, which is beneficial to improving the gesture recognition accuracy. Before using the gesture trajectory to implement the corresponding function, this application also judges whether the trajectory is intentionally recognized. The judgment of intentional gesture trajectory can clearly distinguish between intentional gestures and unintentional actions, improve the accuracy of gesture recognition, and also prevent the vehicle from generating and executing corresponding function commands to activate the function based on gestures that are not actively issued or are unconscious, further improving the accuracy of function activation. Before using the gesture trajectory to implement the corresponding function, this application also judges whether the target vehicle function corresponding to the target gesture trajectory is allowed to be activated under the current vehicle working state. This judgment can ensure that the appropriate function is activated under the vehicle working state, improve the adaptability of the scene and state, avoid invalid responses, and improve the accuracy of gesture recognition and function activation. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0041] Figure 1 This is a flowchart of an intelligent vehicle control method based on gesture recognition disclosed in this application;

[0042] Figure 2 This is a schematic diagram of an intelligent vehicle control system architecture disclosed in this application;

[0043] Figure 3 This is a schematic diagram of the structure of an intelligent vehicle control device based on gesture recognition disclosed in this application;

[0044] Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Currently, vehicle (e.g., motorcycle) function control mainly relies on physical buttons, remote controls, or mobile apps. These methods all depend on signal input from third-party components, and the user experience is relatively limited. Especially in scenarios such as wearing gloves, rainy days, or nighttime, traditional control methods are slow to respond and prone to accidental touches. In recent years, some high-end models have attempted to introduce voice control or simple gesture recognition, but the problem of poor recognition accuracy still exists.

[0047] Therefore, this application proposes an intelligent vehicle control scheme based on gesture recognition, which can be applied to an intelligent vehicle control system to improve the accuracy of gesture recognition and thus improve the precision of vehicle control.

[0048] This application discloses an intelligent vehicle control method based on gesture recognition, applied to an intelligent vehicle control system. (See also...) Figure 1 As shown, the method includes:

[0049] Step S11: Obtain the current gesture timing information through the gesture sensor, process the current gesture timing information through the gesture recognition microcontroller built into the gesture sensor and based on a hybrid deep learning model to identify dynamic gesture information; the dynamic gesture information includes the target gesture trajectory corresponding to the current gesture spatial trajectory determined from the preset gesture trajectory.

[0050] See Figure 2The diagram shows an architecture of an intelligent vehicle control system (also known as an intelligent gesture interaction control system). It includes a gesture sensing layer, a vehicle state layer, and a decision execution layer. The gesture sensing layer includes gesture sensors, a gesture recognition algorithm MCU (Microcontroller Unit), and a low-power wake-up circuit. The vehicle state layer includes an ECU (Electronic Control Unit) (transmitter control unit), a Bluetooth key authentication module, and an IMU (Inertial Measurement Unit). The decision execution layer includes a BCM (Body Control Module), a dashcam (for taking photos or videos), and a seat lock driver. It should be noted that the gesture sensing layer is responsible for collecting and recognizing gesture signals; the vehicle state layer is responsible for key authentication and obtaining vehicle posture and status; and the decision execution layer executes corresponding functions based on gestures and vehicle status.

[0051] It should be noted that this application features a high degree of system integration, can be deployed based on existing BCM and CAN (Controller Area Network) networks, and is compatible with most mid-to-high-end vehicle platforms. Vehicles can include motorcycles, etc.

[0052] It should be noted that the gesture sensor located in the gesture sensing layer is a millimeter-wave radar sensor and / or a time-of-flight (TOF) infrared sensor; wherein, the millimeter-wave radar sensor can be a 60Hz millimeter-wave radar sensor, and the gesture sensor can be installed in a concealed location on the front of the vehicle, the sides of the vehicle body, the dashboard, or the rear of the vehicle. It should also be noted that the gesture sensor needs to support three-dimensional spatial gesture trajectory recognition, with an effective detection distance of 0.1m to 5m, and an angle coverage of ± In addition, the built-in low-power wake-up circuit is normally in sleep mode (power consumption <50uA) and is only woken up after authentication (that is, after Bluetooth key authentication is successful).

[0053] It should be noted that existing gesture recognition systems are mostly based on infrared or cameras, which are easily affected by lighting, occlusion, and background interference, resulting in a recognition rate of less than 85% and frequent false triggers, causing slow progress in recognition. This application achieves high-precision, interference-resistant gesture recognition through millimeter-wave radar / TOF sensors, with a recognition rate of >98%, and is suitable for various environments such as strong light, night, rain, and fog.

[0054] In this embodiment, before acquiring the current gesture timing information through the gesture sensor, the method further includes: when a valid key within a preset distance range is obtained through the vehicle control module, the gesture sensor, which is in a dormant state, is woken up by a low-power wake-up circuit in the vehicle control module, so as to acquire the current gesture timing information through the gesture sensor; the gesture sensor is a millimeter-wave radar sensor and / or a time-of-flight infrared sensor; the valid key is a Bluetooth key; correspondingly, after waking up the gesture sensor, which is in a dormant state, through the low-power wake-up circuit in the vehicle control module, the method further includes: if the gesture sensor does not acquire the current gesture timing information within a preset time range, the gesture sensor automatically enters a dormant state. It should be noted that the recognition distance of the Bluetooth key authentication module is adjustable (e.g., 1 to 10m), supports two-way authentication, and locks the gesture function for a fixed time if the authentication fails for a consecutive number of times. In addition, this application adopts a low-power wake-up mechanism, which only activates the sensor when a valid key is nearby, reducing static power consumption by more than 90%.

[0055] In one specific embodiment, the security and anti-accidental touch strategy for Bluetooth is as follows: 1. Each gesture operation must be completed within 10 seconds after successful authentication; 2. If recognition fails 3 times consecutively, the gesture function will be locked for 60 seconds; 3. All gesture events are recorded through the CAN bus, supporting fault backtracking.

[0056] In one specific embodiment, the BCM in this application periodically (2s) scans the Bluetooth signal. If a valid key (RSSI (Received Signal Strength Indicator) > -70dBm) is detected and the distance is <5m, the BCM wakes up the gesture sensor through a low-power wake-up circuit. If there is no gesture operation within 10 seconds, the sensor automatically goes into sleep mode.

[0057] It should be noted that the sensor is always in a working state, with high power consumption, and is not linked with the vehicle authentication system (such as Bluetooth key), which poses a risk of illegal operation. The wake-up mechanism is not intelligent. This application adopts a low-power wake-up mechanism, which only activates the sensor when a legitimate key is nearby, reducing static power consumption by more than 90%.

[0058] In this embodiment, the hybrid deep learning model sequentially includes a convolutional neural network, a long short-term memory network, a fully connected layer, and a softmax layer. Correspondingly, the step of processing the current gesture temporal information using the gesture recognition microcontroller built into the gesture sensor and based on the hybrid deep learning model to identify dynamic gesture information includes: extracting local spatial information of each time step in the current gesture temporal information in parallel using the gesture recognition microcontroller built into the gesture sensor and based on the multi-scale convolutional kernels of the convolutional neural network to obtain the static morphological features corresponding to the current gesture temporal information; learning the temporal relationship between the static morphological features based on the long short-term memory network to obtain the dynamic change process corresponding to the current gesture temporal information; and sequentially passing the dynamic change process through the fully connected layer and the softmax layer to identify the dynamic gesture information.

[0059] It should be noted that the gesture recognition algorithm is as follows: 1. Convolutional Neural Network (CNN) + Long Short-Term Memory (LSTM) is used to process the gesture temporal data: Data preparation: The sensor collects the spatiotemporal coordinates, velocity, and acceleration of the gesture at a frequency of 10Hz, forming a 30×9 temporal data matrix, which is then input into the model after denoising, normalization, and alignment (preprocessing operation); Spatial feature extraction: The CNN module uses multi-scale convolutional kernels (3 / 5 / 7) in parallel to extract the local spatial pattern of each time step, outputting a 96-dimensional feature vector to capture the static morphological features of the gesture; Temporal dependency modeling: The LSTM module receives the CNN features and learns the temporal relationship between the gesture actions through bidirectional LSTM, outputting a 32-dimensional hidden state to encode the dynamic change process of the gesture; Classification decision: The fully connected layer maps the LSTM output, and the Softmax outputs the probability distribution of 6 types of gestures. The result with a confidence level > 85% is taken as the effective recognition and reported through the CAN bus. It supports dynamic background removal and multi-target gesture separation with a recognition rate >98%. The algorithm runs on the sensor's built-in MCU (such as STM32H7) with a response time <100ms. The entire process achieves an end-to-end latency of <100ms on the STM32H7 with an accuracy rate >98%, realizing integrated recognition of gestures' "spatial form + temporal dynamics".

[0060] It should be noted that when training the network, convergence and stopping conditions should be set, such as the number of iterations reaching the target number, the accuracy reaching the target requirement, or the confidence level reaching the target confidence level; a loss function should also be set, but the specific loss function is not limited here.

[0061] Step S12: Determine whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module.

[0062] It should be noted that this application can determine whether the current gesture spatial trajectory is an intentional gesture trajectory through multimodal verification. Specifically, it can determine whether it is an intentional gesture trajectory through two methods: one is based on whether the current gesture spatial trajectory is continuous within a preset time window, and the other is based on the preset function and operating state prohibition mapping relationship. During the determination, it can be set that satisfying one of the corresponding conditions constitutes an intentional gesture trajectory, or it can be set that all conditions must be met for it to be an intentional gesture trajectory. It should also be noted that this application combines vehicle status with multimodal verification to achieve intelligent mapping of gesture functions and prevention of accidental touches, thereby improving safety and practicality.

[0063] In this embodiment, the dynamic gesture information includes gesture-related time information; correspondingly, the step of determining whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module includes: determining whether the current gesture spatial trajectory is continuous within a preset time window based on the gesture-related time information through the vehicle control module; if not, then determining that the current gesture spatial trajectory is not an intentional gesture trajectory; if so, then determining that the current gesture spatial trajectory is an intentional gesture trajectory.

[0064] It should be noted that when performing timing consistency checks, if the gestures are not continuous within the preset time window, it indicates that the gestures may have been made unintentionally by the user and cannot be considered as intentional gesture trajectories.

[0065] In this embodiment, determining whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module includes: determining whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled under the current vehicle operating state by the vehicle control module and based on a preset function and operating state prohibition mapping relationship; the vehicle operating state includes vehicle-related speed and vehicle posture; if not allowed, it is determined that the current gesture spatial trajectory is not an intentional gesture trajectory; if allowed, it is determined that the current gesture spatial trajectory is an intentional gesture trajectory.

[0066] It should be noted that in some practical situations, users may make unintentional actions in a chain reaction, such as when the vehicle's posture changes, causing changes in hand gestures. In such cases, gestures caused by the vehicle's operating state need to be excluded. In addition, in some cases, it is necessary to set a function that cannot be implemented, such as not being able to open the car mat when the vehicle is started and running. In these cases, functions that are not supported by the operating state need to be excluded.

[0067] It should be noted that the IMU (Inertial Measurement Unit) sensor in the vehicle state layer will detect the vehicle's tilt angle, vibration frequency, etc. in real time. The BCM will obtain relevant information such as vehicle speed, acceleration, tilt angle, and vibration frequency from the IMU, ECU, etc. in real time to determine the operating status and make subsequent judgments based on this relevant information.

[0068] In one specific embodiment, the multimodal verification mechanism is as follows: 1. Temporal consistency detection: the gesture trajectory is continuous within the time window; 2. IMU posture matching: such as vehicle tilt > 3. Speed-related judgment: Power-on / seat opening gestures are disabled when the engine speed is greater than 1000 rpm.

[0069] It should be noted that the operating status in multimodal mode can also include engine and powertrain information, body and comfort system information, battery, motor, fault and diagnostic information, electrical and power information (load, signal), etc., and it is possible to set which functions are prohibited under which operating status. Examples will not be given here.

[0070] It should be noted that this application combines vehicle sensors (such as IMU and speed signals) to verify gesture intent, clearly distinguishing between intentional gestures and unintentional actions.

[0071] Step S13: If yes, then through the vehicle control module and based on the preset function and working state mapping relationship, determine whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled in the current vehicle working state; the vehicle working state includes the start state, the power-off state, and the power-on but not started state.

[0072] In one specific embodiment, the correspondence between gestures and vehicle functions is as follows: Clenching a fist and opening it: seat opening; drawing a circle: taking a photo; V-sign: powering on the vehicle; swiping left and right: switching instrument panel modes; waving up and down: adjusting volume; hovering for 2 seconds: activating the voice assistant. It should be noted that each gesture includes a specific process, such as the starting position of the circle within the entire circle's range, the direction of the circle, the size of the circle, and the overall time setting for drawing the circle. For waving up and down, the amplitude of the wave, the number of fingers waving, and the overall time setting for waving are also considered. Other gestures can be configured similarly.

[0073] In one specific embodiment, the seat opening process is as follows: 1. The vehicle is powered off, and the user enters a 5m range with a valid key; 2. The BCM wakes up the gesture sensor; 3. The user makes a "clenched fist - open" gesture, and the sensor recognizes it and sends a CAN message to the BCM; 4. The BCM verifies that the vehicle status is "not started" and executes the seat lock motor drive command.

[0074] In one specific embodiment, the photo-taking process is as follows: 1. The vehicle is in a powered-on but not started state; 2. The user makes a "circle drawing" gesture; 3. The BCM verifies that the vehicle is powered on and forwards the instruction to the dashcam host; 4. The dashcam takes a photo and stores it.

[0075] In one specific embodiment, the vehicle power-on process is as follows: 1. The vehicle is powered off and the key authentication is successful; 2. The user makes a "V" sign; 3. The BCM performs the power-on action.

[0076] It should be noted that the gestures used in this application can be pre-set or user-defined. Specifically, a personalized recognition model can be generated by collecting 10 samples through an app (application or software). Specifically, the user-defined gesture trajectory and corresponding vehicle function are obtained to update the preset gesture trajectory and corresponding vehicle function. It should be noted that this process can not only add new gesture trajectories and corresponding vehicle functions, but also modify existing gesture trajectories and corresponding vehicle functions; for example, changing the gesture corresponding to a certain function, or the function corresponding to a certain gesture, etc.

[0077] It should be noted that in the decision-making and execution layer, the BCM acts as the main control unit, receiving the gesture recognition results and mapping them to the corresponding functions based on the vehicle status; the dashcam host acts as an auxiliary control node, receiving the gesture commands forwarded by the BCM and performing operations such as taking photos and recording videos.

[0078] In one specific embodiment, based on the above correspondence between gestures and vehicle functions, the preset function and working state mapping relationship can be partially set as follows: when the vehicle is in the start state, functions such as taking photos and voice assistants are supported, but the power-off function is not supported; when the vehicle is in the power-off state, functions such as power-on and seat opening are supported, but the functions such as taking photos, semantic assistants, and instrument switching are not supported; when the vehicle is in the power-on but not start state, only functions such as seat opening, taking photos, instrument switching, and voice assistants are supported, but the power-on function is not supported.

[0079] It should be noted that the lack of dynamic mapping of gesture functions in conjunction with vehicle status (such as power on / off) leads to function conflicts or invalid responses and poor scenario adaptability. However, this application clarifies the mapping relationship between preset functions and working status, thereby improving scenario adaptability.

[0080] Step S14: If permitted, the vehicle control module sends a function command to the corresponding execution device based on the target vehicle function, so that the execution device can enable the target vehicle function based on the function command and complete vehicle control.

[0081] In this embodiment, user interaction and extended functions are added, such as visual / auditory feedback (corresponding feedback is set for both gesture recognition failure and success, which can be configured via turn signals and buzzers): when gesture recognition is successful, the turn signal flashes once or the buzzer sounds briefly once; when recognition fails, the turn signal flashes twice or the buzzer sounds briefly twice. Subsequently, the gesture model supports OTA (Over-the-Air Technology) remote updates; the system can also automatically collect user operating habits and optimize the recognition threshold (including parameters in the network, hyperparameters, loss function content, etc.). It should be noted that this application supports user-defined gestures and OTA upgrades, possessing good scalability and personalized adaptation capabilities.

[0082] In this embodiment, fault handling and logging functions can be added. For example, if gesture recognition fails continuously, the system automatically switches to a backup control mode (such as button control). (For example, if the target number of authentication failures is continuous, the gesture function is locked for a fixed time, and the backup control mode is used intelligently. When not locked, the system can flexibly choose which mode to use.) All operation logs are stored in BCM flash memory and support reading via the UDS (Unified Diagnostic Services) diagnostic protocol.

[0083] As can be seen, this application acquires the current gesture timing information through a gesture sensor, processes the current gesture timing information through a gesture recognition microcontroller built into the gesture sensor and based on a hybrid deep learning model to identify dynamic gesture information; the dynamic gesture information includes a target gesture trajectory determined from preset gesture trajectories that corresponds to the current gesture spatial trajectory; the vehicle control module determines whether the current gesture spatial trajectory is an intentional gesture trajectory; if so, the vehicle control module, based on a preset function and working state mapping relationship, determines whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled in the current vehicle working state; the vehicle working state includes a start state, a power-off state, and a power-on but not started state; if allowed, the vehicle control module sends a function command to the corresponding execution device based on the target vehicle function, so that the execution device enables the target vehicle function based on the function command and completes vehicle control. Therefore, this application obtains the target gesture trajectory corresponding to the current gesture space trajectory from the preset gesture trajectory through a hybrid deep learning model, which is beneficial to improving the gesture recognition accuracy. Before using the gesture trajectory to implement the corresponding function, this application also judges whether the trajectory is intentionally recognized. The judgment of intentional gesture trajectory can clearly distinguish between intentional gestures and unintentional actions, improve the accuracy of gesture recognition, and also prevent the vehicle from generating and executing corresponding function commands to activate the function based on gestures that are not actively issued or are unconscious, further improving the accuracy of function activation. Before using the gesture trajectory to implement the corresponding function, this application also judges whether the target vehicle function corresponding to the target gesture trajectory is allowed to be activated under the current vehicle working state. This judgment can ensure that the appropriate function is activated under the vehicle working state, improve the adaptability of the scene and state, avoid invalid responses, and improve the accuracy of gesture recognition and function activation.

[0084] Accordingly, this application also discloses an intelligent vehicle control device based on gesture recognition, applied to an intelligent vehicle control system, see [link to relevant documentation]. Figure 3 As shown, the device includes:

[0085] The gesture information recognition module 11 is used to acquire the current gesture timing information through the gesture sensor, and process the current gesture timing information through the gesture recognition microcontroller unit built into the gesture sensor and based on a hybrid deep learning model to identify dynamic gesture information; the dynamic gesture information includes a target gesture trajectory corresponding to the current gesture spatial trajectory determined from a preset gesture trajectory.

[0086] The first judgment module 12 is used to determine whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle body control module.

[0087] The second judgment module 13 is used to determine, if yes, whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled in the current vehicle working state by means of the vehicle body control module and based on the preset function and working state mapping relationship; the vehicle working state includes the start state, the power-off state and the power-on but not started state.

[0088] The vehicle control module 14 is used, if permitted, to send function commands to the corresponding execution device based on the target vehicle function through the body control module, so that the execution device can enable the target vehicle function based on the function commands and complete vehicle control.

[0089] The more specific working process of each of the above modules can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0090] Furthermore, embodiments of this application also provide an electronic device. Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0091] Figure 4 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the gesture recognition-based intelligent vehicle control method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0092] In this embodiment, the power supply 26 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 24 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0093] Furthermore, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include computer programs 221, and the storage method can be temporary storage or permanent storage. The computer programs 221, in addition to including computer programs capable of performing the gesture recognition-based intelligent vehicle control method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks.

[0094] Furthermore, embodiments of this application also disclose a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned intelligent vehicle control method based on gesture recognition.

[0095] The specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0096] The various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For the same or similar parts between the various embodiments, refer to each other. As for the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and relevant parts can be referred to in the method section.

[0097] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0098] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0099] Finally, it should be noted that in this document, relational terms such as "first" and "first" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0100] The above provides a detailed description of the intelligent vehicle control method, device, equipment, and storage medium based on gesture recognition provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for controlling intelligent vehicles based on gesture recognition, characterized in that, Applied to intelligent vehicle control systems, including: The gesture sensor acquires the current gesture timing information, and the gesture recognition microcontroller built into the gesture sensor processes the current gesture timing information based on a hybrid deep learning model to identify dynamic gesture information; the dynamic gesture information includes a target gesture trajectory determined from preset gesture trajectories that corresponds to the current gesture spatial trajectory. The vehicle control module determines whether the current gesture spatial trajectory is an intentional gesture trajectory. If so, the vehicle control module determines whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled under the current vehicle working state based on the preset function and working state mapping relationship; the vehicle working state includes the start state, the power-off state, and the power-on but not started state; If permitted, the vehicle body control module sends function commands to the corresponding execution device based on the target vehicle function, so that the execution device can activate the target vehicle function based on the function commands and complete vehicle control.

2. The intelligent vehicle control method based on gesture recognition according to claim 1, characterized in that, The dynamic gesture information includes gesture-related time information; Accordingly, the step of determining whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module includes: The vehicle control module determines whether the current gesture spatial trajectory is continuous within a preset time window based on the gesture-related time information. If not, then the current gesture space trajectory is determined to be an intentional gesture trajectory; if yes, then the current gesture space trajectory is determined to be an intentional gesture trajectory.

3. The intelligent vehicle control method based on gesture recognition according to claim 1, characterized in that, The step of determining whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module includes: The vehicle control module determines, based on a preset function and operating status prohibition mapping relationship, whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled under the current vehicle operating status; the vehicle operating status includes vehicle-related speed and vehicle posture. If not allowed, then it is determined that the current gesture space trajectory is not an intentional gesture trajectory; if allowed, then it is determined that the current gesture space trajectory is an intentional gesture trajectory.

4. The intelligent vehicle control method based on gesture recognition according to claim 1, characterized in that, Before acquiring the current gesture timing information through the gesture sensor, the method further includes: When a valid key within a preset distance range is obtained through the vehicle control module, the gesture sensor, which is in a dormant state, is woken up by a low-power wake-up circuit in the vehicle control module, so as to obtain the current gesture timing information through the gesture sensor; the gesture sensor is a millimeter-wave radar sensor and / or a time-of-flight infrared sensor; the valid key is a Bluetooth key; Accordingly, after waking up the gesture sensor from its dormant state via the low-power wake-up circuit in the vehicle control module, the method further includes: If the gesture sensor does not acquire the current gesture timing information within a preset time range, the gesture sensor will automatically enter a sleep state.

5. The intelligent vehicle control method based on gesture recognition according to claim 1, characterized in that, The hybrid deep learning model comprises, in sequence, a convolutional neural network, a long short-term memory network, a fully connected layer, and a softmax layer; Accordingly, the step of processing the current gesture timing information through the gesture recognition microcontroller built into the gesture sensor and based on a hybrid deep learning model to identify dynamic gesture information includes: The gesture recognition microcontroller built into the gesture sensor extracts the local spatial information of each time step in the current gesture temporal information in parallel based on the multi-scale convolution kernel of the convolutional neural network, so as to obtain the static morphological features corresponding to the current gesture temporal information. Based on the Long Short-Term Memory network, the temporal relationship of the static morphological features is learned to obtain the dynamic change process corresponding to the current gesture temporal information; The dynamic change process is passed sequentially through the fully connected layer and the Softmax layer to identify dynamic gesture information.

6. The intelligent vehicle control method based on gesture recognition according to any one of claims 1 to 5, characterized in that, Also includes: Obtain user-defined gesture trajectories and corresponding vehicle functions to update the preset gesture trajectories and corresponding vehicle functions.

7. A smart vehicle control device based on gesture recognition, characterized in that, Applied to intelligent vehicle control systems, including: The gesture information recognition module is used to acquire the current gesture timing information through a gesture sensor, and process the current gesture timing information through a gesture recognition microcontroller unit built into the gesture sensor and based on a hybrid deep learning model to identify dynamic gesture information; the dynamic gesture information includes a target gesture trajectory determined from preset gesture trajectories that corresponds to the current gesture spatial trajectory; The first judgment module is used to determine whether the current gesture spatial trajectory is an intentional gesture trajectory through the vehicle control module; The second judgment module is used to determine, if yes, whether the target vehicle function corresponding to the target gesture trajectory is allowed to be enabled in the current vehicle working state by means of the vehicle body control module and based on the preset function and working state mapping relationship; the vehicle working state includes the start state, the power-off state, and the power-on but not started state. The vehicle control module is configured to, if permitted, send function commands to the corresponding execution device based on the target vehicle function via the body control module, so that the execution device can enable the target vehicle function based on the function commands and complete vehicle control.

8. The intelligent vehicle control device based on gesture recognition according to claim 7, characterized in that, Also includes: The sensor wake-up module is used to wake up the gesture sensor, which is in a dormant state, through a low-power wake-up circuit in the body control module when a valid key within a preset distance range is obtained by the body control module, so as to obtain the current gesture timing information through the gesture sensor; the gesture sensor is a millimeter-wave radar sensor and / or a time-of-flight infrared sensor; the valid key is a Bluetooth key. The sensor sleep module is used to automatically put the gesture sensor into sleep mode if the gesture sensor does not acquire the current gesture timing information within a preset time range after being woken up by the low-power wake-up circuit in the body control module.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the gesture recognition-based intelligent vehicle control method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the gesture recognition-based intelligent vehicle control method as described in any one of claims 1 to 6.

Images