In-vehicle environment adjusting method and device, electronic equipment and storage medium

Abstract

The application relates to an in-vehicle environment adjusting method and device, electronic equipment and a storage medium. Hand movement data and physiological signal data of a driver are collected through a wearable device worn by the driver; driving situation information of a vehicle is obtained; feature extraction is performed on the hand movement data, and an unconscious gesture is identified, the unconscious gesture being a hand movement of the driver under an active control intention; the identified unconscious gesture, the physiological signal data and the driving situation information are analyzed in combination to obtain a current state of the driver; and control instructions for adjusting in-vehicle environment parameters are generated and executed according to the current state of the driver and historical preference data of the driver. Through sensing of the unconscious gesture and the physiological signal of the driver and deep combination and personalized learning in combination with the driving situation information, natural, intelligent and personalized self-adaptive adjustment of the in-vehicle environment is realized.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method, device, electronic device and storage medium for regulating the in-vehicle environment. Background Technology

[0002] With the rapid development of smart cockpit technology, adaptive adjustment of the in-vehicle environment based on human-computer interaction has become a research hotspot in the industry. Existing gesture control technology based on electromyography (EMG) signals collects forearm muscle electrical signals through multi-channel EMG sensors built into wristbands. AI models then recognize specific gestures made by the user (such as rubbing fingers or making circles), thereby enabling contactless control of vehicle equipment such as windows and the trunk.

[0003] However, existing electromyographic gesture control technology is mainly geared towards scenarios where users consciously and actively control the system. Users need to deliberately make specific gestures to trigger commands. Although this explicit control frees up the hands, it fails to utilize the unconscious behavioral information naturally revealed by the driver during driving. Summary of the Invention

[0004] This application provides a method, device, electronic device, and storage medium for adjusting the in-vehicle environment, in order to solve the technical problem that existing gesture control technology lacks the recognition and control logic for the unconscious gestures that drivers naturally express in specific situations and have individual differences, thus resulting in limited accuracy and personalization of environmental adjustment.

[0005] Firstly, this application provides a method for adjusting the in-vehicle environment, the method comprising: The driver's hand movement data and physiological signal data are collected through wearable devices worn by the driver; and the driving situation information of the vehicle is obtained. Feature extraction is performed on the hand movement data to identify unconscious gestures, which are hand movements made by the driver without active control intention. The driver's current state is obtained by fusing and analyzing the identified unconscious gestures, physiological signal data, and driving context information. Based on the driver's current state and the driver's historical preference data, control commands for adjusting in-vehicle environmental parameters are generated and executed.

[0006] In one possible implementation, feature extraction is performed on the hand movement data to identify unconscious gestures, including: The hand movement data is divided into windows using a sliding window mechanism to obtain the intensity index corresponding to the hand movement data within each time window; If the intensity index corresponding to the hand movement data within the time window exceeds the preset index threshold and the hand movement data does not belong to the preset normal driving operation, the time window is determined as the motion detection window. Extract the feature vector of each motion detection window and input the feature vector into a pre-trained unconscious gesture recognition model to obtain the gesture recognition result corresponding to the motion detection window; A temporal consistency check is performed on the gesture recognition results corresponding to multiple consecutive motion detection windows. If the number of times the same gesture appears in a preset number of consecutive motion detection windows reaches a preset threshold, the same gesture is determined to be an unconscious gesture.

[0007] In one possible implementation, the unconscious gesture recognition model is trained in the following manner: Obtain a training dataset containing hand movement data labeled with unconscious gesture categories; The training dataset is augmented to obtain an augmented training dataset; Using the augmented training dataset, the initial model is iteratively trained using a weighted cross-entropy loss function. During the training process, an adaptive learning optimizer is used to update the model parameters until the preset iteration stopping condition is met, resulting in a well-trained unconscious gesture recognition model.

[0008] In one possible implementation, the identified unconscious gestures, physiological signal data, and driving context information are fused and analyzed to obtain the driver's current state, including: The unconscious gestures, physiological signal data, and driving context information are fused using multimodal methods to obtain a fused feature vector; The fused feature vector is input into a pre-trained state classification model to obtain a first driver state; wherein, the state classification model refers to a pre-trained deep learning model, which is used to input the fused feature vector and output the probability of each preset state; the input layer of the state classification model receives the fused feature vector, multiple hidden layers complete feature mapping, random deactivation layers randomly shut down neurons, and the output layer outputs the probability of each preset state; the first driver state is the preset state with the highest probability. The driver state that matches the fused feature vector in the preset rule base is determined as the second driver state; The current state of the driver is determined based on the first driver state and the second driver state.

[0009] In one possible implementation, determining the driver's current state based on the first driver state and the second driver state includes: Obtain the model confidence level corresponding to the first driver state; If the model confidence level is greater than or equal to a preset confidence threshold, the first driver state is determined as the driver's current state. If the model confidence level is less than a preset confidence threshold, the second driver state is determined as the driver's current state.

[0010] In one possible implementation, based on the driver's current state and the driver's historical preference data, control commands for adjusting in-vehicle environmental parameters are generated and executed, including: Based on the driver's current state, a preset mapping rule is queried to obtain candidate environmental adjustment parameters corresponding to the driver's current state; Obtain the driver's historical preference data, which includes at least the driver's feedback data on environmental adjustment results under the same historical state as the current state; Based on the historical preference data, the candidate environmental adjustment parameters are adjusted, and control commands for adjusting the in-vehicle environmental parameters are generated and executed.

[0011] In one possible implementation, adjusting the candidate environment adjustment parameters based on the historical preference data includes: The driver's current state, unconscious gestures, and driving context information are input into a personalized learning model pre-trained based on the historical preference data to obtain the driver's preference adjustment parameters under the same or similar states; wherein, the personalized learning model refers to a model pre-trained based on collaborative filtering or reinforcement learning, used to input the driver's current state, unconscious gestures, and driving context information, and output the driver's preference adjustment parameters. The candidate environment adjustment parameters are modified or replaced using the preference adjustment parameters.

[0012] Secondly, this application provides an in-vehicle environment adjustment device, the device comprising: The data acquisition module is used to collect the driver's hand movement data and physiological signal data through a wearable device worn by the driver; and to obtain the vehicle's driving context information. The gesture recognition module is used to extract features from the hand movement data and identify unconscious gestures, which are hand movements made by the driver without active control intention. The state determination module is used to fuse and analyze the identified unconscious gestures, physiological signal data, and driving context information to obtain the current state of the driver. The instruction generation module is used to generate and execute control instructions for adjusting in-vehicle environmental parameters based on the driver's current state and the driver's historical preference data.

[0013] In one possible implementation, the gesture recognition module is specifically used for: The hand movement data is divided into windows using a sliding window mechanism to obtain the intensity index corresponding to the hand movement data within each time window; If the intensity index corresponding to the hand movement data within the time window exceeds the preset index threshold and the hand movement data does not belong to the preset normal driving operation, the time window is determined as the motion detection window. Extract the feature vector of each motion detection window and input the feature vector into a pre-trained unconscious gesture recognition model to obtain the gesture recognition result corresponding to the motion detection window; A temporal consistency check is performed on the gesture recognition results corresponding to multiple consecutive motion detection windows. If the number of times the same gesture appears in a preset number of consecutive motion detection windows reaches a preset threshold, the same gesture is determined to be an unconscious gesture.

[0014] In one possible implementation, the unconscious gesture recognition model in the gesture recognition module is trained in the following way: Obtain a training dataset containing hand movement data labeled with unconscious gesture categories; The training dataset is augmented to obtain an augmented training dataset; Using the augmented training dataset, the initial model is iteratively trained using a weighted cross-entropy loss function. During the training process, an adaptive learning optimizer is used to update the model parameters until the preset iteration stopping condition is met, resulting in a well-trained unconscious gesture recognition model.

[0015] In one possible implementation, the state determination module includes: The feature vector extraction module is used to perform multimodal fusion of the unconscious gestures, the physiological signal data, and the driving context information to obtain a fused feature vector; The first driver state determination unit is used to input the fused feature vector into a pre-trained state classification model to obtain the first driver state; wherein, the state classification model refers to a pre-trained deep learning model, which is used to input the fused feature vector and output the probability of each preset state; the input layer of the state classification model receives the fused feature vector, multiple hidden layers complete feature mapping, random deactivation layers randomly shut down neurons, and the output layer outputs the probability of each preset state. The second driver state determination unit is used to determine the driver state that matches the fused feature vector in the preset rule base as the second driver state. The driver current state determination unit is used to determine the current state of the driver based on the first driver state and the second driver state.

[0016] In one possible implementation, the driver's current state determination unit is specifically used for: Obtain the model confidence level corresponding to the first driver state; If the model confidence level is greater than or equal to a preset confidence threshold, the first driver state is determined as the driver's current state. If the model confidence level is less than a preset confidence threshold, the second driver state is determined as the driver's current state.

[0017] In one possible implementation, the instruction generation module includes: An environmental adjustment parameter determination unit is used to query a preset mapping rule based on the driver's current state to obtain candidate environmental adjustment parameters corresponding to the driver's current state. A historical preference data acquisition unit is used to acquire the driver's historical preference data, which includes at least the driver's feedback data on environmental adjustment results in the same historical state as the current state. The control command generation unit is used to adjust the candidate environmental adjustment parameters based on the historical preference data, and generate and execute control commands for adjusting the in-vehicle environmental parameters.

[0018] In one possible implementation, the control instruction generation unit is specifically used for: The driver's current state, unconscious gestures, and driving context information are input into a personalized learning model pre-trained based on the historical preference data to obtain the driver's preference adjustment parameters under the same or similar states; wherein, the personalized learning model refers to a model pre-trained based on collaborative filtering or reinforcement learning, used to input the driver's current state, unconscious gestures, and driving context information, and output the driver's preference adjustment parameters. The candidate environment adjustment parameters are modified or replaced using the preference adjustment parameters.

[0019] Thirdly, this application provides an electronic device, including: a processor and a memory, wherein the processor is configured to execute an in-vehicle environment adjustment program stored in the memory to implement the in-vehicle environment adjustment method described in any one of the first aspects.

[0020] Fourthly, this application provides a storage medium storing one or more programs that can be executed by one or more processors to implement the in-vehicle environment adjustment method described in any one aspect.

[0021] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application collects the driver's hand movement data and physiological signal data through a wearable device worn by the driver; obtains the vehicle's driving context information; extracts features from the hand movement data to identify unconscious gestures, which are hand movements performed by the driver under active control intent; fuses and analyzes the identified unconscious gestures, physiological signal data, and driving context information to obtain the driver's current state; and generates and executes control commands for adjusting in-vehicle environmental parameters based on the driver's current state and historical preference data. By sensing the driver's unconscious gestures and physiological signals, and combining them with driving context information for deep fusion and personalized learning, natural, intelligent, and personalized adaptive adjustment of the in-vehicle environment is achieved. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0025] Figure 1 A flowchart illustrating an embodiment of a method for adjusting the in-vehicle environment provided in this application; Figure 2A flowchart illustrating an embodiment of another in-vehicle environment adjustment method provided in this application; Figure 3 A flowchart illustrating another embodiment of the in-vehicle environment adjustment method provided in this application; Figure 4 A structural block diagram of an in-vehicle environment regulation device provided in an embodiment of this application; Figure 5 An electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0028] To address the technical problem that existing gesture control technologies lack the recognition and control logic for drivers' naturally occurring, individualized, unconscious gestures in specific situations, thus limiting the accuracy and personalization of environmental adjustment, this application provides a method, device, electronic device, and storage medium for in-vehicle environment adjustment. By sensing the driver's unconscious gestures and physiological signals, and combining them with driving context information for deep fusion and personalized learning, natural, intelligent, and personalized adaptive adjustment of the in-vehicle environment is achieved.

[0029] Figure 1 A flowchart illustrating an embodiment of a vehicle interior environment adjustment method provided in this application includes the following steps: Step 101: Collect the driver's hand movement data and physiological signal data through the wearable device worn by the driver; and obtain the vehicle's driving situation information.

[0030] Wearable devices can refer to portable smart devices worn by drivers on their wrists or other parts of their bodies, such as screenless wristbands, smartwatches, smart rings, etc. These smart devices usually have multiple built-in sensors to collect human body data.

[0031] Hand motion data refers to data collected by various sensors in wearable devices that characterize the driver's micro-movements. These micro-movements can be understood as spontaneous, small-amplitude hand movements performed by the driver without any conscious intention. For example, hand motion data reflecting changes in the driver's hand displacement, velocity, angular velocity, and posture can be collected.

[0032] Physiological signal data can refer to the physiological indicator data of the driver collected by various sensors in wearable devices; optionally, the aforementioned physiological indicator data may include heart rate, heart rate variability, skin conductivity, etc.

[0033] Driving context information can refer to contextual information related to the current driving environment and vehicle status obtained through vehicle buses (such as CAN bus, Ethernet). Examples include vehicle speed, steering wheel angle, road conditions (congested / free), driving duration, navigation information, weather, time, etc., but this application embodiment does not impose limitations on this. Here, by interpreting driver behavior within a specific driving environment, misjudgments are avoided, thereby achieving accurate inference of the driver's state.

[0034] In one embodiment, hand movement data and physiological signal data are collected by a wearable device worn by the driver, while driving context information is obtained from the vehicle bus, providing multi-dimensional, spatiotemporally aligned raw data for subsequent unconscious gesture recognition and state determination.

[0035] Specifically, data is collected through various sensors integrated within a wearable device worn on the driver's wrist. Simultaneously, real-time driving context information, such as vehicle speed, steering wheel angle, pedal opening, navigation conditions, driving duration, ambient temperature, and time, is read via the vehicle's CAN bus or in-vehicle Ethernet. Furthermore, the vehicle's infotainment system maintains a multi-source data buffer, using a unified timestamp benchmark to precisely align the collected data across the timeline.

[0036] For example, during rush hour traffic congestion, driver Zhang was wearing a wearable smart bracelet. When the vehicle was moving intermittently on the expressway and the navigation indicated severe congestion ahead, the bracelet's accelerometer detected that Zhang's right hand was repeatedly tapping the edge of the steering wheel with a period of approximately 1.5 seconds, with an acceleration amplitude between 0.5 and 0.8g, and this pattern had lasted for more than 30 seconds. The bracelet's electromyography (EMG) sensor simultaneously captured regular, low-intensity EMG activity of the forearm flexor muscles consistent with the tapping rhythm. The bracelet's photoplethysmography (PPG) sensor showed that Zhang's heart rate increased from 72 bpm at rest to 85 bpm, the LF / HF ratio of heart rate variability increased from 1.2 to 2.5, and the skin conductance level also showed a significant increase.

[0037] Meanwhile, information from the vehicle's bus indicated that the vehicle speed was only 5-15 km / h, the steering wheel was making frequent, minor corrections, the navigation congestion level was "severe congestion," the estimated travel time was 20 minutes, and the automatic headlights were already on at dusk. All of this collected data was packaged into a timestamped multimodal data frame and sent in real time to subsequent steps for unconscious gesture recognition.

[0038] By combining and collecting various data sources, including physiological signals and vehicle information, a rich and complementary source of raw signals is provided for unconscious gesture recognition. This allows for the simultaneous capture of hand displacement, muscle intentions without displacement, and the driver's internal physiological state, fundamentally ensuring the accuracy of subsequent recognition. Secondly, by synchronously collecting data on the driver's hand behavior and physiological changes within specific driving scenarios (such as traffic jams, long-distance travel, and nighttime), the contextual reasons for the actions can be understood. For example, frequent tapping during traffic jams is more likely to represent anxiety than simply a personal habit, effectively avoiding misjudgments caused by isolated data and achieving true "contextualized perception."

[0039] Step 102: Extract features from hand movement data to identify unconscious gestures, which are hand movements made by the driver without active control intent.

[0040] Feature extraction refers to the process of extracting quantifiable feature parameters that can distinguish gestures from raw sensor signals. For example, the extracted features may include time-domain features, frequency-domain features, electromyographic features, etc. Through the above feature extraction process, the raw sensor signals are transformed into quantifiable feature vectors.

[0041] Unconscious gestures refer to hand movements that occur spontaneously by the driver without any subjective intention to operate in-vehicle equipment, influenced by emotions and driving scenarios. These unconscious gestures differ from standardized command gestures deliberately made by the driver to control windows, audio / video systems, etc. The hand movements corresponding to these unconscious gestures are induced by internal states such as anxiety, fatigue, and irritability, as well as driving environments such as traffic jams and long distances. The movements are mostly spontaneous micro-movements such as small tapping, repeated gripping, finger rubbing, and slight wrist rotation. They are not vehicle control commands but only reflect the driver's physical and mental state.

[0042] In one embodiment, feature extraction and pattern recognition are performed on the collected hand movement data to filter out movement patterns that meet the definition of "unconscious gestures," providing clear behavioral labels for subsequent fusion analysis.

[0043] Specifically, one possible approach is to preprocess the collected raw hand movement data, extracting key information reflecting changes in hand movements from the continuously collected data and eliminating invalid or interfering data. Using pre-defined judgment logic, the system distinguishes between normal driving actions, deliberate control gestures, and subconscious hand movements, selecting only those spontaneous hand movements generated when the driver has no intention of controlling the vehicle. These selected movements are then categorized and labeled to create standardized behavioral tags, which are used in subsequent analysis of the driver's state in conjunction with physiological and driving contextual information.

[0044] Optionally, the aforementioned preset judgment logic can refer to pre-set judgment criteria, which are used to identify and filter out unconscious gestures without active control intentions from all hand movements. The judgment can be made from three dimensions: action purpose, action form, and occurrence scenario.

[0045] For example, the determination rule based on the purpose of the action can be: if the action is performed for normal driving operations such as steering, shifting gears, or manually controlling the vehicle, or for actively controlling the in-vehicle equipment, it can be excluded as an unconscious gesture; the determination rule based on the form of the action can be: if the action is a small-amplitude, repetitive activity, rather than a single large-amplitude limb movement, it can be included in the candidate unconscious gesture range; the determination rule based on scene matching can be: if the candidate hand action occurs in a specific scenario such as congested driving or long-term constant speed driving, it can be determined as an unconscious gesture.

[0046] For example, a driver wearing a wearable wristband is driving in congested urban traffic during rush hour. The wristband continuously collects raw data on hand movements. First, the raw data is preprocessed to filter out invalid interference data such as vehicle vibrations and extract key quantitative information reflecting hand movements. Then, it is verified according to a pre-set three-tiered judgment rule: At the purpose level, since the vehicle is moving slowly, the driver is not performing any driving operations such as shifting gears or steering, nor is there any active control behavior such as adjusting the air conditioning or audio system via gestures, thus excluding active actions; at the action form level, since the driver's fingers are captured repeatedly tapping the steering wheel in small, repetitive movements, the action is not considered a large-scale physical movement and is classified as an unconscious gesture; at the scenario level, due to the prolonged traffic congestion and slow speed, the scenario matches the driving scenario corresponding to irritability, ultimately confirming the action as an unconscious gesture. Finally, the action is labeled with a frequent tapping behavior tag. This tag, along with information such as heart rate and traffic congestion, is used to comprehensively determine the driver's emotional state in subsequent analyses.

[0047] As another possible implementation, the raw sensor data is preprocessed and then segmented using a sliding window. Motion detection is performed on each sliding window, and the motion intensity index corresponding to each window is calculated. Windows with hand movements are filtered out based on the motion intensity index, while windows corresponding to normal driving actions such as steering and shifting are removed. The remaining windows are marked as motion detection windows.

[0048] Furthermore, quantifiable key features that can distinguish hand movements are extracted from multiple motion detection windows to generate standardized feature vectors. These standardized feature vectors are then input into a pre-trained unconscious gesture recognition model to obtain gesture recognition results for multiple motion detection windows. Finally, temporal consistency is checked on the recognition results of multiple consecutive motion detection windows. If temporal consistency is satisfied, i.e., the threshold for the number of times the same gesture appears is met, it is ultimately determined to be an unconscious gesture. If the same gesture appears at least twice in three consecutive motion detection windows, then the same gesture appearing multiple times in multiple motion detection windows is confirmed as the final unconscious gesture.

[0049] For example, during rush hour congestion, driver Zhang's wristband collects real-time hand movement and electromyography (EMG) signal data. After anomaly removal, gravity separation, and filtering of the raw data, continuous data windows are created, each with a 5-second window and a 0.5-second step. For one window (and similarly for other motion detection windows), the calculated motion intensity index is 0.8, far exceeding the preset intensity index of 0.6, indicating strong hand movement. This window is then identified as a motion detection window. Features in the time domain, frequency domain, and time-frequency domain are extracted from this window and reduced to 64 dimensions to obtain a feature vector. This feature vector is then input into a pre-trained unconscious gesture recognition model. The model outputs the unconscious gesture recognition result as "frequent tapping," with a model confidence score of 0.92. Since the model confidence score exceeds 0.85, the model's output gesture recognition result, "frequent tapping," is determined as the gesture recognition result corresponding to this motion detection window.

[0050] Finally, multiple motion detection windows and their corresponding gesture recognition results were obtained. Then, the temporal consistency of the gesture recognition results of multiple motion detection windows was checked. It was found that "frequent tapping" appeared twice in three consecutive motion detection windows, satisfying the condition of at least two identical taps. Finally, it was confirmed that Zhang's unconscious gesture was "frequent tapping" with a confidence level of 0.92.

[0051] By using multidimensional feature extraction and unconscious gesture recognition models, the system can accurately capture the subtle, repetitive unconscious gestures naturally expressed by drivers, significantly improving recognition accuracy. In addition, by using temporal consistency verification, the system eliminates recognition fluctuations caused by single-window noise or boundary effects, ensuring the smoothness and stability of gesture output and providing reliable behavioral labels for subsequent state determination.

[0052] Step 103: The identified unconscious gestures, physiological signal data, and driving context information are fused and analyzed to obtain the driver's current state.

[0053] Fusion analysis refers to a technical method that jointly processes and comprehensively infers data from different modalities. Specifically, it can include feature-level fusion, decision-level fusion, or attention-based fusion, and this application does not limit this approach. By comprehensively utilizing three types of heterogeneous information—gesture, physiological, and scene information—it overcomes the susceptibility to interference from single modalities, significantly improving the accuracy of state determination and scene adaptability.

[0054] The driver's current state can refer to the driver's overall internal condition output after multimodal fusion analysis. For example, it can include: calm, anxiety, tension, fatigue, anger, boredom, etc. These states integrate behavioral performance (gestures), physiological indicators (heart rate, skin conductance), and external environment (road conditions, duration) to drive the regulation of the in-vehicle environment.

[0055] In one embodiment, unconscious gestures, physiological signal data, and driving context information are fused in a multimodal manner to generate a fused feature vector; the fused feature vector is input into a pre-trained state classification model to obtain a first driver state, while a preset rule base is queried in parallel to obtain a second driver state; the driver's current state is determined based on the first driver state and the second driver state.

[0056] The aforementioned first driver state can refer to the current state of the driver as determined by a preset state classification model based on multimodal data.

[0057] The second driver status can refer to the driver's current status obtained by matching multimodal data according to a preset rule base.

[0058] Specifically, the system can simultaneously collect the driver's unconscious gesture information, physiological signal data, and driving context information of the current driving environment. This information from three different sources is integrated and refined to form a unified fusion feature vector. First, the fusion feature vector is fed into a pre-trained state classification model, which infers and outputs the driver's current state, identifying it as the first driver state. Second, according to a pre-defined rule base, the system searches for the driver's current state that matches the driver's unconscious gesture information, physiological signal data, and driving context information of the current driving environment, identifying it as the second driver state. Finally, the model output is combined with the first and second driver states obtained from rule matching to determine the driver's final actual state.

[0059] Furthermore, the current state of the driver can be determined based on the first driver state and the second driver state in the following way: First, obtain the model confidence level corresponding to the first driver state output by the model: If the model confidence level corresponding to the first driver state is greater than or equal to a preset confidence level threshold, the first driver state is determined as the final current state of the driver; if the model confidence level corresponding to the first driver state is less than the preset confidence level threshold, the second driver state is determined as the final current state of the driver.

[0060] Specifically, on the one hand, after obtaining the unconscious gesture recognition results, the recognized unconscious gestures are fused with the driver's physiological signal data and the vehicle's driving context information for multimodal analysis. First, gesture feature vectors, physiological feature vectors, and driving context feature vectors are constructed separately, and each modal feature vector is normalized and time-aligned. Then, at least one of the following methods is selected: feature-level fusion, decision-level fusion, or attention-based fusion, to integrate the above feature vectors into a unified fused feature vector. This fused feature vector is input into a pre-trained three-layer fully connected state classification model, which outputs the probability that the driver belongs to a preset state. The preset state can refer to calm, anxiety, tension, fatigue, anger, boredom, etc. The preset state with the highest probability is determined as the first driver state, and its corresponding probability is determined as the model confidence. The current state of the driver output by the model and its corresponding model confidence are output together.

[0061] Meanwhile, on the other hand, a pre-defined rule base is queried in parallel. This rule base includes pre-defined mapping relationships, such as "the current state of the driver corresponding to 'frequent tapping + congestion + a ratio of low-frequency power to high-frequency power in heart rate variability in physiological signals > 2.0' is anxiety." The rule base is used to find the current state of drivers that successfully match the current multimodal data, and this is determined as the second driver state. Finally, a confidence threshold decision logic is used: if the model confidence corresponding to the first driver state is ≥ 0.80, the first driver state is directly adopted; if the confidence corresponding to the first driver state is < 0.80, the second driver state is determined as the final current state of the driver.

[0062] For example, in a congested evening rush hour scenario, Zhang's gestures are recognized as "frequent tapping" (confidence 0.68). The ratio of low-frequency power to high-frequency power in the heart rate variability of the physiological signal is 2.3, indicating severe congestion. The recognized multimodal data is fused and input into a pre-trained state classification model, outputting a probability of "boredom" of 0.68 (first driver state), and matching it with "anxiety" (second driver state) according to preset rules. Because the model confidence 0.68 < 0.80, the second driver state "anxiety" is ultimately determined as the driver's current state, and environmental adjustments are triggered based on this current state.

[0063] The above embodiments achieve accurate and robust assessment of driver status. By comprehensively utilizing three types of multimodal data, the shortcomings of single-modal data being susceptible to interference are effectively overcome, significantly improving the accuracy of status determination and scenario adaptability; it also provides stable and reliable status input for subsequent personalized learning, supporting continuous optimization.

[0064] Step 104: Based on the driver's current state and the driver's historical preference data, generate and execute control commands to adjust the parameters of the in-vehicle environment.

[0065] Historical preference data refers to personalized feedback data recorded from past driver interactions with environmental adjustments. This personalized feedback data can include explicit feedback (such as voice or screen confirmation, rejection, or adjustment of automatic adjustment schemes) and implicit feedback (such as manual adjustment of environmental parameters, changes in unconscious gesture frequency, and restoring parameters to their pre-adjustment state). By learning the unique mapping between each driver's current state and preference adjustments, an adaptive experience that "understands the user better with use" can be achieved.

[0066] Control commands can refer to digital commands generated based on the driver's current state and historical preferences, used to change the operating parameters of in-vehicle environmental equipment. These commands can be sent to actuators such as ambient lighting, audio systems, air conditioning, seats, and fragrance systems; this application embodiment does not impose any limitations on this. Based on these control commands, a single state determination is transformed into an immersive, multi-dimensional environmental change, enhancing the naturalness and satisfaction of the user experience, while reducing secondary user intervention through personalized adjustments.

[0067] In one embodiment, after identifying the driver's current state, the system queries a preset general mapping rule to obtain candidate environment adjustment parameters, retrieves the driver's historical preference data including explicit and implicit feedback, inputs the driver's current state, driving context, and unconscious gestures into a personalized learning model trained on the historical preference data, outputs preference adjustment parameters, and uses the preference adjustment parameters to correct or replace the candidate environment adjustment parameters to generate the final control command. The control command is sent to actuators such as air conditioning, audio system, ambient lighting, seats, and fragrance via the vehicle bus to complete the adjustment of in-vehicle environment parameters through multi-device collaboration.

[0068] The aforementioned personalized learning model relies on the driver's historical preference data for model training. The historical preference data includes both explicit and implicit feedback. During training, the driver's current state, driving situation, and unconscious gestures recorded in each iteration are used as model inputs, and the environmental adjustment parameters actually formed by the driver are used as the model's target outputs. The model continuously iterates and optimizes the internal mapping relationship, learning the unique correspondence between the driver's current state and preference adjustment. After training, the personalized learning model can output the corresponding preference adjustment parameters based on real-time inputs.

[0069] Optionally, the personalized learning model described above is set up with a four-layer structure. The input layer receives three types of input data: the driver's current state, driving context, and unconscious gestures. The feature fusion layer fuses the features of the above three types of data and associates them with explicit and implicit feedback information in historical preferences. The mapping learning layer explores the inherent mapping relationship between various types of input information and user adjustment habits. Finally, the output layer outputs preference adjustment parameters that match the user's usage habits.

[0070] This personalized learning model can set differentiated fusion weights for explicit and implicit feedback in historical preferences during the feature fusion stage, strengthening the correlation between the driver's current state, driving context, unconscious gestures, and user adjustment habits. In the mapping learning layer, it uses various new preference data added by the driver to achieve incremental model updates to adapt to changes in user preferences. At the same time, it combines the candidate environment adjustment parameters corresponding to the general mapping rules to constrain the model output range, and dynamically adjusts the feature weights of the three types of input information according to the actual scenario, thereby optimizing the accuracy of the preference adjustment parameters output by the model.

[0071] For example, in an initial traffic congestion anxiety scenario, the ambient lighting is initially set to blue and soothing piano music is played, following general rules. Driver Wang, finding the blue light too cold, manually changes the lights to a warm purple and switches the music to jazz. This explicit feedback is recorded. When Wang experiences traffic congestion anxiety again, the personalized learning model automatically corrects the candidate parameters from "blue ambient lighting" to "warm purple" and "soothing piano music" to "jazz music" based on historical preference data, generating personalized control commands. With increased usage, it can even learn subtle differences in Wang's preferences under varying levels of congestion, achieving adaptive adjustments.

[0072] By introducing a personalized learning mechanism, the system continuously learns users' unique adjustment preferences from their historical feedback, fundamentally solving the problem that traditional "one-size-fits-all" rules cannot meet personalized needs. The combination of explicit and implicit feedback reduces the burden of active user annotation, achieving seamless preference collection. Based on the learned preference parameters, general candidate solutions are modified or replaced, ensuring that each environmental adjustment meets both current requirements and personal habits, significantly improving user satisfaction and the cabin's intelligence level. Simultaneously, this mechanism possesses an adaptive capability that "understands the user better with use," reducing the frequency of repeated manual intervention by the driver and creating a more natural and comfortable driving experience.

[0073] The method provided in this application collects hand movement data and physiological signal data from a wearable device worn by the driver; acquires vehicle driving context information; extracts features from the hand movement data to identify unconscious gestures, which are hand movements performed by the driver under active control intent; fuses and analyzes the identified unconscious gestures, physiological signal data, and driving context information to obtain the driver's current state; and generates and executes control commands to adjust in-vehicle environmental parameters based on the driver's current state and historical preference data. By sensing the driver's unconscious gestures and physiological signals, and combining them with driving context information for deep fusion and personalized learning, natural, intelligent, and personalized adaptive adjustment of the in-vehicle environment is achieved.

[0074] Figure 2 A flowchart illustrating another embodiment of the in-vehicle environment adjustment method provided in this application is shown below. Figure 1 Based on the illustrated process, this section mainly describes how to identify unconscious gestures, including the following steps: Step 201: Collect the driver's hand movement data and physiological signal data through the wearable device worn by the driver; and obtain the vehicle's driving situation information.

[0075] For step 201 above, please refer to the above. Figure 1 Detailed description of the relevant embodiments.

[0076] Step 202: Use a sliding window mechanism to divide the hand motion data into windows and obtain the intensity index corresponding to the hand motion data in each time window.

[0077] The sliding window mechanism refers to dividing the raw sensor data collected from multiple sensors in the wearable device worn by the driver into overlapping data segments with fixed durations and fixed sliding step sizes, allowing for segment-by-segment analysis. By cutting the continuous sensor data stream into short, highly overlapping segments of fixed length (e.g., 5 seconds) with a 90% overlap rate, it ensures that unconscious gestures (typically lasting 1-4 seconds) are not truncated by window boundaries, while simultaneously enabling real-time segment-by-segment processing. Compared to point-by-point processing, windowing significantly reduces computational complexity; the high overlap rate ensures the temporal sensitivity of gesture detection, triggering analysis within a very short time (approximately 0.5 seconds) after the gesture begins, meeting the real-time requirements of in-vehicle systems.

[0078] A time window can refer to a single, fixed-duration segment of hand movement data, segmented using a sliding window mechanism. For example, driver hand movement data is continuously collected, with each segment defined as a fixed-duration time window. These windows overlap by sliding steps; for instance, a fixed window duration corresponds to a segment of collected data. Each time, a small slide is made to capture a new window, retaining most of the data from the previous window within the new window. The entire process of a driver making an unconscious hand gesture falls within multiple overlapping time windows, without being truncated by window separations. Each time window calculates its own corresponding hand movement intensity index independently.

[0079] Intensity index refers to a quantitative indicator that characterizes the intensity of hand movements during a single time window, calculated from hand movement data within that window. This intensity index can quickly determine whether significant hand movements exist within the current window, serving as a "pre-filter" for subsequent feature extraction and model inference. It avoids unnecessary complex calculations for static windows or minor noise, saving onboard computing resources, reducing power consumption, minimizing invalid model calls, and improving overall processing efficiency.

[0080] In one embodiment, continuous hand movement data transmitted from a wearable device is continuously received. The data is divided into individual time windows according to a set fixed duration, and new time windows are continuously generated by sliding backward according to a fixed step size. Most of the overlapping data between time windows is retained. The hand movement data within each divided time window is processed sequentially to calculate the intensity index of each time window. This intensity index is used as a pre-screening condition, and only time windows exceeding a threshold are sent to the subsequent feature extraction and recognition process, thereby filtering out static or low-energy windows and reducing computational resource consumption.

[0081] Specifically, the system continuously receives the continuous hand movement data stream uploaded in real time from the wearable device. An initial time window is defined according to a pre-set fixed duration. Then, subsequent time windows are generated by periodically sliding backward according to a predetermined sliding step size. Adjacent time windows reuse most of the overlapping data. The hand movement data contained in each time window is extracted one by one to calculate the intensity index. The intensity index is compared with the preset threshold. Only time windows with indices exceeding the threshold are retained and transferred to the subsequent feature extraction and recognition process. Static and low-energy windows with no obvious hand activity are discarded.

[0082] The aforementioned preset threshold is the boundary for distinguishing between invalid and valid windows. When the intensity index calculated by the time window is lower than the threshold, the window content is determined to contain only sensor noise or normal slight hand movements, which are considered static or low-energy windows and are directly blocked without proceeding to feature extraction and recognition. When the intensity index is higher than the threshold, the window is identified as containing valid hand movements, which are likely to contain unconscious gestures, and is allowed to proceed to subsequent processing steps. This threshold filtering reduces invalid calculations and saves onboard computing resources.

[0083] For example, wearable devices continuously upload hand movement data streams while driving. The system first defines an initial time window, then continuously slides backward at fixed steps to generate new time windows, with adjacent windows reusing most of the overlapping data. When the driver is holding the steering wheel normally, the intensity index calculated for each time window is below a preset threshold, and these static, low-energy windows are directly discarded. When the driver unconsciously scratches their face, the intensity index calculated for several time windows containing this action exceeds the preset threshold, and the relevant windows are successfully retained and sent to the subsequent feature extraction and recognition process.

[0084] Optionally, the above intensity index can be obtained by summarizing all hand movement data within a single time window. Select all raw hand movement data collected by wearable devices within a single time window, calculate the fluctuation difference and dispersion of the sampled data within the window, integrate the variation amplitude of all sampling points for summarization, and convert it into a single value as the intensity index. When the hand is almost still, the data fluctuation is small, and the summarization result is low. After limb movements such as raising the hand occur, the data fluctuation becomes larger, and the summarization intensity index value increases accordingly. Finally, use the summarization value and compare it with a preset threshold to complete the window filtering.

[0085] This significantly saves computing resources, eliminating the need to run complex deep learning models in every time window. Only time windows that are truly "likely to be a gesture" require in-depth analysis, reducing the burden and power consumption of the automotive chip. It offers excellent real-time performance; overlapping time windows and checks every 0.5 seconds allow the system to detect gestures less than half a second after they begin, ensuring no subtle movements are missed. It also boasts strong anti-interference capabilities; the intensity index is insensitive to road bumps and slight engine vibrations, triggering only when the driver makes a significant hand movement, avoiding frequent wake-ups of subsequent processes due to noise or accidental touches.

[0086] Step 203: If the intensity index corresponding to the hand movement data within the time window exceeds the preset index threshold and the hand movement data does not belong to the preset normal driving operation, the time window is determined as the motion detection window.

[0087] The index threshold can refer to the pre-set critical value of the intensity index, which is used to distinguish between static motion data and obvious hand activity data. It is the dividing standard for judging the magnitude of the motion intensity index.

[0088] Preset normal driving operations can refer to the range of actions corresponding to routine hand operations during vehicle driving, which are defined in advance, including routine driving hand behaviors such as gripping the steering wheel and making minor steering adjustments.

[0089] The motion detection window can refer to a time window that meets two conditions at the same time: the window intensity index exceeds the index threshold and the hand movement in the window is not within the normal driving operation range. This window is marked as a valid action window and is used for subsequent unconscious gesture recognition.

[0090] In one embodiment, each time window for which the intensity index has been calculated is read one by one. First, the intensity index of the time window is compared with the preset index threshold. Time windows in which the intensity index exceeds the index threshold are filtered out. Then, the hand movement data in the window is compared with the motion features corresponding to the preset normal driving operation. Window belonging to normal driving operation is eliminated. The remaining time windows that meet both conditions are marked as motion detection windows.

[0091] Specifically, each time window after intensity index calculation, along with the corresponding hand movement data and the calculated intensity index, is retrieved sequentially. The intensity index of the current time window is compared with the preset index threshold, and time windows with intensity indices less than or equal to the threshold are filtered out, while candidate time windows with indices exceeding the threshold are retained. For the remaining candidate time windows, the hand movement data within the time window is compared with the motion features corresponding to preset normal driving operations, and time windows where the action matches normal driving operations are eliminated. The remaining time windows that simultaneously meet the conditions of exceeding the intensity index and the action not belonging to normal driving operations are uniformly marked as motion detection time windows and proceed to the subsequent unconscious gesture recognition stage.

[0092] The intensity index corresponding to each time window can be calculated based on the original sensor data of the hand's three-axis acceleration collected by the wearable device. First, the instantaneous motion amplitude after removing the gravity component is calculated for each sampling point within the window. Then, the average value is calculated based on the amplitude of all sampling points. Next, the intensity index corresponding to the time window is obtained by calculating the standard deviation formula. When the driver holds the steering wheel firmly, the sampling amplitude fluctuation is weak, and the calculated intensity index is small and lower than the preset threshold. The corresponding window is directly screened out. When the driver makes actions such as rubbing his eyes, the sampling amplitude fluctuates significantly, and the calculated intensity index exceeds the threshold, becoming a candidate window. Subsequently, the motion characteristics are combined to identify whether it belongs to normal driving operation, and finally determine whether it can be marked as a motion detection window.

[0093] The triaxial acceleration of the hand refers to the raw values ​​of real-time hand acceleration in the three orthogonal directions of X, Y, and Z, which are collected by the accelerometer built into the wearable device worn by the driver. It is used to comprehensively reflect the movement changes of the hand in three dimensions of space and is the basic raw parameter for calculating hand movement data.

[0094] Instantaneous motion amplitude refers to the value obtained by calculating the resultant acceleration based on triaxial acceleration and subtracting the fixed gravity component at a single sampling point. It is used to quantify the magnitude of the actual limb movement generated by the hand swaying at that moment, excluding the static acceleration caused by its own weight. In a static state, the instantaneous motion amplitude approaches zero. The greater the range of hand movement, the higher the instantaneous motion amplitude value.

[0095] For example, the process involves retrieving hand movement data and calculated intensity metrics for each time window. For instance, a wearable sensor worn by the driver simultaneously collects hand acceleration data in three directions: forward / backward, left / right, and up / down. These three sets of data constitute the three-axis hand acceleration. Taking a sampling time window of a stationary grip on the steering wheel as an example, the accelerations in the three directions are integrated to calculate the overall acceleration magnitude. After removing the fixed value caused by gravity, the remaining result is the instantaneous motion amplitude at that sampling point.

[0096] Furthermore, if the driver makes slight adjustments to the direction while holding the steering wheel, and the corresponding window intensity index does not exceed the threshold, it is directly filtered out. When a candidate window with an intensity index exceeding the threshold appears, the action characteristics are verified. If the hand movement data of the candidate window matches the preset normal driving operation such as holding the steering wheel, it is also eliminated. When the driver makes an unconscious eye-rubbing action, the corresponding window intensity index exceeds the threshold, and the hand movement data does not match the characteristics of normal driving operation. This window is marked as a motion detection time window and sent to the subsequent unconscious gesture recognition stage.

[0097] By excluding normal driving behaviors such as steering and gear shifting, it avoids misidentifying necessary driver maneuvers as unconscious gestures, triggering subsequent processing only on micro-movements that truly reflect emotions or fatigue, significantly improving the specificity of recognition. An efficient "coarse screening + fine screening" pipeline is constructed: an intensity index threshold serves as the first coarse screening, quickly filtering out a large number of static or low-noise windows; driving operation exclusion serves as the second fine screening, accurately eliminating normal driving behaviors. After these two screenings, only a small number of windows truly containing unconscious gestures enter the high-overhead deep learning inference, maximizing computational resource utilization. It can adapt to complex driving environments; specific preset driving operation rules can cover common key actions (such as lane changing, turning, and gear shifting), and the thresholds are adjustable, ensuring both effective exclusion and sensitivity to unconscious gestures, demonstrating strong engineering practicality and scenario adaptability.

[0098] Step 204: Extract the feature vector of each motion detection window and input the feature vector into the pre-trained unconscious gesture recognition model to obtain the gesture recognition result corresponding to the motion detection window.

[0099] In one embodiment, the unconscious gesture recognition model is trained by: acquiring a training dataset containing hand motion data labeled with unconscious gesture categories; augmenting the training dataset to obtain an augmented training dataset; using the augmented training dataset, iteratively training the initial model with a weighted cross-entropy loss function, and updating the model parameters using an adaptive learning optimizer during the training process until a preset iteration stopping condition is met, thereby obtaining a trained unconscious gesture recognition model.

[0100] In terms of architecture, the unconscious gesture recognition model uses one-dimensional convolutional downsampling to extract local motion features, combines a two-layer bidirectional LSTM (Long Short-Term Memory) network to learn the temporal relationship of gestures, and adds self-attention to focus on key moments to improve the accuracy of micro-movement recognition. In terms of training, data augmentation processing is performed on the training dataset to simulate real driving interference, and weighted cross-entropy and user-split cross-validation are used to enhance the generalization ability.

[0101] A feature vector can refer to the set of quantized parameters extracted from the motion detection window. For example, at least time-domain features, frequency-domain features, and time-frequency-domain features can be extracted from the motion detection window, and then dimensionality-reduced to form a vector of fixed dimensions (such as 64 dimensions).

[0102] Optionally, the training dataset can refer to a sample library containing a large amount of hand movement data labeled with unconscious gesture categories and labeled current state of the driver, with samples collected from multiple subjects and multiple scenarios.

[0103] Furthermore, the weighted cross-entropy loss function can assign different weights to samples of different classes (the class with fewer samples has a larger weight), alleviating the class imbalance problem. An adaptive learning optimizer can dynamically adjust the learning rate based on gradient history, accelerating convergence. The iteration stopping condition can be that the validation set loss does not decrease for several consecutive training iterations, or that the maximum number of training iterations is reached.

[0104] Specifically, all raw hand motion data within the labeled motion detection window are retrieved one by one. Various quantized feature parameters in the time domain, frequency domain, and time-frequency domain are extracted from the hand motion data. Redundant parameters are then dimensionality-reduced and standardized into a fixed-dimensional parameter combination to generate the feature vector corresponding to that window. The generated feature vector is then input into a pre-trained and converged unconscious gesture recognition model; the model outputs the unconscious gesture recognition result corresponding to the current motion detection window.

[0105] For example, the time window corresponding to the driver's unconscious eyebrow-touching hand movement data has been identified as the motion detection window. All hand movement data within this motion detection window is read, and various quantitative parameters such as temporal fluctuation magnitude, frequency change, and time-frequency change are extracted. Redundant data is removed, and the data is uniformly organized into a fixed 64-dimensional parameter combination to generate a feature vector specific to this window. This 64-dimensional feature vector is then input into a pre-trained unconscious gesture recognition model, yielding the model's output recognition result: the current action is an unconscious eyebrow-touching gesture.

[0106] By combining multidimensional features with a deep learning model, the system can accurately distinguish subtle, unconscious gestures such as tapping, grasping, rubbing, and rotating. Data augmentation strategies during model training simulate various individual differences and environmental disturbances encountered in real-world driving, enabling the model to maintain stable recognition capabilities even with unfamiliar drivers, vibration characteristics of different vehicle models, and poor sensor contact.

[0107] Step 205: Perform temporal consistency verification on the gesture recognition results corresponding to multiple consecutive motion detection windows. If the number of times the same gesture appears in a preset number of consecutive motion detection windows reaches a preset threshold, the same gesture is identified as an unconscious gesture.

[0108] Temporal consistency verification can refer to performing majority voting or conditional judgment on the recognition results of multiple consecutive motion detection windows to smooth the output and avoid misjudgment from a single motion detection window. For example, the gesture is finally confirmed only if at least two of the three consecutive motion detection windows output the same gesture.

[0109] The preset number of consecutive motion detection windows can refer to the pre-defined number of motion detection windows, such as 3 consecutive motion detection windows. The number of times the same gesture occurs can refer to the frequency with which the same gesture category is recognized within the consecutive motion detection windows. The preset threshold can refer to the minimum number of occurrences required to determine if a recognized gesture is valid, such as at least 2 times.

[0110] Unconscious gestures typically last for several seconds, while the time step of a sliding window is usually 0.5 seconds. Single motion detection window recognition may experience brief jitter. The timing consistency check effectively avoids frequent triggering or cancellation of environmental adjustments due to brief misjudgments through a "majority voting" and "lagging confirmation" mechanism, thus improving the stability and naturalness of the user experience.

[0111] In one embodiment, multiple motion detection windows generated consecutively and their respective gesture recognition results are cached in chronological order; a preset number of consecutive motion detection windows are selected as a set of statistical intervals, and the occurrence frequency of each gesture is counted in each interval; the count of each type of gesture is compared with a preset threshold; if the occurrence frequency of a certain gesture meets the standard, the action is confirmed as a genuine unconscious gesture of the driver; recognition results that do not meet the threshold are judged as misrecognitions caused by sensor interference or instantaneous data anomalies, and the recognition conclusion is directly discarded.

[0112] A statistical interval refers to a fixed and continuous set of motion detection windows extracted in chronological order. It serves as the statistical range for centrally analyzing the results of various gesture recognition methods. The frequency of the same gesture is counted only within this defined continuous window, avoiding arbitrary counting across an infinite timeframe. Since a driver's genuine unconscious gestures last for a short period, they will inevitably fall within a continuous series of windows, not appearing sporadically with long intervals. False recognitions mostly occur in isolated instances within a single window; a defined interval is necessary to accurately reflect the actual continuity of the gestures. Without defining an interval and infinitely accumulating the recognition results from all windows throughout the day, two eye rubs a few minutes apart would be counted together, leading to repeated misjudgments as continuous gestures. Defining a fixed-length interval, counting only within the nearest continuous window, avoids interference from invalid data from distant locations. Specifically, the corresponding statistical interval can be dynamically set according to the actual application.

[0113] For example, suppose we preset five consecutive motion detection windows to count the actions, with a threshold of the same gesture appearing ≥ three times. If the five consecutive motion detection windows identify the following actions in sequence: rubbing eyes, rubbing eyes, slightly adjusting the steering wheel, rubbing eyes, and raising a hand, and the driver rubs their eyes three times, reaching the threshold, then the driver is determined to have unconsciously rubbed their eyes. If only one of the five motion detection windows identifies the driver as resting their chin on their hand, and the rest are normal actions, the count does not meet the threshold, and the action is considered a misidentification and not an unconscious gesture.

[0114] Because the gesture is only updated when multiple motion detection windows are consistent, the system avoids frequent triggering or reversal of lighting, music, and other adjustment actions due to brief misrecognitions, resulting in a smoother and more natural environmental response. This aligns with the continuous nature of unconscious gestures, which typically last for several seconds. Timing verification leverages this continuity to ensure that only genuine and stable gestures take effect, thus improving the reliability of recognition.

[0115] Step 206: The identified unconscious gestures, physiological signal data, and driving context information are fused and analyzed to obtain the driver's current state.

[0116] Step 207: Based on the driver's current state and the driver's historical preference data, generate and execute control commands for adjusting the parameters of the in-vehicle environment.

[0117] For steps 206-207 above, please refer to the above. Figure 1 Detailed description of the relevant embodiments.

[0118] pass Figure 2 The description of the embodiment shown shows that by pre-screening with sliding window strength index and excluding normal driving operations, the computational overhead and false alarm rate are greatly reduced. By combining the unconscious gesture recognition model with time consistency verification, while ensuring real-time performance and high accuracy, the output of smooth and stable unconscious gestures provides reliable input for subsequent personalized environmental adjustments.

[0119] Figure 3 A flowchart illustrating another embodiment of the in-vehicle environment adjustment method provided in this application is shown below. Figure 1 Based on the illustrated process, this section mainly describes how to determine the driver's current state based on unconscious gestures and generate control commands based on the driver's current state, including the following steps: Step 301: Collect the driver's hand movement data and physiological signal data through the wearable device worn by the driver; and obtain the vehicle's driving situation information.

[0120] Step 302: Extract features from hand movement data to identify unconscious gestures, which are hand movements made by the driver without active control intention.

[0121] For steps 301-302 above, please refer to the above. Figure 1 Detailed description of the relevant embodiments.

[0122] Step 303: Perform multimodal fusion on unconscious gestures, physiological signal data, and driving context information to obtain a fused feature vector.

[0123] Multimodal fusion can refer to a technical method that combines three heterogeneous data—unconscious gestures, physiological signal data, and driving context information—for joint processing. It can employ features-level splicing, decision-level weighted voting, or attention mechanisms, etc., to generate a unified fusion feature vector. This application does not limit the specific fusion method used in this embodiment.

[0124] In one embodiment, three types of feature vectors are constructed, namely unconscious gestures, physiological signal data and driving context information. After normalization and time alignment, they are fused using at least one of feature-level splicing, decision-level weighting or attention mechanism to generate a unified fused feature vector.

[0125] Optionally, extract the single-dimensional gesture feature vectors corresponding to the identified unconscious gestures; for example, perform noise reduction and normalization processing on physiological signal data such as heart rate and skin conductance to generate physiological feature parameters; analyze driving context information such as vehicle speed, road conditions, day / night, and traffic congestion, and quantify it into standardized context features. Scale the three types of data to the same numerical range to eliminate differences in data dimensions. Concatenate the processed gesture features, physiological features, and driving context features directly in dimensional order to integrate the three heterogeneous information types into a single new vector; alternatively, attention fusion can be used, autonomously allocating the proportion of the three types of information through attention weights, assigning higher weights to key information. After concatenation / weighting, simplify and remove redundancy from the overall vector to generate a fixed-dimensional fusion feature vector for subsequent downstream control logic such as in-vehicle comfort adjustment and fatigue alerts.

[0126] For example, gesture features are 64-dimensional vectors, physiological signal processing results in 32-dimensional vectors, and driving environment quantization results in 16-dimensional vectors. These are directly concatenated to obtain a 112-dimensional fused feature vector, thus completing multimodal fusion.

[0127] This step effectively overcomes the vulnerability of single paths to interference by complementing multimodal information, significantly improving the accuracy of subsequent state determination and scene adaptability, while providing high-quality, low-redundancy input for the classification model.

[0128] Step 304: Input the fused feature vector into the pre-trained state classification model to obtain the first driver's state.

[0129] A fusion feature vector can refer to a fixed-dimensional numerical vector obtained after multimodal fusion, which serves as the input to a state classification model.

[0130] A state classification model can refer to a pre-trained deep learning model that takes a fused feature vector as input and outputs the probability that the driver belongs to each preset state. This model can employ a fully connected neural network: Input layer: receives a fixed-dimensional fused feature vector; Multiple hidden layers: multiple fully connected layers perform deep feature mapping transformations; Random deactivation layers: interspersed between hidden layers, randomly shutting down some neurons; Output layer: activated using a normalized exponential activation function, outputting the classification probability corresponding to each preset state.

[0131] Preset states can refer to predefined labels for the driver's physical and mental state while driving, which may include: comfortable and relaxed, drowsy and tired, irritable and anxious, physical discomfort, etc.

[0132] The first driver state can refer to the state with the highest probability output by the state classification model, and its corresponding model confidence, i.e., the probability value.

[0133] In one embodiment, the generated fused feature vector is input into a pre-trained state classification model. The model uses a three-layer fully connected network and is trained with weighted cross-entropy loss and Adam optimizer. It can output the probability that the driver belongs to a preset state such as calm, anxious, tense, tired, angry, or bored. The state with the highest probability is the first driver state, and its probability value is used as the model confidence.

[0134] The aforementioned state classification model deeply integrates unconscious gestures, physiological signal data, and driving contextual information. It employs a lightweight three-layer fully connected network to output the probability and confidence level of preset states (such as calm, anxiety, tension, fatigue, anger, and boredom). Compared to traditional single-modality or simple rule-based methods, this model can automatically learn complex nonlinear mappings between heterogeneous features, avoiding the explosion problem of manually designed rule combinations. Simultaneously, it alleviates sample imbalance through weighted cross-entropy loss, uses cross-validation based on participant partitioning to assess generalization ability for new users, and utilizes an early stopping strategy to prevent overfitting. The model's output confidence level directly participates in subsequent decisions; high confidence levels are directly adopted, while low confidence levels trigger rule fallback or inertia retention, thus significantly improving the accuracy and robustness of state determination while ensuring real-time performance (millisecond-level inference).

[0135] For example, after the 30-dimensional fused feature vector is inferred by the model, the probability of outputting "anxiety" is 0.86, and the probability of all other states is less than 0.10. Therefore, the first driver's state is "anxiety" with a confidence level of 0.86.

[0136] This step transforms multimodal information into actionable state labels, avoiding the complexity of manually combining rules. At the same time, the given confidence level can quantify the reliability of the judgment, providing a clear basis for whether to trust the model or enable rule fallback. Furthermore, the lightweight network structure meets the real-time requirements of in-vehicle systems.

[0137] Step 305: Determine the driver state that matches the fused feature vector in the preset rule base as the second driver state.

[0138] A predefined rule base can refer to a set of predefined rules, where each rule maps a specific combination of unconscious gestures, physiological thresholds, and driving scenario conditions to a driver state.

[0139] The second driver status can refer to the status obtained after matching the rule base with the current fused feature vector.

[0140] In one embodiment, a preset rule base is queried in parallel, and the current unconscious gestures, physiological signal data, and driving context information are matched with the conditions in the rule base. If all conditions of a certain rule are met, the driver state corresponding to that rule is output as the second driver state; if no matching rule is found, the second driver state is empty.

[0141] For example, a rule in the rule base is "Frequent tapping + Congestion level ≥ 2 + Heart rate > 2.0 → Anxiety". The current driver's gesture is frequent tapping, the congestion level is 3, and the heart rate is 2.3. All conditions are matched, therefore the second driver's state is determined to be "anxious".

[0142] Based on a pre-defined rule base, it can quickly and interpretably provide state determinations in typical scenarios, serving as a supplement or fallback to the model output; especially in the low confidence stage of the model or during the cold start phase, it can ensure that the system still outputs reasonable states, thereby improving overall robustness.

[0143] Step 306: Determine the current state of the driver based on the state of the first driver and the state of the second driver.

[0144] In one embodiment, the model confidence level corresponding to the first driver state is obtained; if the model confidence level is greater than or equal to a preset confidence threshold, the first driver state is determined as the driver's current state; if the model confidence level is less than the preset confidence threshold, the second driver state is determined as the driver's current state.

[0145] Model confidence refers to the maximum probability value output by the state classification model, reflecting the degree of certainty the model has about the first driver's state.

[0146] The preset confidence threshold can refer to a pre-defined confidence level (e.g., 0.80), used to decide whether to use the model output or the rule output. When the model confidence level is greater than or equal to the threshold, the first driver state is used; otherwise, the second driver state is used.

[0147] In one embodiment, the model confidence level corresponding to the first driver state is obtained and compared with a preset confidence threshold (e.g., 0.80): if the confidence level is greater than or equal to the threshold, the first driver state is directly used as the current state of the driver; if the confidence level is less than the threshold, the second driver state is used.

[0148] For example, if the model outputs a confidence score of 0.86 (≥0.80) for "anxiety", then the current state is determined to be "anxiety". If the model confidence score is only 0.65 and the rule matching outputs "nervous", then the current state is "nervous"; if there is no rule matching, then the previous state is maintained.

[0149] By making dynamic decisions based on confidence thresholds, the system trusts the model when confidence is high and uses rules as a fallback when confidence is low. This approach leverages the generalization ability of deep learning while ensuring reliable output under uncertainty, thus avoiding misadjustment.

[0150] Step 307: Based on the driver's current state, query the preset mapping rules to obtain the candidate environmental adjustment parameters corresponding to the driver's current state.

[0151] Preset mapping rules refer to a general rule table that maps the driver's current state to candidate environmental adjustment parameters. For example, "anxious state → blue ambient lighting, soothing music, air conditioning temperature drop of 1°C". This mapping is based on ergonomics and general preferences and serves as an initial reference for personalized adjustments.

[0152] In one embodiment, based on the driver's current state determined in the above steps, a preset state-environment parameter mapping table is queried (e.g., anxiety → blue ambient light + soothing music + air conditioning temperature drop of 1°C) to obtain the corresponding candidate environmental adjustment parameters.

[0153] For example, suppose the current state is "anxiety," and the corresponding solutions in the mapping table are: blue ambient light, playing soft music, and lowering the air conditioner temperature by 1°C. These parameters are then considered as candidate adjustment solutions.

[0154] Step 308: Obtain the driver's historical preference data, which includes at least the driver's feedback data on the results of environmental adjustment under the same historical conditions as the current state.

[0155] Historical preference data refers to a driver's past personalized responses to environmental adjustment results, which can include: explicit feedback, such as actively confirming, rejecting, or adjusting automatic adjustment schemes via voice or screen; and implicit feedback, such as passively collected data on manual adjustments to environmental parameters, changes in unconscious gesture frequency, and restoring environmental parameters to their pre-adjustment state.

[0156] In one embodiment, the vehicle identifier / driver identification identifier is bound, and the driver's first state (e.g., irritability, fatigue) obtained from the current identification is retrieved, using this state as the search keyword. All historical entries from the memory showing the driver in the same physical and mental state are filtered out, and data with inconsistent states is removed. Historical preferences are extracted from the filtered records, such as adjustments to air conditioning temperature, air vent settings, fan speed, windows, and seat ventilation under the same state, as well as records of manual modifications. This constitutes the driver's historical preference data. The operational habits of drivers in similar states are summarized and organized to form a structured preference dataset for subsequent automatic adjustments to the in-vehicle environment.

[0157] For example, when driver Wang was in an anxious state, he manually changed the system's automatic blue headlights to purple and switched the music from piano to jazz. The system records these actions as implicit feedback and historical preference data, linking them to the driver's "anxious" state.

[0158] This step learns personal preferences silently from daily behavior without requiring drivers to actively label them, providing rich training samples for personalized learning models and enabling subsequent adjustments to gradually align with user habits.

[0159] Step 309: Adjust the candidate environmental adjustment parameters based on historical preference data, and generate and execute control commands for adjusting the in-vehicle environmental parameters.

[0160] In one embodiment, the driver's current state, unconscious gestures, and driving context information are input into a personalized learning model pre-trained based on historical preference data to obtain the driver's corresponding preference adjustment parameters; the preference adjustment parameters are then used to correct or replace the candidate environment adjustment parameters.

[0161] Personalized learning models can refer to models pre-trained based on collaborative filtering or reinforcement learning. They take the current driver state, unconscious gestures, and driving context as input and output the driver's preference adjustment parameters (such as preferred light colors, music types, etc.) under the same or similar states.

[0162] As one possible implementation, the aforementioned personalized learning model adopts a user-scenario two-dimensional lookup table matching structure, using the driver's current state, unconscious recognition gestures, and driving context information as feature indexes, linking them to historical cockpit adjustment records, and combining them with weighted calculations based on the habits of users in the same feature group to directly output personalized preference adjustment parameters.

[0163] Specifically, model inference and training can be conducted in the following ways: During the inference phase, the model accurately matches the current input features, associates them with the driver's personal historical cockpit adjustment behavior records, and combines them with the adjustment habits of group users under the same feature conditions for weighted fusion calculation. By integrating individual preferences and group commonalities, personalized environmental preference adjustment parameters adapted to the current driver are output. During the model training phase, driver feedback data on manual cockpit environment adjustments under various driving scenarios, different physical and mental states, and various unconscious gestures are collected over a long period. The raw data is cleaned, denoised, and structured. Subsequently, it is categorized and archived according to three dimensions: driving context, driver state, and unconscious gestures. The high-frequency adjustment parameters and preference patterns of drivers under each feature combination are statistically analyzed, completing the modeling and solidification of the feature-parameter mapping relationship. The trained mapping model data is then stored in a database, enabling offline modeling and rapid table lookup inference, supporting subsequent real-time personalized parameter output.

[0164] As another possible implementation, the aforementioned personalized learning model can rely on the intelligent agent module to receive three types of inputs: the driver's current state, unconscious recognition gestures, and driving context information, and output preference adjustment parameters such as air conditioning and lighting; the reward module collects the driver's subsequent manual changes to generate reward and punishment signals.

[0165] Specifically, the model can be trained in the following way: It employs an online iterative training mechanism using real vehicles, continuously optimizing model parameters based on real-world driver feedback. When the system automatically adjusts cabin parameters, if the driver does not manually modify them, it indicates the current adjustment aligns with user preferences, and the system awards a positive reward to the model. If the driver manually adjusts parameters such as temperature, airflow, lights, and seat position, it indicates the model's adjustment does not meet the user's personalized needs, and the system awards a negative reward to the model. The model iteratively updates network weights based on real-time reward and punishment signals, continuously optimizing parameter output strategies under different driving scenarios, situations, states, and gestures, until the overall model performance converges and the adjustment effect stabilizes. At this point, the iteration terminates and model training is complete, achieving a personalized learning effect that becomes increasingly adapted to user habits with repeated use.

[0166] Candidate environmental adjustment parameters can refer to the basic adjustment scheme obtained from preset mapping rules based on the current state.

[0167] Preference adjustment parameters can refer to the adjustment parameters output by the personalized learning model that conform to the driver's personal habits.

[0168] Control commands can refer to the final generated digital commands used to change the operating parameters of in-vehicle environmental equipment, which can be sent to actuators such as ambient lighting, audio systems, air conditioning, seats, and fragrance systems.

[0169] Optionally, based on the identified driver's current state, the system's preset state-environment mapping rules are retrieved to obtain basic adjustment values ​​for devices such as air conditioning, seats, and ambient lighting, serving as candidate environmental adjustment parameters. The driver's current state, identified unconscious gestures, and driving context information are then fed into a personalized learning model pre-trained with the user's historical preferences. This model, relying on collaborative filtering / reinforcement learning algorithms, outputs preference adjustment parameters tailored to the car owner's driving habits. The personalized preference parameters output by the model are used to modify or completely replace the original candidate parameters, resulting in final preference parameters that match the user's personal preferences. These preference parameters are then converted into electronic control commands recognizable by the vehicle's hardware and issued to actuators such as air conditioning, seat ventilation, ambient lighting, car audio, and fragrance systems, enabling the entire vehicle to adjust the in-vehicle environment.

[0170] For example, based on the current state of "anxiety," the candidate solutions are blue ambient lighting and soothing music. The personalized learning model, based on Wang's historical preferences (previously changing blue to purple and piano music to jazz), outputs preference parameters as purple lighting and jazz music; by replacing blue with purple and piano music with jazz music, the final control command is generated.

[0171] Personalized learning enables a customized experience for each user, allowing the system to better understand user preferences over time, reducing manual intervention and improving naturalness and satisfaction. It also combines explicit and implicit feedback to automatically optimize without additional operations.

[0172] pass Figure 3 The description of the illustrated embodiment significantly improves the intelligence and user experience of in-vehicle environment adjustment through multimodal fusion, model-rule dual-path decision-making, and personalized learning mechanisms. First, by deeply integrating unconscious gestures, physiological signals, and driving context, the vulnerability of single-modality systems to interference is overcome. Combined with a state classification model and confidence-based decision-making using an expert rule base, the accuracy and robustness of state determination are ensured, while avoiding erroneous adjustments due to recognition fluctuations. Second, a personalized learning model based on historical explicit and implicit feedback automatically learns each driver's unique preferences from daily behavior and uses preference parameters to refine general candidate solutions, achieving adaptive adjustment that is "personalized" and "becomes increasingly user-friendly," greatly reducing manual intervention. The entire process employs a lightweight network design and preset rule cold start support, ensuring real-time response and system reliability while truly creating a natural, considerate, and immersive intelligent cockpit experience.

[0173] Figure 4 This application provides a structural block diagram of an in-vehicle environment adjustment device, the device comprising: The data acquisition module 41 is used to collect the driver's hand movement data and physiological signal data through a wearable device worn by the driver; and to obtain the vehicle's driving situation information. The gesture recognition module 42 is used to extract features from the hand movement data and identify unconscious gestures, which are hand movements of the driver without active control intention. The state determination module 43 is used to fuse and analyze the identified unconscious gestures, physiological signal data, and driving context information to obtain the current state of the driver. The instruction generation module 44 is used to generate and execute control instructions for adjusting in-vehicle environmental parameters based on the driver's current state and the driver's historical preference data.

[0174] In one possible implementation, the gesture recognition module 42 is specifically used for: The hand movement data is divided into windows using a sliding window mechanism to obtain the intensity index corresponding to the hand movement data within each time window; If the intensity index corresponding to the hand movement data within the time window exceeds the preset index threshold and the hand movement data does not belong to the preset normal driving operation, the time window is determined as the motion detection window. Extract the feature vector of each motion detection window and input the feature vector into a pre-trained unconscious gesture recognition model to obtain the gesture recognition result corresponding to the motion detection window; A temporal consistency check is performed on the gesture recognition results corresponding to multiple consecutive motion detection windows. If the number of times the same gesture appears in a preset number of consecutive motion detection windows reaches a preset threshold, the same gesture is determined to be an unconscious gesture.

[0175] In one possible implementation, the unconscious gesture recognition model in the gesture recognition module is trained in the following way: Obtain a training dataset containing hand movement data labeled with unconscious gesture categories; The training dataset is augmented to obtain an augmented training dataset; Using the augmented training dataset, the initial model is iteratively trained using a weighted cross-entropy loss function. During the training process, an adaptive learning optimizer is used to update the model parameters until the preset iteration stopping condition is met, resulting in a well-trained unconscious gesture recognition model.

[0176] In one possible implementation, the state determination module includes: The feature vector extraction module is used to perform multimodal fusion of the unconscious gestures, the physiological signal data, and the driving context information to obtain a fused feature vector; The first driver state determination unit is used to input the fused feature vector into a pre-trained state classification model to obtain the first driver state; wherein, the state classification model refers to a pre-trained deep learning model, which is used to input the fused feature vector and output the probability of each preset state; the input layer of the state classification model receives the fused feature vector, multiple hidden layers complete feature mapping, random deactivation layers randomly shut down neurons, and the output layer outputs the probability of each preset state; the first driver state is the preset state with the highest probability. The second driver state determination unit is used to determine the driver state that matches the fused feature vector in the preset rule base as the second driver state. The driver current state determination unit is used to determine the current state of the driver based on the first driver state and the second driver state.

[0177] In one possible implementation, the driver's current state determination unit is specifically used for: Obtain the model confidence level corresponding to the first driver state; If the model confidence level is greater than or equal to a preset confidence threshold, the first driver state is determined as the driver's current state. If the model confidence level is less than a preset confidence threshold, the second driver state is determined as the driver's current state.

[0178] In one possible implementation, the instruction generation module includes: An environmental adjustment parameter determination unit is used to query a preset mapping rule based on the driver's current state to obtain candidate environmental adjustment parameters corresponding to the driver's current state. A historical preference data acquisition unit is used to acquire the driver's historical preference data, which includes at least the driver's feedback data on environmental adjustment results in the same historical state as the current state. The control command generation unit is used to adjust the candidate environmental adjustment parameters based on the historical preference data, and generate and execute control commands for adjusting the in-vehicle environmental parameters.

[0179] In one possible implementation, the control instruction generation unit is specifically used for: The driver's current state, unconscious gestures, and driving context information are input into a personalized learning model pre-trained based on the historical preference data to obtain the driver's preference adjustment parameters under the same or similar states; wherein, the personalized learning model refers to a model pre-trained based on collaborative filtering or reinforcement learning, used to input the driver's current state, unconscious gestures, and driving context information, and output the driver's preference adjustment parameters. The candidate environment adjustment parameters are modified or replaced using the preference adjustment parameters.

[0180] like Figure 5 As shown, Figure 5 A schematic diagram of an electronic device provided in this application embodiment includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the in-vehicle environment adjustment method provided in any of the foregoing method embodiments, including: The driver's hand movement data and physiological signal data are collected through wearable devices worn by the driver; and the driving situation information of the vehicle is obtained. Feature extraction is performed on the hand movement data to identify unconscious gestures, which are hand movements made by the driver without active control intention. The driver's current state is obtained by fusing and analyzing the identified unconscious gestures, physiological signal data, and driving context information. Based on the driver's current state and the driver's historical preference data, control commands for adjusting in-vehicle environmental parameters are generated and executed.

[0181] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the in-vehicle environment adjustment method provided in any of the foregoing method embodiments.

[0182] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0184] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also mean including the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0185] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for regulating the in-vehicle environment, characterized in that, The method includes: The driver's hand movement data and physiological signal data are collected through wearable devices worn by the driver; and the driving situation information of the vehicle is obtained. Feature extraction is performed on the hand movement data to identify unconscious gestures, which are hand movements made by the driver without active control intention. The driver's current state is obtained by fusing and analyzing the identified unconscious gestures, physiological signal data, and driving context information. Based on the driver's current state and the driver's historical preference data, control commands for adjusting in-vehicle environmental parameters are generated and executed. The identified unconscious gestures, physiological signal data, and driving context information are fused and analyzed to obtain the driver's current state, including: The unconscious gestures, physiological signal data, and driving context information are fused using multimodal methods to obtain a fused feature vector; The fused feature vector is input into a pre-trained state classification model to obtain a first driver state; wherein, the state classification model refers to a pre-trained deep learning model, which is used to input the fused feature vector and output the probability of each preset state; the input layer of the state classification model receives the fused feature vector, multiple hidden layers complete feature mapping, random deactivation layers randomly shut down neurons, and the output layer outputs the probability of each preset state; the first driver state is the preset state with the highest probability. The driver state that matches the fused feature vector in the preset rule base is determined as the second driver state; The current state of the driver is determined based on the first driver state and the second driver state; Determining the current state of the driver based on the first driver state and the second driver state includes: Obtain the model confidence level corresponding to the first driver state; If the model confidence level is greater than or equal to a preset confidence threshold, the first driver state is determined as the driver's current state. If the model confidence level is less than a preset confidence threshold, the second driver state is determined as the driver's current state.

2. The method according to claim 1, characterized in that, Feature extraction is performed on the hand movement data to identify unconscious gestures, including: The hand movement data is divided into windows using a sliding window mechanism to obtain the intensity index corresponding to the hand movement data within each time window; If the intensity index corresponding to the hand movement data within the time window exceeds the preset index threshold and the hand movement data does not belong to the preset normal driving operation, the time window is determined as the motion detection window. Extract the feature vector of each motion detection window and input the feature vector into a pre-trained unconscious gesture recognition model to obtain the gesture recognition result corresponding to the motion detection window; A temporal consistency check is performed on the gesture recognition results corresponding to multiple consecutive motion detection windows. If the number of times the same gesture appears in a preset number of consecutive motion detection windows reaches a preset threshold, the same gesture is determined to be an unconscious gesture.

3. The method according to claim 2, characterized in that, The unconscious gesture recognition model was trained in the following way: Obtain a training dataset containing hand movement data labeled with unconscious gesture categories; The training dataset is augmented to obtain an augmented training dataset; Using the augmented training dataset, the initial model is iteratively trained using a weighted cross-entropy loss function. During the training process, an adaptive learning optimizer is used to update the model parameters until the preset iteration stopping condition is met, resulting in a well-trained unconscious gesture recognition model.

4. The method according to claim 1, characterized in that, Based on the driver's current state and historical preference data, control commands for adjusting in-vehicle environmental parameters are generated and executed, including: Based on the driver's current state, a preset mapping rule is queried to obtain candidate environmental adjustment parameters corresponding to the driver's current state; Obtain the driver's historical preference data, which includes at least the driver's feedback data on environmental adjustment results under the same historical state as the current state; Based on the historical preference data, the candidate environmental adjustment parameters are adjusted, and control commands for adjusting the in-vehicle environmental parameters are generated and executed.

5. The method according to claim 4, characterized in that, The adjustment of the candidate environment adjustment parameters based on the historical preference data includes: The driver's current state, unconscious gestures, and driving context information are input into a personalized learning model pre-trained based on the historical preference data to obtain the driver's preference adjustment parameters; wherein, the personalized learning model refers to a model pre-trained based on collaborative filtering or reinforcement learning, used to input the driver's current state, unconscious gestures, and driving context information, and output the driver's preference adjustment parameters. The candidate environment adjustment parameters are modified or replaced using the preference adjustment parameters.

6. A vehicle interior environment control device, characterized in that, The device is used to implement the in-vehicle environment adjustment method according to any one of claims 1-5.

7. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute an in-vehicle environment adjustment program stored in the memory to implement the in-vehicle environment adjustment method according to any one of claims 1-5.

8. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the in-vehicle environment adjustment method according to any one of claims 1-5.

Images