Vehicle light control method, device, equipment and medium

Abstract

The invention discloses a vehicle light control method. The method comprises the steps of obtaining driving behavior multi-modal data; based on the driving behavior multi-modal data, state recognition is carried out through a pre-trained deep learning model, and a state evaluation result of the driver is obtained; generating a light control strategy according to the state evaluation result; and controlling at least one optical parameter of the in-vehicle lighting system through the lighting control strategy. According to the method, the driving behavior multi-modal data are fused and processed, the pre-trained deep learning model is utilized to perform comprehensive analysis, the state of the driver is judged, the physiological fatigue state and the psychological emotion state of the driver are evaluated, and the light control strategy is generated based on the evaluation result. According to the method, passive ambient lighting can be converted into active intervention, intervention is carried out in time through scientifically verified optical parameter adjustment when a driver is in the early stage of fatigue or mood tension, fatigue is effectively relieved, the mood is pacified, and therefore the driving safety is improved.

Description

Technical Field

[0001] This application relates to the field of lighting control technology, specifically to a vehicle lighting control method, device, equipment, and medium. Background Technology

[0002] Currently, with the increasing intelligence of automobiles, the comfort and safety of the in-vehicle environment have gradually become research hotspots. In-vehicle lighting, as a crucial component of the driving environment, directly impacts the visual experience and emotional state of drivers and passengers due to its optical characteristics. However, most current mainstream in-vehicle lighting adjustment solutions rely on preset fixed modes or manual operation by the user, essentially exhibiting static or passive responses and lacking the intelligence to autonomously adjust based on the driver's real-time physical and mental state and dynamic in-vehicle environmental parameters.

[0003] For example, when a driver is fatigued from prolonged driving, the brightness and color of the vehicle's interior lights may not provide effective comfort and safety support. The root cause is that the vehicle's lighting control module and driver status monitoring system are typically independent, creating a technical barrier between "perception" and "execution," leading to data flow disruptions and isolated decision-making. This lack of a linkage mechanism makes it difficult for the system to achieve closed-loop optimization from environmental information to personalized lighting environment output, severely hindering further improvements in the intelligence level of the driving experience. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, this application provides a vehicle lighting control method, device, equipment and medium to solve the above technical problems.

[0005] This application provides a vehicle lighting control method, the processing method comprising: Acquire multimodal data on driving behavior; Based on the aforementioned multimodal driving behavior data, a pre-trained deep learning model is used to identify the driver's state and obtain the driver's state assessment results. Based on the state assessment results, a lighting control strategy is generated; The lighting control strategy controls at least one optical parameter of the vehicle's interior lighting system.

[0006] In one embodiment of this application, the state assessment result includes a driver fatigue index, and the method for generating the driver fatigue index includes: Multiple first features characterizing the driver's fatigue state are extracted from the multimodal driving behavior data; The multiple first features are combined to construct a multimodal feature matrix; The multimodal feature matrix is ​​calculated based on an attention mechanism to generate dynamic weight coefficients representing each modality of data. Based on the dynamic weighting coefficients, multiple features in the multimodal feature matrix are weighted to obtain weighted fusion features; By mapping the weighted fusion features using a preset activation function, a driving fatigue index that characterizes whether the driver is in a state of fatigue is obtained.

[0007] In one embodiment of this application, the step of calculating the multimodal feature matrix based on an attention mechanism to generate dynamic weight coefficients representing each modality of data includes: Based on the multimodal feature matrix, calculate the query matrix, key matrix, and value matrix; Calculate the attention score matrix based on the query matrix and the key matrix; The attention score matrix is ​​normalized to generate an attention weight matrix; Multiplying the attention weight matrix by the value matrix yields the attention output matrix; The dynamic weight coefficients are determined based on the attention output matrix.

[0008] In one embodiment of this application, the state assessment result includes an emotion fluctuation index, and the method for generating the emotion fluctuation index includes: Extract multiple second features representing the driver's emotional state from the multimodal driving behavior data; The multiple second features are fused to obtain a comprehensive feature vector; The comprehensive feature vector is mapped by a preset classifier to obtain an emotion fluctuation index that characterizes whether the driver is in an emotional fluctuation state.

[0009] In one embodiment of this application, the at least one optical parameter includes: brightness and color; generating a lighting control strategy based on the state evaluation result includes: In response to the driver fatigue index exceeding a preset driver fatigue threshold, a first control strategy in the lighting control strategy is executed. The first control strategy includes: reducing the brightness of the in-vehicle lighting area based on the driver fatigue index and adjusting the hue of the in-vehicle lights to a preset warm hue range based on the environmental comfort assessment value. In response to the mood fluctuation index exceeding a preset mood fluctuation threshold, the second control strategy in the lighting control strategy is executed. The second control strategy includes: reducing the brightness of the interior lighting area based on the driver fatigue index, and adjusting the interior lights to a soft, warm light with a gradient effect based on the mood fluctuation index.

[0010] In one embodiment of this application, the method for determining the environmental comfort assessment value includes: Acquire at least one type of in-vehicle environmental data; Assign preset weighting coefficients to each type of in-vehicle environment data; According to the preset weighting coefficient, the corresponding at least one in-vehicle environment data is weighted and then combined to generate an environmental comfort assessment value.

[0011] In one embodiment of this application, the vehicle lighting control method further includes: iteratively optimizing the lighting control strategy using a reinforcement learning algorithm; the iterative optimization includes: The current state is obtained from the state space based on the driver fatigue index, emotion fluctuation index, and environmental comfort assessment value; wherein, the state in the state space is defined by the driver fatigue index, emotion fluctuation index, and environmental comfort assessment value. Based on the current state, a lighting control strategy corresponding to the current state is determined from the action space; wherein, the lighting control strategy in the action space is defined by the brightness and color values ​​of the in-vehicle lights; Based on the driver's feedback data collected after the execution of the lighting control strategy, a reward signal corresponding to the lighting control strategy is generated; Based on the reward signal, the state transition results observed after executing the lighting control strategy, the preset learning rate and discount factor, the value evaluation function of the lighting adjustment strategy is updated to optimize the decision mapping relationship from state to action; wherein, the discount factor is used to balance the weight of immediate reward and future reward in value evaluation, and the learning rate is used to adjust the degree of influence of this update on historical value estimation.

[0012] This application provides a vehicle lighting control device, wherein the vehicle lighting control includes: The data acquisition module is used to acquire multimodal data on driving behavior; The state recognition module is used to perform state recognition based on the multimodal driving behavior data and a pre-trained deep learning model to obtain the driver's state assessment result. The strategy generation module is used to generate a lighting control strategy based on the state evaluation results. The parameter control module is used to control at least one optical parameter of the in-vehicle lighting system through the lighting control strategy.

[0013] This application provides a vehicle lighting control device, the vehicle lighting control device comprising: One or more processors; A memory for storing one or more programs that, when executed by one or more processors, cause the vehicle lighting control device to implement the steps of the vehicle lighting control method described above.

[0014] This application provides a computer-readable storage medium storing a computer program that, when executed by a computer's processor, causes the computer to perform the steps of the aforementioned vehicle lighting control method.

[0015] The beneficial effects of this application are as follows: A vehicle lighting control method of this application includes: acquiring multimodal driving behavior data; based on the multimodal driving behavior data, performing state recognition through a pre-trained deep learning model to obtain a driver state assessment result; generating a lighting control strategy based on the state assessment result; and controlling at least one optical parameter of the in-vehicle lighting system through the lighting control strategy. This application, by fusing and processing multimodal driving behavior data and using a pre-trained deep learning model for comprehensive analysis, judges the driver's state, thereby providing accurate and real-time assessment of the driver's physiological fatigue and psychological emotional state. The lighting control strategy generated based on the assessment result enables the lighting system to shift from passive environmental lighting to active intervention. When the driver experiences initial fatigue or emotional tension, timely intervention is provided through scientifically validated optical parameter adjustments (such as reducing brightness or adjusting to a warmer color tone), effectively alleviating fatigue, calming emotions, and thus improving driving safety.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This is a schematic flowchart illustrating an exemplary vehicle lighting control method of this application; Figure 2 This is a schematic diagram illustrating the workflow of an exemplary AI algorithm analysis module of this application; Figure 3 This is a schematic diagram of an exemplary lighting control architecture of this application; Figure 4 This is a schematic flowchart illustrating an exemplary vehicle lighting control method of this application; Figure 5 This is a flowchart illustrating an exemplary method for generating a driver fatigue index according to this application. Figure 6 This is an exemplary flowchart of the process for generating dynamic weighting coefficients representing each modality data in this application; Figure 7 This is a flowchart illustrating an exemplary method for generating an emotion fluctuation index according to this application. Figure 8 This is a flowchart illustrating an exemplary method for determining environmental comfort assessment values ​​according to this application. Figure 9 This is an exemplary flowchart of the process of iteratively optimizing a lighting control strategy using a reinforcement learning algorithm, as described in this application. Figure 10 This is a block diagram of an exemplary vehicle lighting control device according to this application; Figure 11 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0018] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0021] Figure 1 This is a schematic diagram illustrating an exemplary vehicle lighting control method implementation environment according to this application. Please refer to... Figure 1The implementation environment includes: a driver status acquisition module, an in-vehicle environment acquisition module, a data processing module, an AI (Artificial Intelligence) algorithm analysis module, and an intelligent lighting adjustment module. The driver status acquisition module includes an eye-tracking sensor, a heart rate sensor, a facial expression recognition camera, a steering wheel vibration sensor, a lane departure sensor, and a brake pedal sensor. The in-vehicle environment acquisition module includes a light sensor, a temperature sensor, a humidity sensor, and an air quality sensor. The eye-tracking sensor collects eye data, including blink frequency and eye closure degree. The heart rate sensor collects heart rate data. The facial expression recognition camera collects facial expressions, including eye movements and changes in the corners of the mouth. The steering wheel vibration sensor collects steering wheel data. The lane departure sensor collects lane departure data. The brake pedal sensor collects brake pedal data. The light sensor collects light intensity data. The temperature sensor collects in-vehicle temperature data. The humidity sensor collects in-vehicle humidity data. The air quality sensor collects air quality data. After data acquisition via sensors, the data processing module extracts features using CNN (Convolutional Neural Network) / LSTM (Long Short-Term Memory) algorithms to obtain physiological, behavioral, and environmental characteristics. The AI ​​algorithm analysis module then assesses the driver's state based on these physiological and behavioral characteristics, determining whether the driver is fatigued or experiencing emotional fluctuations. Based on this assessment, it decides whether to generate a headlight adjustment command in conjunction with environmental characteristics. The intelligent headlight adjustment module then adjusts the headlights according to the corresponding command. When the driver is fatigued, the system generates a headlight adjustment command based on the driver fatigue index and environmental characteristics to adjust the optical parameters of the headlight system; when the driver is experiencing emotional fluctuations, the system generates a headlight adjustment command based on the emotional fluctuation index to adjust the optical parameters of the headlight system.

[0022] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating the workflow of an exemplary AI algorithm analysis module of this application. Based on physiological and behavioral characteristics, the AI ​​algorithm analysis module maps the fused features of these characteristics using a preset activation function (Sigmoid function) to generate a driving fatigue index, representing whether the driver is fatigued. It then maps the comprehensive feature vector using a preset classifier (Sigmoid function) to obtain an emotion fluctuation index, representing whether the driver is experiencing emotional fluctuations. Simultaneously, the AI ​​analysis algorithm module generates an environmental comfort assessment value based on environmental characteristics. Finally, it controls the lighting based on the emotion fluctuation index, driving fatigue index, and environmental comfort assessment value.

[0023] Please see Figure 3 , Figure 3 This is a schematic diagram of an exemplary lighting control architecture according to this application. Figure 2 In this system, temperature data from the temperature sensor, humidity data from the humidity sensor, and air quality data from the air quality sensor are all input to the air conditioning controller, which then transmits the data to the central computing platform. Light intensity data from the light sensor, heart rate data from the heart rate sensor, and steering wheel data from the steering wheel vibration sensor are all transmitted to the body controller, which then transmits the data to the central computing platform. Data from the lane departure sensor is transmitted to the central computing platform via the intelligent driving controller, while eye images from the eye-tracking camera and facial data from the facial expression recognition camera are transmitted to the central computing platform via the cockpit controller. The AI ​​algorithm analysis module in the central computing platform assesses the driver's state based on this data and then generates a lighting control strategy based on the driver's state (fatigue, emotional fluctuation) and / or environmental comfort assessment values. This strategy is then sent to the lighting controller, which controls the left front ambient light, right front ambient light, left rear ambient light, and right rear ambient light based on the lighting control strategy. After the lighting adjustment is complete, the intelligent lighting adjustment module provides feedback on the lighting status to the AI ​​algorithm analysis module, allowing the AI ​​algorithm analysis module to update the lighting control strategy.

[0024] The embodiments of this application respectively propose a vehicle lighting control method, a vehicle lighting control device, a vehicle lighting control equipment, and a computer-readable storage medium, which will be described in detail below.

[0025] Please see Figure 4 , Figure 4 This is a schematic flowchart illustrating an exemplary vehicle lighting control method according to this application. The vehicle lighting control method includes at least steps S410 to S440, which are described in detail below: Step S410: Obtain multimodal data of driving behavior; Among them, driving behavior multimodal data is heterogeneous data reflecting the driver's state from different physiological and behavioral dimensions. Specifically, it includes driver physiological data (eye images, facial expressions, heart rate signals), driving behavior data (steering wheel operation, lane departure signals, brake pedal operation signals), and in-vehicle environmental data (light intensity, temperature, humidity, air quality). This data is collected in real time by on-board sensors to form multimodal time-series data, which serves as input data for subsequent deep learning models. It is directly bound to the vehicle lighting control scenario to ensure that the input source is clear and the data format is reproducible, providing a reliable data foundation for model training and inference.

[0026] Step S420: Based on multimodal driving behavior data, state recognition is performed using a pre-trained deep learning model to obtain the driver's state assessment result; The AI ​​algorithm analysis module used in this step is a deep learning-based multimodal fusion model, specifically including an eye feature extraction network (2D-CNN), a heart rate feature extraction network (LSTM), a steering wheel behavior feature extraction network (1D-CNN), a multimodal attention fusion module, and a state output layer. These modules are connected sequentially to form an end-to-end driver state assessment structure. The specific structure, hierarchical connections, functions, and training steps of each network are as follows: (1) Eye Feature Extraction Network Eye images are captured by an in-vehicle camera and input into a 2D convolutional neural network (2D-CNN), which outputs standardized eye data. The 2D convolutional neural network consists of the following layers in sequence: input layer, convolutional layer, activation layer, pooling layer, convolutional layer, activation layer, pooling layer, fully connected layer, and feature output layer. The input layer sequentially performs size normalization, grayscale conversion, pixel normalization, and denoising on the eye image, ultimately outputting a standard tensor format. Size normalization scales the eye image to a fixed size, grayscale conversion and normalization normalize pixel values ​​to the [0,1] range, and Gaussian filtering is used to remove image noise.

[0027] As a specific implementation, the eye image is normalized to 128×128 or 64×64 data. Of course, those skilled in the art can adjust it according to actual needs. The final output of the eye feature extraction network is a standard tensor with dimensions [128, 128, 1].

[0028] The convolutional layer is used to extract eye features. Specifically, it uses a 3×3 convolutional kernel (stride of 1 and padding of 1) to extract hierarchical features through a local receptive field and weight sharing mechanism.

[0029] In one embodiment, the convolutional layer includes: Shallow convolution is used to extract low-level visual features such as edges, eyelid contours, brightness variations, and pupil areas. Mid-level convolutions are used to extract intermediate features such as blinking motion, eyelid opening and closing amplitude, and eye movement trends. Deep convolution is used to extract fatigue-related semantic features, such as prolonged eye closure, frequent slight eye closure, and staring blankly, which are directly related to driver fatigue. As a specific implementation, the shallow convolution uses 16 convolution kernels and outputs a feature map of [128, 128, 16]; the middle convolution uses 32 convolution kernels and outputs a feature map of [128, 128, 32]; and the deep convolution uses 64 convolution kernels and outputs a feature map of [128, 128, 64]. The outputs of the shallow convolution, the middle convolution, and the deep convolution represent the feature responses of the input eye image in different dimensions.

[0030] The activation layer uses the ReLU activation function to perform a nonlinear mapping on the output of each convolutional layer. The function expression is f(x)=max(0,x). Its purpose is to introduce a nonlinear transformation to improve the model's ability to fit complex eye states (such as different degrees of eyelid closure and blinking frequency) and avoid insufficient recognition accuracy caused by model linearization.

[0031] The pooling layer uses max pooling with a 2×2 kernel and a stride of 2, and no padding. Its function is to compress features, reduce dimensionality, and resist interference. By retaining the maximum pixel value in the feature map, redundant information is removed, the number of model parameters is reduced, the robustness of the model is improved, and feature misidentification caused by slight eye image jitter (such as a small head movement of the driver) is avoided, making it suitable for the complex environment of in-vehicle scenarios.

[0032] The fully connected layer includes a first fully connected layer and a second fully connected layer. The input of the first fully connected layer is a one-dimensional feature vector flattened from the pooled feature map (in this embodiment, it is [32×32×64]=65536 dimensions), and the output is a 1024-dimensional feature vector. The input of the second fully connected layer is a 1024-dimensional feature vector, and the output is a 256-dimensional feature vector.

[0033] In one embodiment, the input of the first fully connected layer is a one-dimensional feature vector of [32×32×64] dimensions, and the output is a 1024-dimensional feature vector; the input of the second fully connected layer is a 1024-dimensional feature vector, and the output is a 256-dimensional feature vector.

[0034] The feature output layer outputs an eye fatigue feature vector; this vector is directly used in the subsequent multimodal fusion module. Each dimension of the vector corresponds to features related to eye fatigue, such as blink frequency features and eyelid closure degree features.

[0035] In one embodiment, the eye fatigue feature vector is a 256-dimensional eye fatigue feature vector.

[0036] (2) Heart rate feature extraction network The heart rate feature extraction network consists of the following layers connected in sequence: input layer, long short-term memory layer, dropout layer, fully connected layer, and feature output layer. The input layer is used to receive heart rate time series data and normalize it into an input sequence with a predetermined format; wherein the sampling frequency of the heart rate time series is 1Hz and the duration is 10s, that is, the input dimension is [10,1].

[0037] The long short-term memory layer is used to capture the temporal characteristics of heart rate change trends, heart rate variability, and abnormal fluctuations; through these characteristics, it can effectively memorize the long-term change patterns of heart rate and adapt to the dynamic changes of heart rate during long-term driving (such as a slower heart rate when fatigued and a faster heart rate when emotionally stressed). Random deactivation layers (dropout) are used to randomly discard some neuron outputs to prevent overfitting. Fully connected layers are used to perform dimensionality mapping and fusion of features output from long short-term memory layers; The feature output layer outputs a heart rate stability feature vector, which characterizes the driver's state of tension, fatigue, or anxiety. For example, a decrease in heart rate variability corresponds to driver fatigue, while a sudden increase in heart rate corresponds to driver tension, thus clarifying the correlation between the output features and the driver's state.

[0038] In one embodiment, the long short-term memory layer is a single long short-term memory network with 64 hidden units and uses the tanh activation function; the dropout probability of the random deactivation layer is set to 0.2; the fully connected layer takes a 64-dimensional feature vector as input and outputs a 128-dimensional feature vector; the feature output layer outputs a 128-dimensional heart rate stability feature vector.

[0039] (3) Steering wheel behavior feature extraction network The steering wheel behavior feature extraction network specifically includes the following layers connected in sequence: input layer, first one-dimensional convolutional layer, first activation layer, first pooling layer, second one-dimensional convolutional layer, second activation layer, second pooling layer, fully connected layer, and feature output layer; The input layer is used to receive steering wheel operation timing data and standardize it into an input sequence of a predetermined format; the steering wheel operation timing data consists of signals such as vibration amplitude and operation frequency collected by the steering wheel vibration sensor, which are directly bound to the vehicle driving scenario. The first one-dimensional convolutional layer is used to extract high-frequency vibration features of steering wheel operation; the second one-dimensional convolutional layer is used to extract low-frequency stationary features of steering wheel operation. The first and second activation layers use the ReLU function to introduce nonlinear transformations and improve the model's ability to fit complex steering wheel operation features. The first and second pooling layers use average pooling for feature dimensionality reduction and noise suppression; Fully connected layers are used to perform dimensionality mapping and fusion on the pooled features; The feature output layer is used to output the driving operation stability feature vector, which is used to characterize the driver's operating state and is directly related to the driver's fatigue and emotional state.

[0040] In one embodiment, the first one-dimensional convolutional layer uses a 1×3 convolutional kernel, with 16 kernels and a stride of 1; the second one-dimensional convolutional layer uses a 1×5 convolutional kernel, with 32 kernels and a stride of 1; the first pooling layer and the second pooling layer use average pooling with a 1×2 kernel and a stride of 2; the fully connected layer takes a one-dimensional vector of flattened pooled features as input and outputs a 64-dimensional feature vector; the feature output layer outputs a 64-dimensional driving operation stability feature vector.

[0041] (5) Deep learning model training process The training steps and parameter settings of the deep learning model (including 2D-CNN, LSTM, 1D-CNN and attention fusion module) in this application are as follows. Those skilled in the art can reproduce the training process based on the following description: ① Dataset Construction: A multimodal labeled dataset was constructed, with data sourced from real-vehicle data collection (data collected from 100 drivers of different ages and driving habits in different driving scenarios (highway, urban roads, and nighttime)). The total number of samples was 10,000. Each sample included input data (eye images, heart rate time-series signals, and steering wheel operation time-series signals) and corresponding labels (fatigue / non-fatigue, emotional stability / emotional fluctuation). Labeling rules: Labels were created by professionals combining the driver's physiological state (e.g., blinking frequency ≥15 times / minute is considered fatigue tendency) and behavioral state (e.g., steering wheel vibration amplitude ≥0.5mm is considered fatigue) to ensure label accuracy. ② Dataset partitioning: The dataset is divided into a training set (7000 sets), a validation set (2000 sets), and a test set (1000 sets) in a ratio of 7:2:1 for model training, parameter tuning, and performance verification. ③ Model Initialization: Initialize the weights and biases of the CNN, LSTM, and attention modules. The initial weight values ​​adopt a random normal distribution (mean 0, variance 0.01), and the initial bias value is set to 0. Set the training hyperparameters: learning rate η = 0.001, batch size = 32, maximum number of iterations epochs = 50, and the optimizer uses the Adam optimizer. =0.9, =0.999, =1e-8), the loss function adopts the cross-entropy loss function (used for binary classification recognition, i.e. fatigue / non-fatigue, emotional stability / emotional fluctuation). ④ Forward propagation: Input the multimodal training data into the model, and pass it through each feature extraction network and attention fusion module in sequence to output the driving fatigue index and the mood fluctuation index (both are values ​​between 0 and 1). ⑤ Loss Calculation and Backpropagation: The error between the predicted value (fatigue / mood index) and the label is calculated using the cross-entropy loss function. The Adam optimizer is then used to update the weights and biases of each layer through backpropagation to minimize the loss function. ⑥ Model Validation and Tuning: After each training epoch, validate the model performance using the validation set and calculate the recognition accuracy. When the validation set accuracy no longer improves after 5 consecutive epochs, stop training using the early stopping strategy to avoid model overfitting. At the same time, adjust hyperparameters (such as learning rate decay, halving the learning rate every 10 epochs) to optimize model performance. ⑦ Model Testing and Saving: After training, test the model performance using a test set to ensure that the fatigue recognition accuracy is ≥92% and the emotion fluctuation recognition accuracy is ≥90%, meeting the real-time recognition requirements of in-vehicle scenarios; save the trained model structure and weights (in h5 format), which can be directly deployed on the vehicle terminal for real-time inference with an inference latency of ≤100ms, adapting to the real-time control requirements of in-vehicle systems.

[0042] Step S430: Generate a lighting adjustment strategy based on the status assessment results; Step S440: Control at least one optical parameter of the in-vehicle lighting system by generating a lighting adjustment strategy.

[0043] After acquiring multimodal driving behavior data, feature extraction and fusion are performed on the input multimodal driving behavior data to obtain fused features. These fused features are then processed by the aforementioned deep learning model to obtain the driver's state assessment result. This state assessment result determines whether the driver is experiencing driving fatigue or emotional fluctuations. Based on the real-time acquired state assessment result, the lighting control module calculates the target brightness and color temperature values, and drives the actuator to adjust the lights based on these values.

[0044] This application integrates multimodal driving behavior data and utilizes a pre-trained deep learning model for comprehensive analysis to assess the driver's state, thereby providing a precise and real-time evaluation of the driver's physiological fatigue and psychological emotional state. Based on this assessment, a lighting control strategy is generated, enabling the system to shift from passive ambient lighting to proactive intervention. When the driver experiences early signs of fatigue or emotional stress, timely intervention is achieved through scientifically validated optical parameter adjustments (such as reducing brightness or adjusting to a warmer color tone), effectively alleviating fatigue, calming emotions, and thus improving driving safety.

[0045] Please see Figure 5 , Figure 5 This is a schematic flowchart illustrating an exemplary method for generating a driver fatigue index according to this application. Figure 5 In the assessment, the condition evaluation results include a driver fatigue index, which is generated using methods including: Step S510: Extract multiple first features representing the driver's fatigue state from the multimodal driving behavior data; Multimodal driving behavior data includes physiological and behavioral data: physiological data includes eye data, heart rate data, and facial expression data; behavioral data includes steering wheel data, lane departure data, and brake pedal data. Eye data, such as blink frequency and eye closure degree, is collected via eye-tracking cameras to assess driver fatigue. Heart rate data is collected via contact or non-contact sensors on the steering wheel or seat to analyze the driver's emotional state. Facial expression data, including eye movements and mouth corner changes, is collected via facial expression recognition cameras to determine emotional state. Steering wheel data, collected via steering wheel vibration sensors, monitors driver operational stability. Lane departure data, collected via lane departure sensors, detects whether the vehicle deviates from its lane and assesses driver attention. Brake pedal data is collected via brake pedal sensors.

[0046] Among them, several first features include: eye features extracted from eye images of driving behavior multimodal data using a two-dimensional convolutional neural network (2D-CNN) to characterize blinking frequency and eyelid closure degree; heart rate features extracted from heart rate time-series signals of driving behavior data using a long short-term memory network (LSTM) to characterize heart rate variability and trend; and steering wheel behavior features extracted from steering wheel operation time-series signals of driving behavior data using a one-dimensional convolutional neural network (1D-CNN) to characterize the frequency and amplitude of micro-vibrations during operation.

[0047] Step S520: Combine multiple first features to construct a multimodal feature matrix; A multimodal feature matrix is ​​formed by combining eye features, heart rate features, and steering wheel behavior features.

[0048] Step S530: Calculate the multimodal feature matrix based on the attention mechanism to generate dynamic weight coefficients representing each modality of data; The dynamic weight coefficient is calculated through an attention mechanism and represents the weight value of the contribution to fatigue judgment at the current moment.

[0049] Step S540: Based on the dynamic weight coefficients, weight multiple features in the multimodal feature matrix to obtain weighted fusion features; Step S550: The weighted fusion features are mapped using a preset activation function to obtain a driving fatigue index that represents whether the driver is in a state of fatigue.

[0050] In the feature extraction stage, different neural networks (2D-CNN, LSTM, and 1D-CNN) process the multimodal driving behavior data, outputting corresponding features (eye features, heart rate features, and steering wheel behavior features). A multimodal feature matrix is ​​constructed by combining these features (eye features, heart rate features, and steering wheel behavior features - steering wheel vibration amplitude). After constructing the multimodal feature matrix, an attention mechanism is used to process the matrix and obtain the dynamic weight coefficients corresponding to each modality. In the weighted fusion stage, each modality feature is multiplied by its corresponding dynamic weight coefficient, and the results are summed to obtain a comprehensive feature vector that integrates multi-source information. Finally, this comprehensive feature vector is input into the Sigmoid function, which outputs a value between 0 and 1, representing the driver fatigue index. A higher value indicates a greater probability that the driver is fatigued. The driver fatigue index can be used to determine whether the driver is fatigued.

[0051] This application introduces an attention mechanism to achieve dynamic weight allocation, which significantly improves the accuracy and reliability of driver fatigue assessment.

[0052] Please see Figure 6 , Figure 6 This is an exemplary flowchart illustrating the process of generating dynamic weighting coefficients representing each modality data according to this application. Figure 6 In this process, the dynamic weighting coefficients representing each modality of data include: Step S610: Calculate the query matrix, key matrix, and value matrix based on the multimodal feature matrix; Step S620: Calculate the attention score matrix based on the query matrix and the key matrix; Step S630: Normalize the attention score matrix to generate the attention weight matrix; Step S640: Multiply the attention weight matrix by the value matrix to obtain the attention output matrix; Step S650: Determine the dynamic weight coefficients based on the attention output matrix.

[0053] In one embodiment, the multimodal attention mechanism fusion module constructs a multimodal feature matrix from eye features, heart rate features, and steering wheel operation features, and dynamically assigns weights through an attention mechanism, specifically including: The eye feature vector, heart rate feature vector, and driving behavior feature vector are concatenated to construct a multimodal feature matrix; The multimodal feature matrix is ​​multiplied by the three pre-trained weight matrices to generate the query matrix, key matrix, and value matrix. Calculate the attention score matrix based on the query matrix and the key matrix; The attention score matrix is ​​normalized to obtain the attention weight matrix; Multiply the attention weight matrix by the value matrix to obtain the attention output matrix; The attention output matrix is ​​combined with the preset static weights for each modality to generate dynamic weight coefficients for each modality. The eye fatigue feature vector, heart rate feature vector, and driving behavior feature vector are weighted and summed using dynamic weighting coefficients to obtain weighted fusion features.

[0054] Through the attention mechanism, the model can automatically focus on the modalities that are more important for fatigue / emotion recognition. For example, when the driver is extremely fatigued, the dynamic weight of eye features will be automatically increased, while the weight of steering wheel features will be appropriately reduced, significantly improving recognition accuracy and adapting to the dynamic changes in the driver's state in the in-vehicle scenario.

[0055] In one embodiment, the steps of inputting eye fatigue feature vectors, heart rate features, and driving behavior features into a multimodal attention fusion module and fusing them through an attention mechanism to generate fused features specifically include: The 256-dimensional eye fatigue feature vector, the 128-dimensional heart rate feature vector, and the 64-dimensional driving behavior feature vector are concatenated to obtain a 448×1 multimodal feature matrix. The multimodal feature matrix is ​​multiplied by the first weight matrix Wq, the second weight matrix Wk, and the third weight matrix Wv, which have dimensions of 448×64, respectively, to generate a query matrix Q, a key matrix K, and a value matrix V with dimensions of 64×1. The initial values ​​of the first weight matrix Wq, the second weight matrix Wk, and the third weight matrix Wv are generated using a random normal distribution with a mean of 0 and a variance of 0.01. According to the formula Calculate the attention score matrix, where d is the feature dimension (d=64 in this example), K is the key matrix, and d is the feature dimension. K T The transpose of the key matrix K yields an attention score matrix with dimensions 64×64; The attention score matrix is ​​normalized by applying the Softmax function to each row to obtain an attention weight matrix with a dimension of 64×64. After normalization, the sum of the weights in each row is 1, which is used to characterize the contribution of each modality feature to driver state recognition. Multiplying the attention weight matrix by the value matrix V yields an attention output matrix with a dimension of 64×1; The attention output matrix is ​​used as a dynamic adjustment factor and combined with preset sensor static weights (eye feature static weight 0.5, heart rate feature static weight 0.3, steering wheel feature static weight 0.2) to generate dynamic weight coefficients for each modality. , , The calculation method is as follows =Attention output matrix corresponding dimension value × original static weight; The eye feature vector, heart rate feature vector, and driving behavior feature vector are respectively coupled with their corresponding dynamic weight coefficients using the formula... The weighted summation is mapped to the Sigmoid function to calculate the final driver fatigue index. σ () is the Sigmoid activation function: 1 / (1+e^{-x}), which maps any real number to the probability interval (0,1). X i Let be the characteristic value of the i-th type of sensor.

[0056] In one embodiment, the driver fatigue index (probability) can be calculated using the following formula.

[0057] Indicates dynamic weighting coefficients. ω i =Relevance score Original static weights (preset fixed weights); b represents the bias term (learned during training, initially 0 in this embodiment, and converges to -0.8 after training).

[0058] Please see Figure 7 , Figure 7 This is a flowchart illustrating an exemplary method for generating a mood fluctuation index according to this application. Figure 7 In the assessment, the state evaluation results include a mood fluctuation index, which is generated using methods including: Step S710: Extract multiple second features representing the driver's emotional state from the multimodal driving behavior data; The second feature includes: facial expression feature vectors representing the movement of specific muscle groups extracted from facial image sequences via a three-dimensional convolutional neural network (3D-CNN) or a specific facial action unit analysis model; and driving behavior features representing the urgency and non-stability of the operation extracted from lane departure and brake pedal operation timing signals. The specific structure of 3D-CNN is as follows: 3D-CNN includes an input layer (input is a sequence of facial images, with dimensions [16, 128, 128, 3], where 16 is the number of frames, 128×128 is the size of a single frame, and 3 is the number of RGB channels), two 3D convolutional layers (with 3×3×3 kernels, 16 and 32 kernels respectively, and a stride of 1), an activation layer (ReLU), a 3D pooling layer (with 2×2×2 pooling kernels and a stride of 2), two fully connected layers (with output dimensions of 256 and 128 respectively), and a feature output layer (outputting a 128-dimensional facial expression feature vector), used to extract facial muscle movements (such as frowning, pursing lips, etc.) and emotion-related features; driving behavior features are extracted through 1D-CNN, extracting features related to emotional fluctuations such as lane departure frequency and brake pedal operation speed.

[0059] Step S720: Fuse multiple second features to obtain a comprehensive feature vector; The comprehensive feature vector can be formed by splicing or weighted fusion of the second features of the different modalities mentioned above (128-dimensional facial expression feature vector and 64-dimensional driving behavior feature vector).

[0060] Step S730: The comprehensive feature vector is mapped by a preset classifier to obtain an emotion fluctuation index that represents whether the driver is in an emotional fluctuation state.

[0061] The classifier is a classification model built on a deep neural network. The last layer of the classifier uses the Softmax activation function to map the input feature vector to a probability distribution of emotional states as an emotional fluctuation index.

[0062] Driving behavior features and facial expression features are fed into a feature fusion layer, where a weighted fusion is used to generate a comprehensive feature vector. This comprehensive feature vector is then input into a classifier, which classifies the driver based on the input features to obtain a probability value representing whether the driver is in an emotionally fluctuating state, i.e., the emotional fluctuation index.

[0063] In one embodiment, the classifier consists of several fully connected layers, with the last layer being a Softmax layer. The combined feature vector is passed through the Softmax layer and outputs a sentiment fluctuation index between 0 and 1, with a higher value indicating greater emotional fluctuation of the driver.

[0064] In one embodiment, at least one optical parameter includes: brightness and color; based on the state assessment results, a lighting control strategy is generated, including: In response to the driver fatigue index exceeding a preset driver fatigue threshold, the first control strategy in the lighting control strategy is executed. The first control strategy includes at least one of the following: reducing the brightness of the interior lighting area based on the driver fatigue index, and adjusting the hue of the interior lights to a preset warm hue range based on the environmental comfort assessment value. In response to the mood fluctuation index exceeding the preset mood fluctuation threshold, the second control strategy in the lighting control strategy is executed. The second control strategy includes: reducing the brightness of the interior lighting area based on the driver fatigue index and adjusting the interior lights to a soft warm light with a gradient effect based on the mood fluctuation index.

[0065] The driver fatigue index and mood fluctuation index are monitored in real time and compared with their respective preset thresholds. When the driver fatigue index exceeds the preset driver fatigue threshold, the first control strategy is invoked: The brightness of the in-vehicle lighting area is reduced based on the driver fatigue index. Specifically, the brightness B is adjusted using the following formula. ,in, Indicates the base brightness. Indicates the brightness adjustment coefficient. Indicates driver fatigue index; Based on environmental comfort assessment values, the color tone of the vehicle's interior lighting is adjusted towards a preset warm color range. Specifically, color C is adjusted using the following formula. Color adjustment: ,in, Represents the basic color value. Indicates the color adjustment factor. This indicates the environmental comfort assessment value.

[0066] When the mood fluctuation index exceeds the preset mood fluctuation threshold, the second control strategy is invoked: The brightness of the in-vehicle lighting area is reduced based on the driver fatigue index. Specifically, the brightness B is adjusted using the following formula. ,in, Indicates the base brightness. Indicates the brightness adjustment coefficient. Indicates driver fatigue index; Based on the mood fluctuation index, the car interior lights are adjusted to a soft, warm light with a gradient effect. Specifically, color C is adjusted using the following formula: Color adjustment: ,in, Represents the basic color value. Indicates the color adjustment factor. This indicates an index of emotional fluctuations.

[0067] Please see Figure 8 , Figure 8 This is a flowchart illustrating an exemplary method for determining environmental comfort assessment values ​​according to this application. Figure 8 In China, the methods for determining environmental comfort assessment values ​​include: Step S810: Obtain at least one type of in-vehicle environmental data; The in-vehicle environmental data includes light intensity, temperature, humidity, and air quality. Light intensity is collected by a light sensor to assess visual comfort inside the vehicle; temperature is collected by a temperature sensor to assess thermal comfort; humidity is collected by a temperature sensor to record the humidity level inside the vehicle; and air quality is collected by an air quality sensor to determine the health status of the in-vehicle environment. Air quality can be reflected by air quality indicators such as CO2 concentration and PM2.5 inside the vehicle.

[0068] Step S820: Assign a preset weighting coefficient to each type of in-vehicle environment data; Step S830: According to the preset weighting coefficient, at least one corresponding in-vehicle environment data is weighted and processed, and the weighted in-vehicle environment data is combined to generate an environmental comfort assessment value.

[0069] The vehicle system receives environmental data in real time from various environmental sensors. The environmental data is weighted using a pre-set set of weighting coefficients (e.g., α for light intensity L, β for temperature T, γ for humidity H, and δ for air quality AQ) to obtain an environmental comfort assessment value S. .

[0070] Please see Figure 9 , Figure 9 This is a schematic diagram illustrating the process of iteratively optimizing a lighting control strategy using a reinforcement learning algorithm according to an embodiment of this application. Figure 9 In this context, iterative optimization methods include: S910 obtains the current state from the state space based on the driver fatigue index, the emotion fluctuation index, and the environmental comfort assessment value; wherein, the state in the state space is defined by the driver fatigue index, the emotion fluctuation index, and the environmental comfort assessment value. In this context, the state space is a mathematical representation space composed of multidimensional parameters. Driver fatigue index, mood fluctuation index, and environmental comfort assessment value together constitute a state; a set of multiple states forms the state space.

[0071] S920 determines the lighting control strategy corresponding to the current state from the action space based on the current state; wherein, the lighting control strategy in the action space is defined by the brightness and color values ​​of the interior lights; The action space is the collection of all lighting control strategies (actions). A lighting control strategy is specifically defined by the brightness and color values ​​of the interior lights; each specific combination of brightness and color constitutes an action.

[0072] S930 generates a reward signal corresponding to the lighting control strategy based on the driver's feedback data collected after the lighting control strategy is executed. The feedback data includes driver physiological feedback data and / or behavioral feedback data. A preset reward function maps the feedback data to a reward value. A positive reward value indicates desired behavioral improvement; a negative reward value indicates undesirable behavior.

[0073] S940 updates the value assessment function of the lighting adjustment strategy based on the reward signal, the state transition results observed after executing the lighting control strategy, the preset learning rate and discount factor, so as to optimize the decision mapping relationship from state to action. Among them, the discount factor is used to balance the weight of immediate reward and future reward in value assessment, and the learning rate is used to adjust the degree of influence of this update on historical value estimation.

[0074] The value assessment function, usually denoted as the Q-function, is a core component of reinforcement learning algorithms. It estimates the long-term cumulative expected reward of taking a certain action in a specific state. The learning rate is a hyperparameter used to control the strength of updates to the existing value estimate by new experiences; the learning rate is used in... express, A discount factor is used to discount future rewards when calculating long-term returns, balancing immediate gains with long-term benefits. The discount factor is used... express, .

[0075] In each optimization process, a state vector consisting of fatigue index, mood index, and comfort value is obtained, and an action (brightness and color value) is selected from the action space and executed. Driver feedback within a time window after the action execution is collected, a reward signal is generated, and the observation environment is transferred to the new state. The value evaluation function Q is updated using the collected empirical data. Its update formula is:

[0076] in, This represents the Q value of taking action a in state s; Indicates the learning rate; Indicates an immediate reward; Indicates the discount factor; Indicates the next state; Indicates the next action.

[0077] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0078] Figure 10 This is a block diagram of an exemplary vehicle lighting control device according to this application. Figure 9 As shown, this application provides a vehicle lighting control device, which includes: Data acquisition module 1010 is used to acquire multimodal driving behavior data; The state recognition module 1020 is used to perform state recognition based on multimodal driving behavior data and a pre-trained deep learning model to obtain the driver's state assessment results. The strategy generation module 1030 is used to generate lighting control strategies based on the state evaluation results; The parameter control module 1040 is used to control at least one optical parameter of the in-vehicle lighting system through a lighting control strategy.

[0079] It should be noted that the vehicle lighting control device and the vehicle lighting control method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the vehicle lighting control device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0080] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the vehicle lighting control method provided in the above embodiments.

[0081] Figure 11 A schematic diagram of a computer system suitable for implementing the memory of embodiments of this application is shown. It should be noted that... Figure 11 The computer system with the memory shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0082] like Figure 11As shown, the computer system 1100 includes a Central Processing Unit (CPU) 1101, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 1102 or programs loaded from storage into Random Access Memory (RAM) 1103. The RAM also stores various programs and data required for system operation. The CPU 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An Input / Output (I / O) interface 1105 is also connected to the bus 1104.

[0083] The following components are connected to I / O interface 1105: an input section 1106 including a keyboard, mouse, etc.; an output section 1107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to I / O interface 1105 as needed. Removable media 1111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1110 as needed so that computer programs read from them can be installed into storage section 1108 as needed.

[0084] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the vehicle lighting control method of the aforementioned embodiments. In such embodiments, the computer program can be downloaded and installed from a network via a communication portion, and / or installed from removable medium 1111. When the computer program is executed by the central processing unit (CPU) 1101, it performs various functions defined in the system of this application.

[0085] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM) 1103, read-only memory (ROM) 1102, erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block or combination of blocks in a block diagram or flowchart may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0087] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0088] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a computer's processor, causes the computer to perform the aforementioned vehicle lighting control method. This computer-readable storage medium may be included in the memory described in the above embodiments, or it may exist independently and not incorporated into that memory.

[0089] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the vehicle lighting control method provided in the various embodiments described above.

[0090] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A vehicle lighting control method, characterized in that, include: Acquire multimodal data on driving behavior; Based on the aforementioned multimodal driving behavior data, a pre-trained deep learning model is used to identify the driver's state and obtain the driver's state assessment results. Based on the state assessment results, a lighting control strategy is generated; The lighting control strategy controls at least one optical parameter of the vehicle's interior lighting system.

2. The vehicle lighting control method according to claim 1, characterized in that, The condition assessment result includes a driver fatigue index, and the method for generating the driver fatigue index includes: Multiple first features characterizing the driver's fatigue state are extracted from the multimodal driving behavior data; The multiple first features are combined to construct a multimodal feature matrix; The multimodal feature matrix is ​​calculated based on an attention mechanism to generate dynamic weight coefficients representing each modality of data. Based on the dynamic weighting coefficients, multiple features in the multimodal feature matrix are weighted to obtain weighted fusion features; By mapping the weighted fusion features using a preset activation function, a driving fatigue index that characterizes whether the driver is in a state of fatigue is obtained.

3. The vehicle lighting control method according to claim 2, characterized in that, The calculation of the multimodal feature matrix based on the attention mechanism to generate dynamic weight coefficients representing the data of each modality includes: Based on the multimodal feature matrix, calculate the query matrix, key matrix, and value matrix; Calculate the attention score matrix based on the query matrix and the key matrix; The attention score matrix is ​​normalized to generate an attention weight matrix; Multiplying the attention weight matrix by the value matrix yields the attention output matrix; The dynamic weight coefficients are determined based on the attention output matrix.

4. The vehicle lighting control method according to claim 1, characterized in that, The state assessment result includes an emotion fluctuation index, and the method for generating the emotion fluctuation index includes: Extract multiple second features representing the driver's emotional state from the multimodal driving behavior data; The multiple second features are fused to obtain a comprehensive feature vector; The comprehensive feature vector is mapped by a preset classifier to obtain an emotion fluctuation index that characterizes whether the driver is in an emotional fluctuation state.

5. The vehicle lighting control method according to claim 1, characterized in that, The at least one optical parameter includes: brightness and color; the step of generating a lighting control strategy based on the state evaluation result includes: In response to the driver fatigue index exceeding a preset driver fatigue threshold, a first control strategy in the lighting control strategy is executed. The first control strategy includes: reducing the brightness of the in-vehicle lighting area based on the driver fatigue index and adjusting the hue of the in-vehicle lights to a preset warm hue range based on the environmental comfort assessment value. In response to the mood fluctuation index exceeding a preset mood fluctuation threshold, the second control strategy in the lighting control strategy is executed. The second control strategy includes: reducing the brightness of the interior lighting area based on the driver fatigue index, and adjusting the interior lights to a soft, warm light with a gradient effect based on the mood fluctuation index.

6. The vehicle lighting control method according to claim 5, characterized in that, The method for determining the environmental comfort assessment value includes: Acquire at least one type of in-vehicle environmental data; Assign preset weighting coefficients to each type of in-vehicle environment data; According to the preset weighting coefficient, the corresponding at least one in-vehicle environment data is weighted and then combined to generate an environmental comfort assessment value.

7. The vehicle lighting control method according to claim 1, characterized in that, The vehicle lighting control method further includes: iteratively optimizing the lighting control strategy using a reinforcement learning algorithm; the iterative optimization includes: The current state is obtained from the state space based on the driver fatigue index, the emotion fluctuation index, and the environmental comfort assessment value; wherein, the state in the state space is defined by the driver fatigue index, the emotion fluctuation index, and the environmental comfort assessment value. Based on the current state, a lighting control strategy corresponding to the current state is determined from the action space; wherein, the lighting control strategy in the action space is defined by the brightness and color values ​​of the in-vehicle lights; Based on the driver's feedback data collected after the execution of the lighting control strategy, a reward signal corresponding to the lighting control strategy is generated; Based on the reward signal, the state transition results observed after executing the lighting control strategy, the preset learning rate and discount factor, the value evaluation function of the lighting adjustment strategy is updated to optimize the decision mapping relationship from state to action; wherein, the discount factor is used to balance the weight of immediate reward and future reward in value evaluation, and the learning rate is used to adjust the degree of influence of this update on historical value estimation.

8. A vehicle lighting control device, characterized in that, The vehicle lighting control includes: The data acquisition module is used to acquire multimodal data on driving behavior; The state recognition module is used to perform state recognition based on the multimodal driving behavior data and a pre-trained deep learning model to obtain the driver's state assessment result. The strategy generation module is used to generate a lighting control strategy based on the state evaluation results. The parameter control module is used to control at least one optical parameter of the in-vehicle lighting system through the lighting control strategy.

9. A vehicle lighting control device, characterized in that, include: One or more processors; A memory for storing one or more programs that, when executed by the one or more processors, cause the vehicle lighting control device to perform the steps of the vehicle lighting control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the steps of the vehicle lighting control method according to any one of claims 1 to 7.

Images