Vehicle atmosphere lamp control method and device based on intention recognition and electronic equipment

Abstract

The invention discloses a vehicle atmosphere lamp control method and device based on intention recognition and electronic equipment. The method comprises the steps that fixation point information and expression information of a driver and passengers are obtained through recognition according to face image information; determining interaction feature data according to the fixation point information and the interaction information, determining emotional feature data according to the expression information and the physiological state information, and determining scene feature data according to the driving state information and the vehicle exterior environment information; inputting the interaction feature data, the emotion feature data and the scene feature data into a pre-trained atmosphere lamp control intention recognition model to obtain a target control intention and a corresponding confidence coefficient; and determining target atmosphere lamp parameters according to the target control intention, the confidence coefficient and a pre-constructed atmosphere lamp parameter mapping library, and performing atmosphere lamp control on the current vehicle according to the target atmosphere lamp parameters. According to the invention, the efficiency and accuracy of vehicle atmosphere lamp control are improved, refined and personalized atmosphere lamp control is realized, and the method can be applied to the technical field of intelligent cabins.

Description

Technical Field

[0001] This invention relates to the field of smart cockpit technology, and in particular to a vehicle ambient lighting control method, device, and electronic device based on intent recognition. Background Technology

[0002] In existing technologies, most automotive ambient lighting control methods rely on fixed mode switching, such as selecting "Sport" or "Comfort" modes via the vehicle's infotainment system, or triggering based on a single condition, such as adjusting brightness according to vehicle speed or time. These methods suffer from the following problems: 1) Limited intent recognition dimensions: Relying solely on basic data such as vehicle speed and time, it fails to capture the potential needs of drivers and passengers; 2) Insufficient safety adaptability: It does not take into account the emotional and attentional state of the driver and passengers. For example, if the user is in a bad mood, the high saturation light will still be triggered, which may easily aggravate visual interference. 3) Lack of feedback loop: Only control commands are executed, but the actual status of the ambient light and subsequent user operations are not collected, making it impossible to correct parameter deviations.

[0003] In summary, existing technologies cannot establish a dynamic relationship between "driver intent, driving status, and lighting parameters," making it difficult to meet the needs of intelligent cockpits for refined and personalized ambient lighting control, thus affecting the user's driving experience. Summary of the Invention

[0004] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0005] Therefore, one objective of this invention is to provide a vehicle ambient lighting control method based on intent recognition. This method improves the efficiency and accuracy of vehicle ambient lighting control, enables refined and personalized ambient lighting control, and enhances the user's driving experience.

[0006] Another objective of this invention is to provide a vehicle ambient lighting control device based on intent recognition.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include: On one hand, embodiments of the present invention provide a vehicle ambient lighting control method based on intent recognition, comprising the following steps: Acquire facial image information, interaction information, physiological status information of drivers and passengers, as well as current vehicle driving status information and external environment information; The gaze point and facial expression information of the driver and passengers are obtained based on the facial image information. Interaction feature data is determined based on the gaze point information and the interaction information; emotional feature data is determined based on the facial expression information and the physiological state information; and scene feature data is determined based on the driving state information and the external environment information. The interaction feature data, the emotion feature data, and the scene feature data are input into a pre-trained ambient light control intent recognition model to obtain the target control intent and the corresponding confidence level. The target ambient light parameters are determined based on the target control intent, the confidence level, and the pre-built ambient light parameter mapping library, and the ambient light is controlled on the current vehicle based on the target ambient light parameters.

[0008] Furthermore, in one embodiment of the present invention, the step of identifying the gaze point information and facial expression information of the driver / passenger based on the facial image information specifically includes: Based on the facial image information, the eye region image and the face region image are extracted; The pupil center position and corneal reflection point position are obtained by locating the pupil center and extracting the corneal reflection point position from the eye region image. The eyeball is modeled based on the pupil center position and the corneal reflection point position to obtain the gaze direction; The gaze direction is mapped to the cockpit space coordinate system to obtain the gaze point information; The facial expression information is obtained by performing expression recognition on the facial image information.

[0009] Furthermore, in one embodiment of the present invention, the interaction information is voice interaction information or action interaction information, and the step of determining interaction feature data based on the gaze point information and the interaction information specifically includes: Obtain the ambient lighting distribution information of the current vehicle, match the gaze point information with the ambient lighting distribution information, and determine the attention features based on the matching result; The voice interaction information is subjected to speech recognition and keyword extraction to obtain speech features, or the action interaction information is subjected to action recognition and action database matching to obtain action features; The attention features and the speech / action features are temporally aligned and fused to obtain the interaction feature data.

[0010] Furthermore, in one embodiment of the present invention, determining emotional characteristic data based on the facial expression information and the physiological state information specifically includes: The emotional state of the driver and passengers is identified based on the facial expression information, and the emotional state is vectorized based on the valence-arousal model to obtain a two-dimensional emotion vector. Based on the physiological state information, multidimensional physiological features are determined, and the two-dimensional emotion vector is expanded in dimension based on the multidimensional physiological features to obtain the emotion feature data.

[0011] Furthermore, in one embodiment of the present invention, the driving state information includes driving mode, vehicle speed, vehicle acceleration, vehicle steering angle, and continuous driving duration; the external environment information includes ambient light intensity, weather conditions, road type, and traffic congestion; and the step of determining scene feature data based on the driving state information and the external environment information specifically includes: The driving state information and the external environment information are normalized to obtain the scene feature data.

[0012] Furthermore, in one embodiment of the present invention, the ambient light control intent recognition model is trained through the following steps: Acquire interaction feature samples, emotion feature samples, and scene feature samples from historical driving scenarios; Training samples are constructed based on the interaction feature samples, the emotion feature samples, and the scene feature samples, and the control intent labels corresponding to the training samples are determined by manual annotation. A multi-branch convolutional neural network is constructed, which includes an input layer, an interaction branch network, an emotion branch network, a scene branch network, a feature fusion layer, and an output layer. The training samples are input into the input layer, and the interaction feature vector is obtained by extracting features from the interaction feature samples through the interaction branch network. The emotion feature vector is obtained by extracting features from the emotion feature samples through the emotion branch network. The scene feature vector is obtained by extracting features from the feature fusion layer through the scene branch network. The feature fusion layer performs feature fusion on the interaction feature vector, the emotion feature vector, and the scene feature vector based on an attention mechanism to obtain a fused feature vector. The output layer maps the fused feature vector into a control intent recognition result. The loss value is determined based on the control intent recognition result and the corresponding control intent label; The parameters of the multi-branch convolutional neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained ambient light control intention recognition model.

[0013] Furthermore, in one embodiment of the present invention, determining the target ambient light parameters based on the target control intent, the confidence level, and a pre-built ambient light parameter mapping library specifically includes: According to the target control intent, multiple sets of candidate ambient light parameters are matched in the ambient light parameter mapping library, and the priority of each candidate ambient light parameter is determined. When the confidence level is greater than or equal to a preset first threshold, the candidate ambient light parameter with the highest priority is determined as the target ambient light parameter. When the confidence level is greater than or equal to a preset second threshold and less than the first threshold, the historical ambient lighting parameters preferred by the driver and passengers are obtained, and the candidate ambient lighting parameters are optimized based on the historical ambient lighting parameters to obtain the target ambient lighting parameters. When the confidence level is less than the second threshold, the candidate ambient light parameters are displayed through the vehicle system, and the target ambient light parameters are determined based on the driver's and passenger's selection.

[0014] On the other hand, embodiments of the present invention provide a vehicle ambient lighting control device based on intent recognition, comprising: The information acquisition module is used to acquire facial image information, interaction information, physiological status information of drivers and passengers, as well as current vehicle driving status information and external environment information; The information recognition module is used to identify the gaze point information and facial expression information of the driver and passenger based on the facial image information; The feature determination module is used to determine interaction feature data based on the gaze point information and the interaction information, determine emotion feature data based on the facial expression information and the physiological state information, and determine scene feature data based on the driving state information and the external environment information. The intent recognition module is used to input the interaction feature data, the emotion feature data and the scene feature data into a pre-trained ambient light control intent recognition model to obtain the target control intent and the corresponding confidence level. An ambient lighting control module is used to determine target ambient lighting parameters based on the target control intent, the confidence level, and a pre-built ambient lighting parameter mapping library, and to control the ambient lighting of the current vehicle based on the target ambient lighting parameters.

[0015] On the other hand, embodiments of the present invention provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above-described vehicle ambient lighting control method based on intent recognition.

[0016] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described intention-based vehicle ambient lighting control method.

[0017] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the above-described vehicle ambient lighting control method based on intent recognition.

[0018] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention: This invention acquires facial image information, interaction information, physiological state information of the driver and passengers, as well as the current driving state information and external environment information of the vehicle. Based on the facial image information, it identifies the driver's and passengers' gaze points and facial expressions. Based on the gaze point information and interaction information, it determines interaction feature data; based on the facial expression information and physiological state information, it determines emotional feature data; and based on the driving state information and external environment information, it determines scene feature data. The interaction feature data, emotional feature data, and scene feature data are input into a pre-trained ambient lighting control intent recognition model to obtain the target control intent and corresponding confidence level. Based on the target control intent, confidence level, and a pre-built ambient lighting parameter mapping library, it determines the target ambient lighting parameters and controls the ambient lighting of the current vehicle according to the target ambient lighting parameters. This invention identifies the gaze point and facial expression information of drivers and passengers based on facial image information. It determines interaction feature data based on gaze point and interaction information, emotion feature data based on facial expression and physiological state information, and scene feature data based on driving state information and external environment information. Using a pre-trained ambient lighting control intent recognition model, it identifies control intent from three branches: interaction features, emotion features, and scene features. Based on the identified target control intent, corresponding confidence level, and a pre-built ambient lighting parameter mapping library, it determines the target ambient lighting parameters. This improves the efficiency and accuracy of vehicle ambient lighting control, achieving refined and personalized ambient lighting control and enhancing the user's driving experience. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1A flowchart illustrating the steps of a vehicle ambient lighting control method based on intent recognition, provided in an embodiment of the present invention; Figure 2 A structural block diagram of a vehicle ambient lighting control device based on intent recognition provided in an embodiment of the present invention; Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0023] The vehicle ambient lighting control method based on intent recognition provided in this invention can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the vehicle ambient lighting control method based on intent recognition, but is not limited to the above forms.

[0024] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0025] It should be noted that in various specific embodiments of the present invention, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of the present invention require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of the embodiments of the present invention acquired.

[0026] The purpose of this invention is to: 1) achieve automatic ambient lighting control without manual operation, capture potential intentions through data such as facial images, interactions, and physiological states, reduce driver distraction, and improve safety; 2) construct an intention recognition system that integrates multi-source perception data to accurately match the ambient lighting needs of drivers and passengers in different emotional and attentional states; 3) establish a feedback loop of "command execution - deviation detection - parameter update" to continuously optimize control accuracy and adapt to changes in user habits.

[0027] Reference Figure 1 This invention provides a vehicle ambient lighting control method based on intent recognition, specifically including the following steps: S101. Obtain facial image information, interaction information, physiological status information of drivers and passengers, as well as current vehicle driving status information and external environment information; S102. Based on facial image information, the gaze point information and facial expression information of the driver and passengers are obtained; S103. Determine interaction feature data based on gaze point information and interaction information, determine emotional feature data based on facial expression information and physiological state information, and determine scene feature data based on driving state information and external environment information. S104. Input the interaction feature data, emotion feature data and scene feature data into the pre-trained ambient light control intention recognition model to obtain the target control intention and the corresponding confidence level. S105. Determine the target ambient light parameters based on the target control intent, confidence level, and pre-built ambient light parameter mapping library, and control the ambient light of the current vehicle based on the target ambient light parameters.

[0028] Specifically, this embodiment of the invention identifies the gaze point and facial expression information of drivers and passengers based on facial image information. It determines interaction feature data based on gaze point and interaction information, determines emotional feature data based on facial expression and physiological state information, and determines scene feature data based on driving state information and external environment information. Based on a pre-trained ambient lighting control intention recognition model, it identifies control intentions from three branches: interaction features, emotional features, and scene features. Based on the identified target control intention, the corresponding confidence level, and a pre-built ambient lighting parameter mapping library, it determines the target ambient lighting parameters, thereby improving the efficiency and accuracy of vehicle ambient lighting control, realizing refined and personalized ambient lighting control, and enhancing the user's driving experience.

[0029] This invention utilizes existing automotive hardware and newly added sensors to collect facial image information, interaction information, physiological state information of drivers and passengers, as well as current vehicle driving status information and external environment information, as detailed below: Facial image information: Facial image information (including eyes and face images) of drivers and passengers is collected through the DMS (Driver Monitoring System) camera; Interactive information: Voice data of drivers and passengers is collected through the vehicle microphone (such as "It's too bright"), and action data of drivers and passengers is collected through the gesture sensor (such as raising a hand to block the light, pointing at the light strip, etc.). Physiological status information: Collects physiological data from the seat's heart rate sensor (to assist in emotion assessment); Driving status information: Vehicle speed, acceleration, steering angle, and braking frequency are collected via CAN bus (e.g., high-frequency braking corresponds to congested road conditions), and the driving mode (Sport / Comfort / Economy) is obtained through the vehicle system. External environment information: The system collects ambient light intensity (e.g., at night, in tunnels) through external light sensors, determines the current road type (urban road, highway) through GPS positioning, and obtains current weather conditions through rain sensors.

[0030] As a further optional implementation, the gaze point information and facial expression information of the driver and passengers are obtained based on facial image information, specifically including: S1021. Extract the eye region image and the face region image based on the face image information; S1022. Perform pupil center localization and corneal reflection point extraction on the eye region image to obtain the pupil center position and corneal reflection point position; S1023. Model the eyeball based on the position of the pupil center and the position of the corneal reflex point to obtain the gaze direction; S1024. Map the gaze direction to the cockpit space coordinate system to obtain gaze point information; S1025. Perform facial expression recognition on the facial image information to obtain facial expression information.

[0031] Specifically, the eye region and face region are separated from the face image, and background interference is filtered to obtain eye region images and face region images. Eye movement feature extraction is performed on the eye region image. Specifically, pupil center localization and corneal reflection point extraction are performed on the eye region image to obtain the pupil center position and corneal reflection point position. Then, the gaze direction corresponding to the current frame image is obtained through eyeball modeling, and then mapped to the real cabin space coordinate system to generate gaze point coordinates, which is convenient for subsequent matching of the ambient light position coordinates in the cabin to obtain attention information (such as "gazing at the door panel light strip" to indicate the intention to adjust). For facial image information, facial expressions are recognized (such as frowning and smiling), which is convenient for subsequent mapping of expressions to emotional information (irritability / relaxation / focus) through a pre-trained regression model (refer to linear regression / random forest).

[0032] As a further optional implementation, the interaction information is either voice interaction information or action interaction information. Interaction feature data is determined based on gaze point information and interaction information, specifically including: S1031. Obtain the ambient light distribution information of the current vehicle, match the gaze point information with the ambient light distribution information, and determine the attention characteristics based on the matching result. S1032. Perform speech recognition and keyword extraction on the speech interaction information to obtain speech features, or perform action recognition and action database matching on the action interaction information to obtain action features. S1033. Perform temporal alignment and data fusion on attention features and speech / motor features to obtain interaction feature data.

[0033] Specifically, the distribution locations of ambient lights within the cabin are obtained, and the coordinates of the gaze points of the identified occupants are matched with the distribution locations of the ambient lights. Based on the matching results, attention characteristics are determined, such as whether the occupants are paying attention to the ambient lights or the light strips. Voice interaction information is converted into text using speech recognition technology, and keywords are extracted using natural language processing (NLP) to obtain voice features (e.g., "too bright" indicates an intention to reduce brightness). Skeletal key point recognition is performed on motion data, and it is matched with a preset motion library to obtain motion features (e.g., raising a hand to block the light). The attention features and voice / motion features are temporally aligned and fused to obtain interaction feature data.

[0034] As a further optional implementation, emotional characteristic data is determined based on facial expression information and physiological state information, specifically including: S1034. The emotional state of the driver and passengers is identified based on facial expression information, and the emotional state is vectorized based on the valence-arousal model to obtain a two-dimensional emotion vector. S1035. Determine multidimensional physiological features based on physiological state information, and expand the dimensions of the two-dimensional emotion vector based on the multidimensional physiological features to obtain emotion feature data.

[0035] Specifically, the emotional states of drivers and passengers are obtained based on facial expression information, including eight basic emotions (anger, happiness, sadness, surprise, fear, disgust, neutrality, and contempt). Then, the emotional states are vectorized based on a valence-arousal model, where valence represents the degree of emotional pleasure (negative → positive), and arousal represents the degree of physiological activation (calm → excitement). A pre-established correspondence between emotion categories and valence-arousal is based on psychological research; for example, "happiness" has a valence of 0.8 and an arousal of 0.7, while "anger" has a valence of -0.9 and an arousal of 0.9. The valence of "sadness" is -0.8 and the arousal value is 0.3, while the valence of "neutrality" is 0 and the arousal value is 0.1, ultimately forming a two-dimensional emotion vector (valence, arousal). In addition, the vector values ​​can be fine-tuned by combining facial expression intensity (such as the amplitude of muscle movements), for example, the arousal value of high-intensity anger can be increased to 0.95. Multidimensional physiological characteristics are determined based on physiological state information, which may include statistical data such as heart rate variability, electrodermal signals, and electromyographic signals. Then, the multidimensional physiological characteristics are combined with the aforementioned two-dimensional emotion vector to obtain higher-dimensional emotion feature data, such as (valence, arousal, heart rate variability, and variance of electrodermal signals).

[0036] As an optional implementation, the driving status information includes driving mode, vehicle speed, vehicle acceleration, vehicle steering angle, and continuous driving duration; the external environment information includes ambient light intensity, weather conditions, road type, and traffic congestion. Scene feature data is determined based on the driving status information and the external environment information, specifically as follows: S1036. Normalize the driving status information and the external environment information to obtain scene feature data.

[0037] Specifically, driving status information and external environment information are normalized. For example, ambient light intensity is converted into a 0-100 brightness index (0 represents nighttime and 100 represents noon), and data such as vehicle speed and acceleration are normalized to eliminate differences in magnitude.

[0038] As an optional implementation, the ambient lighting control intent recognition model is trained through the following steps: S201. Obtain interaction feature samples, emotion feature samples, and scene feature samples from historical driving scenarios; S202. Construct training samples based on interaction feature samples, emotion feature samples, and scene feature samples, and determine the control intent labels corresponding to the training samples through manual annotation. S203. Construct a multi-branch convolutional neural network, which includes an input layer, an interaction branch network, an emotion branch network, a scene branch network, a feature fusion layer, and an output layer. S204. Input the training samples into the input layer, extract the interactive feature vector from the interactive feature samples through the interactive branch network, extract the emotional feature vector from the emotional feature samples through the emotion branch network, and extract the scene feature vector from the feature fusion layer through the scene branch network. S205. The interaction feature vector, emotion feature vector and scene feature vector are fused by the feature fusion layer based on the attention mechanism to obtain the fused feature vector. S206. The fused feature vector is mapped to the control intent recognition result through the output layer; S207. Determine the loss value based on the control intent recognition result and the corresponding control intent label; S208. Update the parameters of the multi-branch convolutional neural network based on the loss value using the backpropagation algorithm to obtain the trained ambient light control intention recognition model.

[0039] Specifically, based on the same steps used to acquire interaction feature data, emotion feature data, and scene feature data, interaction feature samples, emotion feature samples, and scene feature samples from historical driving scenarios are obtained. Then, the collected interaction, emotion, and scene feature samples are aligned by timestamps to form a multimodal feature sequence. An expert team labels the samples with corresponding control intent tags. The tags need to be detailed to specific operations (such as "turn on ambient light", "switch color to blue", "adjust brightness to 50%", "turn off ambient light", etc.) to ensure the correlation between the tags and the feature sequence.

[0040] The multi-branch convolutional neural network of this invention includes an input layer, an interaction branch network, an emotion branch network, a scene branch network, a feature fusion layer, and an output layer, as detailed below: Input layer: Receives standardized interaction, emotion, and scene feature samples (e.g., interaction features are time-series data, emotion features are image / physiological signal vectors, and scene features are numerical parameters).

[0041] Interactive branch network: Employs 1D convolutional layers (Conv1D) to extract temporal features of speech / gestures and capture dynamic changes in operation sequences (such as the combined intent of "waving + pointing").

[0042] Emotional branch network: Use 2D convolutional layers (Conv2D) to process facial expression image features, or fully connected layers to process physiological signals, and output emotional state vectors (such as "pleasant → preference for warm colors").

[0043] Scene branch network: Through convolutional layers or embedding layers, discrete scene parameters (such as weather and road conditions) are transformed into high-dimensional feature vectors to reflect the influence of the environment on intention (such as "rainy day → preference for cool colors to improve alertness").

[0044] Feature fusion layer: Based on the attention mechanism, the output features (interaction / emotion / scene vectors) of the three branches are weighted and fused, and the importance weight of the features is automatically assigned (e.g., voice commands have a higher weight than vague gestures).

[0045] Output layer: The fused feature vector is mapped to a control intention probability distribution through a fully connected layer and a Softmax activation function (e.g., "turn on ambient light" probability 0.85, "adjust color" probability 0.12).

[0046] The input layer assigns training samples to corresponding branch networks, outputting interaction feature vectors (V1), emotion feature vectors (V2), and scene feature vectors (V3) respectively; the fusion layer calculates weights (α, β, γ) through an attention mechanism, outputting a fused feature vector: V 融合 =α·V1+β·V2+γ·V3 (where α+β+γ=1).

[0047] The output layer will V 融合 The mapping is to the recognition result (probability distribution). The loss value (Loss) between the predicted result and the labeled label is calculated by the cross-entropy loss function. Based on the loss value, the weights of each layer of the network (including branch convolutional kernels, attention weights, and fully connected layer parameters) are updated by the backpropagation algorithm (such as Adam optimizer). The training is iterated until the loss converges, and finally the ambient light control intention recognition model is obtained. This model can output the control intention category (such as lowering brightness, switching color temperature, and flashing mode) and the corresponding confidence level (such as judging it as "relaxation intention" with 95% confidence) based on the input data.

[0048] It should be noted that if the driver's intentions conflict with the passengers' intentions (e.g., the driver needs anti-drowsiness lights while the passengers need dim lighting to rest), the principle of "driving safety first" should be followed (e.g., only the dashboard light strip should be turned on, and the rear seats should be kept at a low brightness).

[0049] As a further optional implementation, the target ambient light parameters are determined based on the target control intent, confidence level, and a pre-built ambient light parameter mapping library, specifically including: S1051. Based on the target control intent, match and obtain multiple sets of corresponding candidate ambient light parameters in the ambient light parameter mapping library, and determine the priority of each candidate ambient light parameter. S1052. When the confidence level is greater than or equal to the preset first threshold, the candidate ambient light parameter with the highest priority is determined as the target ambient light parameter. S1053. When the confidence level is greater than or equal to the preset second threshold and less than the first threshold, obtain the historical ambient light parameters preferred by drivers and passengers, optimize the candidate ambient light parameters based on the historical ambient light parameters, and obtain the target ambient light parameters. S1054. When the confidence level is less than the second threshold, the candidate ambient light parameters are displayed through the vehicle system, and the target ambient light parameters are determined based on the driver's and passengers' selections.

[0050] Specifically, embodiments of the present invention pre-establish a dedicated "intent-lighting parameter" mapping library for automotive scenarios, dynamically matching parameters based on confidence levels and user preferences. The mapping library design covers various dimensions of lighting parameters, including color temperature, brightness, light strip area, and flicker frequency. For example, the lighting parameters corresponding to "emotional agitation adjustment" could be "color temperature: 5000K, brightness: 40%, light strip area: full vehicle light strip, flicker frequency: none." It should be noted that embodiments of the present invention design at least two sets of lighting parameters (with different priorities) as alternatives for each type of control intent.

[0051] After matching multiple sets of candidate ambient light parameters in the ambient light parameter mapping library according to the target control intent, different measures are taken according to the confidence level of the target control intent: 1) Confidence level ≥ 90%: Use the highest priority candidate ambient light parameter as the target ambient light parameter; 2) 70% ≤ confidence level < 90%: Combine user historical preferences (e.g., users prefer 5200K color temperature when they are irritable in the past) to match multiple sets of candidate ambient light parameters, and optimize and adjust the candidate ambient light parameters with the highest matching degree (weighted summation) based on user historical preferences to finally obtain the target ambient light parameters; 3) Confidence level <70%: Multiple sets of candidate ambient light parameters (such as "annoying mode A / B / C") are displayed on the vehicle's infotainment system. The driver selects the desired mode using the steering wheel shortcut keys, and the priority of the candidate ambient light parameters in the mapping library is updated based on the driver's selection.

[0052] The target ambient light parameters are converted into LIN bus control commands and sent to the ambient light controller to adjust the lighting status in real time; at the same time, the actual status information of the ambient light (such as current brightness and color temperature) is collected to facilitate deviation detection and optimization.

[0053] Deviation comparison: The difference between the actual state and the command parameters is calculated to obtain the deviation information (e.g., command brightness 40%, actual brightness 45% → deviation +5%). Feedback Adjustment: If the deviation is greater than 5% or the user manually adjusts the lights within 1 minute, the control command will be automatically corrected (such as reducing the brightness to 35%), and the current data (interaction, emotion, and scene) will be added to the model training samples to update the ambient light control intent recognition model and mapping library.

[0054] The following example, using the scenario of "drivers becoming irritable in congested urban traffic," details the implementation process: 1) Data collection: Time 18:30 (evening rush hour), GPS showed urban roads, ambient light index 30 (evening); CAN bus collected vehicle speed 20km / h, braking frequency 8 times / minute (congested road conditions), driving mode was "Comfort"; DMS collected driver's gaze at door panel light strip (eye movement data), facial expression frowning (irritability), heart rate sensor collected heart rate 95 beats / min (higher than the daily average of 80 beats / min); no voice input, gesture sensor did not capture any movement.

[0055] 2) Data preprocessing: Interactive data: gaze point mapped to door panel light strip coordinates → attention information "focus on ambient light"; Emotional data: Frowning + high heart rate → Regression model outputs "irritability", predicts demand as "low-stimulation light"; Scene data: Congestion + Evening → Labeled as "Urban Congestion Evening Scene".

[0056] 3) Intent recognition: The multimodal model takes "attention information + irritability + congested scene, comfort mode" as input and outputs "reduce brightness + cool color intent" with a confidence level of 83%.

[0057] 4) Parameter matching: The mapping library's "Emotional Irritability Adjustment" parameters (color temperature 5000K, brightness 40%, full vehicle light strip) were called; combined with the user's historical preferences (brightness was adjusted to 38% when irritable in traffic jams in the past), the final parameters were determined as follows: color temperature 5000K, brightness 39%, full vehicle light strip.

[0058] 5) Command execution and feedback: The control command is sent to the ambient light controller via the LIN bus, and the light switches to a cool white low-brightness mode; after 1 minute, the status is collected: the actual brightness is 38% (no deviation), the driver does not manually adjust, and the heart rate drops to 85 beats / min → the control is deemed effective, and "congestion + irritability + 5000K + 39%" is added to the mapping library. At the same time, the ambient light control intent recognition model is optimized based on the current interaction, emotion, and scene data.

[0059] The method steps of the embodiments of the present invention have been described above. It can be understood that the embodiments of the present invention identify the gaze point information and facial expression information of the driver and passengers based on facial image information, determine interaction feature data based on gaze point information and interaction information, determine emotional feature data based on facial expression information and physiological state information, and determine scene feature data based on driving state information and external environment information. Based on a pre-trained ambient lighting control intention recognition model, control intention is recognized from three branches: interaction features, emotional features, and scene features. Target ambient lighting parameters are determined based on the identified target control intention, the corresponding confidence level, and a pre-built ambient lighting parameter mapping library. This improves the efficiency and accuracy of vehicle ambient lighting control, achieves refined and personalized ambient lighting control, and enhances the user's driving experience.

[0060] Reference Figure 2 This invention provides a vehicle ambient lighting control device based on intent recognition, comprising: The information acquisition module is used to acquire facial image information, interaction information, physiological status information of drivers and passengers, as well as current vehicle driving status information and external environment information; The information recognition module is used to identify the gaze point and facial expression information of the driver and passengers based on facial image information; The feature determination module is used to determine interaction feature data based on gaze point information and interaction information, determine emotion feature data based on facial expression information and physiological state information, and determine scene feature data based on driving state information and external environment information. The intent recognition module is used to input interaction feature data, emotion feature data and scene feature data into a pre-trained ambient light control intent recognition model to obtain the target control intent and the corresponding confidence level. The ambient lighting control module is used to determine the target ambient lighting parameters based on the target control intent, confidence level, and a pre-built ambient lighting parameter mapping library, and to control the ambient lighting of the current vehicle based on the target ambient lighting parameters.

[0061] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0062] Reference Figure 3 This invention provides an electronic device, comprising: At least one processor; At least one memory for storing at least one program; When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned vehicle ambient lighting control method based on intent recognition.

[0063] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0064] This invention also provides a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the aforementioned vehicle ambient lighting control method based on intent recognition.

[0065] This invention provides a computer-readable storage medium that can execute a vehicle ambient lighting control method based on intent recognition provided in the method embodiments of this invention. It can execute any combination of the implementation steps of the method embodiments and has the corresponding functions and beneficial effects of the method.

[0066] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described intention recognition-based vehicle ambient lighting control method.

[0067] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0068] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0069] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0070] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0071] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0072] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0073] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0075] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0076] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0077] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0078] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0079] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for controlling vehicle ambient lighting based on intent recognition, characterized in that, Includes the following steps: Acquire facial image information, interaction information, physiological status information of drivers and passengers, as well as current vehicle driving status information and external environment information; The gaze point and facial expression information of the driver and passengers are obtained based on the facial image information. Interaction feature data is determined based on the gaze point information and the interaction information; emotional feature data is determined based on the facial expression information and the physiological state information; and scene feature data is determined based on the driving state information and the external environment information. The interaction feature data, the emotion feature data, and the scene feature data are input into a pre-trained ambient light control intent recognition model to obtain the target control intent and the corresponding confidence level. The target ambient light parameters are determined based on the target control intent, the confidence level, and the pre-built ambient light parameter mapping library, and the ambient light is controlled on the current vehicle based on the target ambient light parameters.

2. The vehicle ambient lighting control method based on intent recognition according to claim 1, characterized in that, The step of identifying the gaze point information and facial expression information of the driver and passenger based on the facial image information specifically includes: Based on the facial image information, the eye region image and the face region image are extracted; The pupil center position and corneal reflection point position are obtained by locating the pupil center and extracting the corneal reflection point position from the eye region image. The eyeball is modeled based on the pupil center position and the corneal reflection point position to obtain the gaze direction; The gaze direction is mapped to the cockpit space coordinate system to obtain the gaze point information; The facial expression information is obtained by performing expression recognition on the facial image information.

3. The vehicle ambient lighting control method based on intent recognition according to claim 1, characterized in that, The interaction information is either voice interaction information or action interaction information. The step of determining interaction feature data based on the gaze point information and the interaction information specifically includes: Obtain the ambient lighting distribution information of the current vehicle, match the gaze point information with the ambient lighting distribution information, and determine the attention features based on the matching result; The voice interaction information is subjected to speech recognition and keyword extraction to obtain speech features, or the action interaction information is subjected to action recognition and action database matching to obtain action features; The attention features and the speech / action features are temporally aligned and fused to obtain the interaction feature data.

4. The vehicle ambient lighting control method based on intent recognition according to claim 1, characterized in that, The step of determining emotional characteristic data based on the facial expression information and the physiological state information specifically includes: The emotional state of the driver and passengers is identified based on the facial expression information, and the emotional state is vectorized based on the valence-arousal model to obtain a two-dimensional emotion vector. Based on the physiological state information, multidimensional physiological features are determined, and the two-dimensional emotion vector is expanded in dimension based on the multidimensional physiological features to obtain the emotion feature data.

5. The vehicle ambient lighting control method based on intent recognition according to claim 1, characterized in that, The driving status information includes driving mode, vehicle speed, vehicle acceleration, vehicle steering angle, and continuous driving duration. The external environment information includes ambient light intensity, weather conditions, road type, and traffic congestion. The determination of scene feature data based on the driving status information and the external environment information specifically involves: The driving state information and the external environment information are normalized to obtain the scene feature data.

6. The vehicle ambient lighting control method based on intent recognition according to claim 1, characterized in that, The ambient lighting control intent recognition model is trained through the following steps: Acquire interaction feature samples, emotion feature samples, and scene feature samples from historical driving scenarios; Training samples are constructed based on the interaction feature samples, the emotion feature samples, and the scene feature samples, and the control intent labels corresponding to the training samples are determined by manual annotation. A multi-branch convolutional neural network is constructed, which includes an input layer, an interaction branch network, an emotion branch network, a scene branch network, a feature fusion layer, and an output layer. The training samples are input into the input layer, and the interaction feature vector is obtained by extracting features from the interaction feature samples through the interaction branch network. The emotion feature vector is obtained by extracting features from the emotion feature samples through the emotion branch network. The scene feature vector is obtained by extracting features from the feature fusion layer through the scene branch network. The feature fusion layer performs feature fusion on the interaction feature vector, the emotion feature vector, and the scene feature vector based on an attention mechanism to obtain a fused feature vector. The output layer maps the fused feature vector into a control intent recognition result. The loss value is determined based on the control intent recognition result and the corresponding control intent label; The parameters of the multi-branch convolutional neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained ambient light control intention recognition model.

7. A vehicle ambient lighting control method based on intent recognition according to any one of claims 1 to 6, characterized in that, The step of determining the target ambient light parameters based on the target control intent, the confidence level, and a pre-built ambient light parameter mapping library specifically includes: According to the target control intent, multiple sets of candidate ambient light parameters are matched in the ambient light parameter mapping library, and the priority of each candidate ambient light parameter is determined. When the confidence level is greater than or equal to a preset first threshold, the candidate ambient light parameter with the highest priority is determined as the target ambient light parameter. When the confidence level is greater than or equal to a preset second threshold and less than the first threshold, the historical ambient lighting parameters preferred by the driver and passengers are obtained, and the candidate ambient lighting parameters are optimized based on the historical ambient lighting parameters to obtain the target ambient lighting parameters. When the confidence level is less than the second threshold, the candidate ambient light parameters are displayed through the vehicle system, and the target ambient light parameters are determined based on the driver's and passenger's selection.

8. A vehicle ambient lighting control device based on intent recognition, characterized in that, include: The information acquisition module is used to acquire facial image information, interaction information, physiological status information of drivers and passengers, as well as current vehicle driving status information and external environment information; The information recognition module is used to identify the gaze point information and facial expression information of the driver and passenger based on the facial image information; The feature determination module is used to determine interaction feature data based on the gaze point information and the interaction information, determine emotion feature data based on the facial expression information and the physiological state information, and determine scene feature data based on the driving state information and the external environment information. The intent recognition module is used to input the interaction feature data, the emotion feature data and the scene feature data into a pre-trained ambient light control intent recognition model to obtain the target control intent and the corresponding confidence level. An ambient lighting control module is used to determine target ambient lighting parameters based on the target control intent, the confidence level, and a pre-built ambient lighting parameter mapping library, and to control the ambient lighting of the current vehicle based on the target ambient lighting parameters.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a vehicle ambient lighting control method based on intent recognition as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a vehicle ambient lighting control method based on intent recognition as described in any one of claims 1 to 7.

Images