Control system of three-dimensional interactive automobile atmosphere lamp

Abstract

The invention relates to the technical field of automobile electronic control, discloses a control system of a three-dimensional interactive automobile atmosphere lamp, and aims at solving the problems that an existing atmosphere lamp is single in function, insufficient in space sense, poor in adaptability and poor in user experience. The system comprises a multi-mode sensing unit which collects environment, operation, passenger behavior and interaction instruction data in real time; the data fusion and situation analysis unit is used for preprocessing the fused data, identifying a prediction situation and generating a situation representation vector; the interaction intention recognition unit is used for analyzing gesture, voice and sight line information of the passenger and recognizing an interaction intention; the three-dimensional light field generation engine is used for dynamically generating a three-dimensional light effect scheme based on the situation representation and the interaction intention; and the distributed light-operated execution network drives the light units to realize a lighting effect scheme. According to the scheme, intelligent dynamic and humanized control over the light environment is achieved, and system intelligence, interaction experience and driving and riding comfort and safety are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of automobile electronic control, and particularly relates to a control system of a three-dimensional interactive automobile atmosphere lamp. BACKGROUND

[0002] With the rapid development of the automobile industry and the increasing demand of consumers for driving experience, the vehicle-mounted electronic system and interior design have become important indicators for measuring the competitiveness of automobile products. In creating a personalized and immersive driving environment, the lighting technology in the automobile interior, especially the atmosphere lamp system, plays an increasingly important role. Through the artistic use of light and shadow, it not only enhances the aesthetics of the car interior, but also effectively adjusts the mood and comfort of the driver and passenger, and is an indispensable part of the modern intelligent cockpit.

[0003] Among them, the automobile atmosphere lamp, as a core component of the visual experience inside the vehicle, aims to adjust the atmosphere inside the vehicle, improve comfort and create a sense of exclusivity through light changes. Early atmosphere lamps mainly used fixed colors or simple cycle modes, but with the development of human-computer interaction technology and digital imaging technology, consumers have higher requirements for the dynamic, immersive and personalized interactive capabilities of atmosphere lamps, expecting them to provide a more in-depth and participatory light experience.

[0004] In the prior art, traditional automobile atmosphere lamp systems generally have limitations. They are mostly two-dimensional planar light effects, which are difficult to simulate the rich light changes in real three-dimensional space and cannot create an immersive visual experience with a sense of hierarchy and depth. Existing interaction methods are often simple and non-intuitive, such as switching modes through preset buttons or basic linkage with a small number of vehicle states, lacking deep, multi-modal, real-time, and natural interaction capabilities with drivers and passengers. In addition, existing control systems lack the ability to handle complex multi-source light information, real-time perception of user intent, and coordination of multi-dimensional light effect presentation, making it difficult to support complex three-dimensional light field construction and smooth interaction logic implementation, resulting in monotonous light effects, delayed or mismatched interaction feedback. These deficiencies seriously restrict the potential of automobile atmosphere lamps in improving driving experience, personalized customization, and safety, and urgent technical problems need to be solved. SUMMARY

[0005] To solve the technical problems of existing automobile atmosphere lamp systems, such as single function, lack of spatial sense and dynamic interaction, inability to intelligently perceive the environment inside and outside the vehicle and the state of the passengers for adaptive adjustment, and low system integration leading to poor user experience, the present application provides a control system for a three-dimensional interactive automobile atmosphere lamp. The system uses multi-modal data perception, context intelligent analysis, three-dimensional light field generation, and distributed collaborative control to provide an immersive, personalized, adaptive, and highly interactive automobile atmosphere lamp experience, thereby realizing the intelligent, dynamic, and humanized control of the light environment inside the vehicle.

[0006] According to one aspect of the present application, a control system of a stereoscopic interactive automobile atmosphere lamp is provided, which comprises: a multi-modal perception unit, configured to collect in real time data of an internal and external environment of the vehicle, data of a running state of the vehicle, data of physiological behaviors of passengers, and data of interactive instructions of the passengers; a data fusion and context analysis unit, electrically connected to the multi-modal perception unit, configured to pre-process, extract features, multi-scale fuse, identify contexts, and predict trends of various heterogeneous data collected by the multi-modal perception unit, so as to generate a representation vector of a current in-vehicle context; an interactive intention recognition unit, electrically connected to the multi-modal perception unit, configured to analyze gesture input, voice input, and gaze tracking information of the passengers, so as to identify accurate interactive intentions of the passengers; a stereoscopic light field generation engine, electrically connected to the data fusion and context analysis unit and the interactive intention recognition unit, configured to dynamically plan and generate a three-dimensional light effect scheme according to the representation vector of the current in-vehicle context and the accurate interactive intentions of the passengers, the three-dimensional light effect scheme including color parameters, brightness parameters, dynamic trajectory parameters, spatial distribution parameters, and timing change parameters of each light execution unit; a distributed light control execution network, electrically connected to the stereoscopic light field generation engine, configured to receive the three-dimensional light effect scheme and accurately drive a plurality of independent light execution units, so as to realize the three-dimensional light effect scheme in a vehicle space.

[0007] As an embodiment of the present application, the multi-modal perception unit specifically comprises: an ambient light sensor array, configured to acquire in real time ambient light intensity and color temperature data inside and outside the vehicle; an in-vehicle visual sensor system, configured to capture facial expressions, body postures, gesture information, and gaze directions of the passengers; an in-vehicle voice recognition module, configured to capture voice instructions and voice emotional features of the passengers; a vehicle state sensor interface, configured to receive vehicle speed, steering angle, throttle opening, brake pedal pressure, gear position, wiper state, and door switch state data from a vehicle controller area network; an external environment sensor interface, configured to receive external temperature, humidity, rainfall intensity, geographical position, and time data; a biological physiological sensor, configured to acquire electrodermal response, heart rate, and respiration rate data of the passengers.

[0008] As an embodiment of the present application, the ambient light sensor array comprises a plurality of photoresistors, which are distributed on a vehicle roof, an instrument panel, and edges of vehicle windows.

[0009] As an embodiment of the present application, the in-vehicle visual sensor system comprises a plurality of high-resolution image sensors and a depth sensor, the image sensors and the depth sensor are integrated on the roof, A-pillar and B-pillar, the depth sensor adopts a structured light or time-of-flight technology to obtain the three-dimensional spatial coordinates of the passenger gesture.

[0010] As an embodiment of the present application, the in-vehicle voice recognition module comprises a microphone array, the microphone array is integrated in the roof or the steering wheel.

[0011] As an embodiment of the present application, the biological physiological sensor is integrated inside the seat or the steering wheel.

[0012] As an embodiment of the present application, the data fusion and context analysis unit specifically comprises: a data preprocessing module for time synchronization, spatial calibration, noise filtering and missing value filling of various types of data collected by the multi-modal perception unit; a feature extraction module for extracting visual features, voice features, vehicle state features and ambient light features from the preprocessed data; a multi-modal data fusion module for fusing the heterogeneous features to construct a unified context representation vector; a context recognition and prediction module for recognizing the current in-vehicle context and predicting the context change trend based on the context representation vector. As an embodiment of the present application, the feature extraction module extracts the visual features using a convolutional neural network, extracts the voice semantic features and emotion features using a recurrent neural network, and extracts the driving behavior pattern features using time series analysis.

[0013] As an embodiment of the present application, the multi-modal data fusion module adopts an attention mechanism or a Kalman filter technology.

[0014] As an embodiment of the present application, the context recognition and prediction module adopts a deep learning model, the deep learning model is a long short-term memory network or a Transformer model, the deep learning model inputs the context representation vector and outputs the context probability distribution of the future preset time step.

[0015] As an embodiment of the present application, the interaction intention recognition unit specifically comprises: a gesture recognition algorithm module for recognizing the passenger gesture posture sequence and mapping to a predefined gesture instruction; a voice semantic understanding algorithm module for analyzing the passenger voice instruction to extract the key intention and parameters; A line-of-sight tracking and gaze point recognition module is configured to calculate the line-of-sight direction of the occupant and determine the attention area of the occupant according to the visual sensor system data. An intention fusion module is configured to fuse the gesture instruction, the voice instruction, and the line-of-sight tracking information to determine the precise interaction intention of the occupant.

[0016] As an embodiment of the present application, the gesture recognition algorithm module adopts a three-dimensional convolutional neural network model.

[0017] As an embodiment of the present application, the voice semantic understanding algorithm module adopts a natural language processing model.

[0018] As an embodiment of the present application, the light field generation engine specifically includes: A light effect library management module is configured to store preset light modes, color schemes, dynamic trajectories, and particle effects. A dynamic light effect generation module is configured to dynamically generate a three-dimensional light effect sequence including color, brightness, direction, diffusion angle, spatial position change, and timing change parameters according to the context representation vector and the precise interaction intention of the occupant, in combination with the light effect library. A light effect parameter optimization module is configured to adjust the parameters of the dynamic light effect generation module according to user feedback to optimize the user experience of the light effect.

[0019] As an embodiment of the present application, the dynamic light effect generation module includes a rule-based generation module and a learning-based generation module. The rule-based generation module triggers a preset light effect according to the context recognition result, and the learning-based generation module generates a light effect sequence adapted to the context and the intention using a generative adversarial network or a variational autoencoder model. The dynamic light effect generation module internally maintains a three-dimensional model of the in-vehicle space and maps the light effect to specific areas and curved surfaces in the model.

[0020] As an embodiment of the present application, the light propagation simulation function of the dynamic light effect generation module calculates the reflection, scattering, and absorption of light in the in-vehicle environment in real time.

[0021] As an embodiment of the present application, the dynamic light effect generation module generates a motion trajectory of a light point in the three-dimensional space in the vehicle based on a Bezier curve or a spline curve.

[0022] As an embodiment of the present application, the distributed light control execution network specifically includes: A master controller is configured to receive the three-dimensional light effect scheme and distribute it to multiple regional sub-controllers. a region sub-controller array electrically connected to the main controller, each sub-controller being responsible for controlling a plurality of light execution units of a specific region; a light execution unit array electrically connected to the region sub-controller array, for realizing light effect output; a power management module for providing stable and adjustable power supply for the light execution unit array.

[0023] As an embodiment of the present application, each sub-controller of the region sub-controller array is a microcontroller, the light execution unit array includes an addressable multi-color light emitting diode array, a micro projection unit and a variable focus light beam projector, the addressable multi-color light emitting diode array is integrated in an optical fiber, a light guide plate or a flexible light emitting diode strip, the micro projection unit is integrated inside a ceiling or a door panel, and the variable focus light beam projector can adjust the diffusion angle and direction of the light beam.

[0024] As an embodiment of the present application, the main controller and the region sub-controller array adopt Ethernet or a dedicated serial bus communication protocol.

[0025] Compared with the prior art, the present application has the advantages and positive effects that: The present application provides a control system of a three-dimensional interactive automobile atmosphere lamp, which realizes intelligent, dynamic and humanized control of the light environment inside the automobile, effectively solving the technical problems of single function of the atmosphere lamp, lack of spatial sense and dynamic interaction, inability to intelligently perceive the environment inside and outside the vehicle and the state of the passengers for adaptive adjustment, and low system integration in the prior art.

[0026] Specifically, the present application realizes comprehensive and deep perception of the situation inside the vehicle by collecting multi-source heterogeneous data such as the environment inside and outside the vehicle, vehicle operation and physiological behavior of the passengers in real time through a multi-modal perception unit. The data fusion and situation analysis unit can accurately identify the current driving situation and the emotional state of the passengers and predict future trends by using advanced data processing and deep learning algorithms, thereby providing a solid foundation for intelligent generation of light effects.

[0027] The interactive intention recognition unit supports multi-modal natural interaction such as gestures, voice and gaze, breaking through the limitations of traditional keys or touch screens, so that the passengers can interact more intuitively and naturally, greatly improving the fluency and immersion of human-computer interaction.

[0028] The stereoscopic light field generation engine is the core innovation point of the present application, which can not only control color and brightness, but also dynamically generate light and shadow effects with direction, diffusion and trajectory changes in the three-dimensional space in the vehicle. By integrating the light effect library and the learning-based generation algorithm, the system can intelligently plan and output highly personalized three-dimensional light effect schemes that meet the current atmosphere according to complex situations and interaction intentions, realizing truly three-dimensional and spatialized atmosphere lighting effects.

[0029] The distributed light control execution network is composed of a master controller, an array of regional sub-controllers and an array of various light execution units, ensuring real-time transmission and precise synchronous execution of light effect instructions. The addressable multi-color light-emitting diodes, micro projection units and variable focus beam projectors and other types of light execution units contained therein work together to create an unprecedented immersive, dynamically changing and hierarchically rich three-dimensional light environment in the vehicle.

[0030] In summary, through the innovative design at the system level, the present application realizes the perfect combination of multi-dimensional perception, intelligent situation understanding, personalized light effect generation and refined light field control, significantly improving the intelligent level, interactive experience and the comfort and safety of passengers and drivers of the automobile atmosphere lamp system, and providing a revolutionary solution for the intelligent upgrading of future automobile cabins. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is the overall technical scheme architecture schematic diagram of the control system of the stereoscopic interactive automobile atmosphere lamp proposed by the present application; Figure 2 is the core principle framework schematic diagram of the stereoscopic light field generation engine in the present application; Figure 3 is the core principle framework schematic diagram of multi-modal data perception in the present application; Figure 4 is the core principle framework schematic diagram of data fusion and situation analysis in the present application; Figure 5 is the core principle framework schematic diagram of interaction intention recognition in the present application; Figure 6 is the logic flow framework diagram of the distributed light control execution network in the present application; DETAILED DESCRIPTION

[0032] In the following description, specific terminology and descriptions are used for the sake of convenience in setting forth the embodiments of the present application. However, the application is not limited to the specific details described and it will be apparent to those skilled in the art that various modifications and substitutions can be made without departing from the spirit and scope of the application. The drawings are not to scale and are solely for purposes of illustration in accordance with the embodiments of the present application. The present application proposes a control system for a stereoscopic interactive automotive ambient light. Traditional automotive ambient light systems are usually limited to a single preset color or a simple cycling mode, with relatively static lighting effects that cannot be dynamically adjusted according to the in-vehicle environment, occupant state, and interactive intent. The prior art lacks the ability to deeply perceive and fuse multi-modal data, making it difficult to achieve truly contextual, personalized, and stereoscopic light field effects. This limitation results in ambient lights only serving as a background decoration, failing to fully realize their potential in enhancing driving experience, creating a comfortable in-vehicle space, and assisting human-machine interaction. To address the above technical problems, the present application aims to provide a control system that can perceive multi-modal data in real time, deeply analyze the context and user intent, and generate and execute stereoscopic light field effects accordingly, thereby achieving high intelligence, personalization, and interactivity of automotive ambient lights.

[0033] The control system for a stereoscopic interactive automotive ambient light proposed by the present application has an overall architecture that includes a multi-modal data perception module, a data fusion and context analysis module, an interactive intent recognition module, a stereoscopic light field generation engine, and a distributed light control execution network. These modules work together to achieve complete closed-loop control from environmental perception to light effect output.

[0034] The multi-modal data perception module is responsible for real-time collection of in-vehicle and out-vehicle environment, occupant state, and vehicle operation data. This module integrates multiple sensor units internally. Specifically, it contains a sound sensor array for capturing voice commands, environmental noise levels, and occupant's voice emotional features in the vehicle environment. The sound sensor array is usually composed of multiple high-sensitivity microphones, for example, installed on the roof, steering wheel, and seat backrest, etc. to achieve all-around sound collection. These microphones perform preliminary time and frequency domain analysis through a digital signal processor, for example, noise reduction processing of voice signals, and convert raw audio data into feature vectors. Secondly, the module also includes a visual sensor unit, such as a high-definition camera and an infrared camera installed inside the vehicle, for monitoring the occupant's facial expressions, gaze direction, body movements, and in-vehicle lighting conditions. The image data of the visual sensor unit is analyzed in real time by an image processing unit, for example, using a convolutional neural network model to identify the emotional state of the occupant, or through a skeleton tracking algorithm to identify the occupant's gesture instructions. Furthermore, the module contains a tactile sensor array, such as pressure sensors and touch sensors integrated in the steering wheel, seat surface, and door armrest, for sensing the occupant's physical contact, pressing force, and sliding gestures. The raw data of the tactile sensor is sampled and preliminarily processed by a microcontroller to identify the type and duration of touch events. In addition, the module also obtains vehicle operation state data through a vehicle bus interface, such as vehicle speed, steering angle, acceleration or deceleration amplitude, brake state, parking state, door opening and closing state, and external environment light intensity. The vehicle bus interface is responsible for parsing these digital signals according to the vehicle bus communication protocol, such as the Controller Area Network protocol, and converting them into standardized data formats. In order to ensure the accuracy and timeliness of the data, all sensor data are time-stamped synchronized, and their integrity is verified by data checking algorithms such as cyclic redundancy check code, and transmitted to the upstream module at a preset sampling frequency, for example, fifty times per second.

[0035] The data fusion and context analysis module receives multiple heterogeneous data streams from the multi-modal data perception module and performs deep fusion and context reasoning through advanced algorithms. This module internally contains a data preprocessing unit, a feature extraction unit, a multi-modal fusion unit, and a context reasoning unit. The data preprocessing unit first cleans, normalizes, and aligns the received sensor data. For example, linear interpolation or cubic spline interpolation is used for resampling data with different sampling rates to ensure that all data is processed on a unified time dimension. Missing values are filled and outliers are filtered. The feature extraction unit extracts high-level semantic features from the preprocessed data. For example, speech emotion features, volume size features, and speech rate features are extracted from sound data; facial expression features, pupil state features, gesture type features, and in-vehicle passenger number features are extracted from visual data; press intensity features, touch area features, and gesture trajectory features are extracted from tactile data; driving mode features such as sport mode, comfort mode, and economy mode, and driving environment features such as high-speed driving, urban congestion, and parking are extracted from vehicle bus data. The multi-modal fusion unit deeply fuses the features extracted from different modalities. The fusion process uses a deep neural network model based on an attention mechanism that dynamically assigns weights to features from different modalities to capture the interaction and complementary information between modalities. For example, when the occupant issues a voice command, the weight of the sound modality increases; when the occupant makes a gesture, the weights of the visual and tactile modalities increase. The fused feature vector carries comprehensive information about the in-vehicle environment and occupant state. The context reasoning unit, based on the fused feature vector, combines pre-set context models to infer the current in-vehicle context in real time. Context models may include driving contexts such as normal driving, sudden acceleration, sudden braking, and turning, occupant contexts such as fatigue driving, joy, tension, and concentration, and environmental contexts such as day, night, sunny, and rainy. Context reasoning uses sequence modeling techniques such as Hidden Markov Model or Long Short-Term Memory Network to identify the dynamic evolution of the context, such as transitioning from "static waiting" to "slow start" and then to "high-speed driving". The output of this module is a high-dimensional context vector that accurately describes various aspects of the current in-vehicle environment. The interactive intent recognition module receives the context vector output from the context analysis module and direct user interaction signals such as voice command text and gesture recognition results to accurately determine the occupant's interactive intent. This module internally contains an intent classification unit and an instruction analysis unit. The intent classification unit uses a multi-classifier model based on deep learning to process the received information and identify the occupant's active or passive interactive intent. For example, when the occupant says "adjust to comfortable mode", the intent classifier classifies it as "ambience mode switching" intent; when the system detects the occupant's fatigue state and combines it with the night driving context, the intent classifier will infer the "refresh alert" passive intent.The classifier model learns the complex mapping between situational information and user intent by training on a large amount of labeled data. The instruction parsing unit further extracts specific instruction parameters from the identified intent. For example, in the "ambience mode switching" intent, the specific parameter "relaxing mode" is parsed; in the "adjust brightness" intent, the percentage of brightness increase or decrease is parsed. The instruction parsing unit usually adopts a combination of rule-based parsers and natural language understanding techniques to handle instructions of different granularities. To improve the accuracy of recognition, the module also introduces a confidence evaluation mechanism. Each identified intent is accompanied by a confidence score, indicating the reliability of the recognition result. When the confidence is below a preset threshold, the system may trigger a fuzzy intent confirmation mechanism, such as prompting the occupant through the screen or asking through voice, or adopt a default conservative strategy. The output of this module is a structured interactive intent instruction, containing intent type, specific parameters and confidence, for the use of the light field generation engine.

[0036] The stereoscopic light field generation engine is the core of the system, responsible for generating specific three-dimensional light field rendering schemes according to interactive intent instructions and context information. The module internally contains a light field model library, a light field rendering unit, a light field optimization unit, and a light effect sequence generation unit. The light field model library stores pre-set light field templates for various atmosphere modes, such as starry sky mode, forest mode, ocean mode, and parameterized models of basic light effect elements such as point light sources, line light sources, and surface light sources. Each template defines the basic spatial distribution, color, brightness range, and dynamic variation law of the light field. The light field rendering unit selects one or more matching light field templates from the light field model library as the basis according to the received interactive intent instructions and context vectors, and dynamically adjusts the template parameters according to instruction parameters such as brightness adjustment values, color selection, and context information such as passenger emotions and vehicle speed. For example, when the passenger selects "relaxing mode" and the context shows that the passenger is in a happy mood, the rendering unit may generate warm-toned, slow-changing flowing light effects; when the system detects a fatigue driving context, it may generate cold-toned, high-contrast, rapidly flickering warning light effects. The light field rendering unit can calculate the three-dimensional spatial coordinates, color, intensity, and transparency of each light path, constructing a discrete three-dimensional light field data structure such as a voxel array or a point cloud. The light field optimization unit optimizes the initially generated light field data to ensure the smoothness, consistency, and safety of the visual effect. The optimization process takes into account the reflection, refraction, and scattering effects of light on various material surfaces in the vehicle, and simulates the perception characteristics of the human eye, such as brightness adaptability. In addition, safety constraint optimization ensures that the light effect does not cause glare or interference for the driver, for example, the light intensity in the driver's forward view area is limited within a safe range. The light effect sequence generation unit converts the optimized three-dimensional light field data into light effect sequence instructions that can be understood and executed by the distributed light control execution network. This unit slices the light field in the time dimension and encodes the light field data in each time slice to generate a series of time-synchronized light effect control frames. These frames contain the color, brightness, and dynamic effect parameters that each independent light-emitting unit such as a light-emitting diode or a laser projection unit should present at a specific time. The light field generation formula can be expressed as: L(p, t) = ∑i=1Nlightαi(I, C)⋅G(p, pi, di, λi, S(I, C, K, M)) where L(p, t) represents the light field radiance at spatial position p and time t. Nlight is the total number of light-emitting units. αi(I, C) is the intensity and color weight of the i-th light-emitting unit, which depends on the current interactive intent I and context information C. G(p, pi, di, λi, S) is a light function that describes the i-th light-emitting unit at spatial position pi with direction di and wavelength λi, and is modulated by dynamic effect parameters S. The dynamic effect parameters S are influenced by the interactive intent I, context information C, pre-set light field model K, and vehicle interior material characteristics M.This formula describes how a light field volume is dynamically generated from a variety of inputs without departing from the spirit and scope of the invention.

[0037] The distributed light control execution network is responsible for receiving the light effect sequence instructions output by the light field volume generation engine and converting these instructions into actual light effects. This module includes a network communication unit, a light driving unit, and a self-checking and feedback unit. The network communication unit is responsible for efficient and reliable data transmission between various independent light modules in the system. The network uses a high-bandwidth, low-latency real-time communication protocol, such as a vehicle Ethernet or a dedicated synchronous bus protocol. The light effect sequence instructions are broken down into independent control packets for each light module and sent to the corresponding light driving unit through the network communication unit. To ensure light effect synchronization, all control packets carry precise timestamps, and the network communication unit supports hardware-level time synchronization, such as the IEEE 802.1AS standard. The light driving unit directly controls each light-emitting unit. Each light driving unit is typically composed of a microcontroller, a power management module, and a driving circuit. The microcontroller receives control packets from the network communication unit and accurately adjusts the current or voltage of the light-emitting diode array, laser projector, or flexible light strip it controls based on instruction parameters such as color value, brightness level, and dynamic mode, thereby achieving the specified color and brightness. For example, for a light-emitting diode lamp bead with four channels of red, green, blue, and white, the driving unit will accurately control the corresponding pulse width modulation signals based on the received brightness values of each channel to produce the required color and brightness. The self-checking and feedback unit monitors the working state of each light module in real time, including power voltage, current, temperature, and the health status of the light-emitting unit. When any abnormalities are detected, such as a damaged lamp bead or overheated driving circuit, the self-checking and feedback unit will report the abnormal information to the upper-layer control system through the network communication unit for fault diagnosis and handling, such as degraded operation or starting a backup module. At the same time, this unit also provides feedback data of the actual output effect of the light, such as measuring the actual brightness through a light sensor, for adaptive calibration and optimization of the system to ensure the consistency of the light effect. The distributed network design ensures the high scalability, robustness, and real-time response capability of the system, and even if some light modules fail, it will not affect the normal operation of the entire system. The working method of the system includes the following steps: First, real-time multi-modal data perception and preprocessing are performed. This step aims to continuously and accurately obtain various dynamic information inside and outside the vehicle. The system collects data through various sensors integrated inside and outside the vehicle. For example, an array of sound sensors captures real-time voice commands, conversation content, and ambient noise levels of passengers inside the vehicle; a visual sensor unit such as a high-definition camera and an infrared camera continuously captures facial expressions, gaze directions, and hand gestures of passengers inside the vehicle, and monitors the lighting conditions inside the vehicle; an array of tactile sensors such as pressure sensors integrated into the steering wheel and seat perceive the touch behavior and body posture of passengers. At the same time, the system continuously obtains the running state data of the vehicle through the vehicle bus interface, such as the current vehicle speed, acceleration, braking state, steering angle, navigation information, and geographical position of the vehicle. All collected raw data will be strictly time-stamped synchronized to ensure the time consistency between different sensor data, and integrity check through data verification algorithm, eliminate the error data in the transmission process. Then, the data is preprocessed, including removing sensor noise, filling missing data, smoothing processing, and normalization. For example, for voice data, acoustic feature extraction is performed, including Mel frequency cepstral coefficient calculation and fundamental frequency identification; for image data, face detection, key point recognition, and background segmentation are performed. The preprocessed data is prepared for transmission to the next processing step in a standardized format, such as a unified sampling frequency and a unified numerical range. The refinement of this step is the basis for subsequent context understanding and intent recognition, directly affecting the intelligent perception ability of the entire system.

[0038] Second, data fusion and context deep analysis are performed. The core of this step is to deeply fuse the pre-processed heterogeneous data from different modalities and infer the semantic context of the current in-vehicle environment. The system first extracts high-order features from the pre-processed data through a feature extraction unit. For example, the emotional intensity and speech rate change rate are extracted from the sound features; the fatigue index, attention concentration, and body activity are extracted from the visual features; the pressing force and touch frequency are extracted from the tactile features; and the driving style such as aggressive, smooth, and leisure is extracted from the vehicle running state. Subsequently, the multi-modal fusion unit inputs these high-order features of different modalities into a deep neural network model based on the cross-attention mechanism. This model can dynamically learn and adjust the importance weights of different modalities and capture the complex interaction between modalities. For example, when the system detects anxiety in the passenger's voice and tense facial expressions in the visual display, the model will strengthen the association between the two modalities and judge that the passenger is in a tense state. The fused feature vector carries more rich and abstract semantic information. On this basis, the context reasoning unit uses a time sequence reasoning model such as a neural network based on a gated recurrent unit to combine the fused feature vector and the pre-set context knowledge graph to infer the current in-vehicle context in real time. These contexts include but are not limited to driving contexts such as urban congestion, high-speed cruising, and parking into the warehouse; passenger contexts such as focused driving, leisure and entertainment, relaxation, and fatigue warning; and external environment contexts such as sunny day, rainy night, and tunnel traversal. The context reasoning model can identify the dynamic conversion and superposition of contexts, such as "nighttime fatigue driving accompanied by relaxing music". The output of this step is a structured context description containing multiple context dimensions and their confidence scores, providing a solid foundation for subsequent interactive intent recognition.

[0039] Again, the occupant interaction intention recognition is performed. This step uses situational information and direct interaction signals from the occupant to accurately determine the occupant's operation intention and the response the system should take. The system first receives the situational description output by the situational analysis module and uses it as background information for intention recognition. At the same time, the system continuously monitors direct interaction signals from the occupant, such as commands given by the occupant through voice, such as "please turn up the ambient light" or "switch to sports mode", or physical operations performed by the occupant through the touch screen, steering wheel buttons, or gesture sensors. The intention classification unit takes the situational information and interaction signals as input and performs real-time analysis through a pre-trained deep classification network. This classification network can identify multiple pre-set interaction intention types, such as "light brightness adjustment", "ambiance mode switching", "color theme selection", "interactive effect triggering", and "safety warning response". For example, when the occupant says "increase brightness by ten percent", the classification network will identify the "light brightness adjustment" intention; when the occupant blinks excessively during night driving, combined with situational analysis, the system may identify the "fatigue warning" passive intention. The command analysis unit follows the intention classification and extracts specific command parameters from the identified intention. For example, for the "light brightness adjustment" intention, the specific brightness increment value such as ten percent is analyzed; for the "ambiance mode switching" intention, the target mode name such as "sports mode" is analyzed. To handle possible ambiguity or uncertainty in the intention, the module introduces a confidence evaluation mechanism. Each identified intention is accompanied by a confidence score indicating the reliability of the recognition result. When the confidence is below a pre-set threshold, the system will not immediately execute the command, but will trigger an intention confirmation process, such as through a voice assistant for confirmation or through a visual interface to prompt the occupant to choose. This step ensures that the system can accurately understand the occupant's true needs and avoid misoperation, thereby improving the smoothness and safety of human-machine interaction. Next, the dynamic generation of the light field is performed. This step constructs and optimizes the three-dimensional light field effect based on the identified interaction intention and the current situation. The system first retrieves the most suitable light field template and basic light effect elements from the light field model library based on the current interaction intention and situation. The light field model library contains multiple pre-set ambiance themes such as romantic starry night, energetic rhythm, and quiet forest, as well as parameterizable light effect primitives such as point light sources, line light sources, and surface light sources with color, brightness, dynamic mode, and spatial distribution information. The light field rendering unit adjusts and parameterizes the selected light field template based on the specific parameters of the intention command, such as the target color, brightness, and dynamic effect type, as well as the situational description, such as the occupant's mood, vehicle speed, and external light intensity. For example, when the "relaxing" intention is recognized and the situation is "night driving", the rendering unit will select soft warm colors and generate a slow-flowing, low-brightness light field effect; when the "sports passion" intention is recognized and the situation is "high-speed driving", it may generate high-contrast, rapidly flashing, and dynamic light beams along the vehicle body lines.This unit is responsible for calculating the specific color, intensity, and transparency of each discrete light point or area in the car in three-dimensional space, and constructing a fine three-dimensional light field data model. The light field optimization unit then performs physical simulation and visual perception optimization on the generated light field. This includes simulating the reflection, refraction, and scattering effects of light on various materials in the car, such as leather, metal, and glass, ensuring that the final light effect is natural and realistic. At the same time, the optimization unit will also adjust according to the perception characteristics of the human eye, such as enhancing the brightness contrast in certain areas to highlight the visual focus, or reducing the brightness in other areas to avoid visual fatigue. In addition, the safety constraint optimization mechanism ensures that the generated light field does not interfere with the driver's line of sight or cause glare, for example, in the area in front of the driver's line of sight, the maximum brightness of the light is strictly limited. The core calculation process of dynamic light field generation can be summarized as the following formula: Vcolor,brightness(p,t)=∑j=1Mtemplatewj(I,C)⋅Tj(p,t,P(I,C)) Where Vcolor,brightness(p,t) represents the color and brightness value of the light field voxel at position p and time t in three-dimensional space. Mtemplate is the total number of templates involved in light field generation. wj(I,C) is the weight coefficient of the jth light field template, which is determined by the current interaction intention I and context information C. Tj(p,t,P(I,C)) represents the light field contribution of the jth light field template at position p and time t, where P(I,C) is a set of template parameters dynamically adjusted according to the interaction intention I and context information C. This formula describes how to combine multiple light field templates according to the weight and adjust their parameters to generate a dynamic light field effect that meets the specific intention and context.

[0040] Finally, distributed light control instruction execution and adaptive adjustment are performed. This stage is responsible for converting the light effect sequence instructions output by the light field generation engine into actual physical light effects and monitoring and feeding back the execution process. The system decomposes the fine light effect sequence instructions generated by the light field generation engine into control packages for each independent light module in the vehicle through the network communication unit. These control packages are transmitted through a high-bandwidth, low-latency real-time communication network such as a dedicated vehicle Ethernet, ensuring that the instructions can reach the target light module within milliseconds. Each control package contains the color, such as red, green, blue, and white four-channel values, brightness level, and dynamic effect parameters that the light module should present within a specific time frame. After receiving the control package, the light drive unit's built-in microcontroller immediately parses the instructions and adjusts the current or voltage of the light-emitting unit it controls, such as a light-emitting diode array or a laser projection module, through precise power management and drive circuits such as pulse width modulation circuits, to achieve the specified color, brightness, and dynamic changes. To ensure the visual synchronization and smoothness of the entire in-vehicle light field, the system uses a hardware-level time synchronization mechanism, such as nanosecond-level time synchronization through the Network Time Protocol, to ensure that all distributed light modules respond to light effect instructions at the same time. In addition, the self-checking and feedback unit continuously monitors the working state of each light module. This includes real-time collection of module power consumption data, working temperature, and actual light output parameters of each light-emitting unit. When any abnormal situation is detected, such as drive circuit overload, damage to a certain lamp bead, or light output that does not match the instructions, the self-checking and feedback unit will immediately generate an exception report and report it to the central control unit through the network communication unit. The central control unit performs fault diagnosis according to the feedback information and takes appropriate corrective measures, such as adjusting the drive parameters of the faulty module to compensate for the light effect deviation, or enabling the backup light path. At the same time, the feedback data of the actual light output is also used for adaptive calibration and optimization of the light field generation engine, which dynamically fine-tunes the rendering parameters by learning the light reflection and scattering characteristics in the actual environment, ensuring that the final visual effect is highly consistent with the design intent. This stage ensures the accuracy, stability, and high reliability of the light effect output, and has the ability of self-diagnosis and adaptive optimization.

Claims

1. A control method for a three-dimensional interactive automotive ambient light, characterized in that, include: Real-time acquisition of multimodal data, including in-vehicle and external environmental data, vehicle operating status data, occupant physiological behavior data, and occupant interaction command data; The multimodal data is preprocessed, feature extracted, multi-scale fused, contextualized, and trend predicted to generate a representation vector of the current in-vehicle context; the occupant's gesture input, voice input, and eye-tracking information are analyzed to identify the occupant's precise interaction intent. Based on the representation vector of the current in-vehicle situation and the precise interaction intention of the occupant, a three-dimensional lighting effect scheme is dynamically planned and generated. The three-dimensional lighting effect scheme includes the color parameters, brightness parameters, dynamic trajectory parameters, spatial distribution parameters, and temporal change parameters of each lighting execution unit. The system receives the three-dimensional lighting scheme and precisely drives multiple independent lighting execution units to realize the three-dimensional lighting scheme in the vehicle interior space.

2. The control method for three-dimensional interactive automotive ambient lighting according to claim 1, characterized in that, The real-time acquisition of multimodal data includes: Real-time data on ambient light intensity and color temperature inside and outside the vehicle are acquired through an ambient light sensor array. The vehicle captures occupants' facial expressions, body posture, gestures, and gaze direction through an in-vehicle vision sensor system; it captures occupants' voice commands and emotional characteristics through an in-vehicle voice recognition module; it receives data on vehicle speed, steering angle, throttle opening, brake pedal pressure, gear position, wiper status, and door open / close status through a vehicle status sensor interface; and it receives data on external temperature, humidity, rainfall intensity, geographical location, and time through an external environment sensor interface. Data on occupant skin conductance, heart rate, and respiratory rate are acquired using biophysiological sensors.

3. The control method for three-dimensional interactive automotive ambient lighting according to claim 2, characterized in that, The ambient light sensor array includes multiple photoresistors distributed in the vehicle's roof, dashboard, and window edges; the in-vehicle vision sensor system includes multiple high-resolution image sensors and a depth sensor, integrated with the roof, A-pillar, and B-pillar; the depth sensor uses structured light or time-of-flight technology to acquire the three-dimensional spatial coordinates of occupant gestures; the in-vehicle voice recognition module includes a microphone array integrated in the roof or steering wheel. The biophysiological sensors are integrated inside the seat or steering wheel.

4. The control method for three-dimensional interactive automotive ambient lighting according to claim 1, characterized in that, The process of preprocessing, feature extraction, multi-scale fusion, context recognition, and trend prediction of the multimodal data to generate a representation vector of the current in-vehicle context includes: The multimodal data is subjected to time synchronization, spatial calibration, noise filtering, and missing value imputation; visual features, speech features, vehicle state features, and ambient light features are extracted from the preprocessed data; heterogeneous features are fused to construct a unified context representation vector; the current in-vehicle context is identified based on the context representation vector and the context change trend is predicted.

5. The control method for three-dimensional interactive automotive ambient lighting according to claim 4, characterized in that, The feature extraction employs a convolutional neural network to extract visual features, a recurrent neural network to extract speech semantic and emotional features, and time series analysis to extract driving behavior pattern features. The multimodal data fusion employs an attention mechanism or a Kalman filter technique. The context recognition and prediction employs a deep learning model, which is a long short-term memory network or a Transformer model. The deep learning model takes the context representation vector as input and outputs the context probability distribution for a future preset time step.

6. The control method for three-dimensional interactive automotive ambient lighting according to claim 1, characterized in that, The process of analyzing occupant gesture input, voice input, and eye-tracking information to identify the occupant's precise interaction intentions includes: The system uses a gesture recognition algorithm module to identify occupant gesture sequences and map them to predefined gesture commands; a speech semantic understanding algorithm module to parse occupant speech commands to extract key intentions and parameters; a gaze tracking and fixation recognition module to calculate occupant gaze direction and determine occupant attention area; and an intention fusion module to fuse the gesture commands, speech commands, and gaze tracking information to determine the occupant's precise interaction intention.

7. The control method for three-dimensional interactive automotive ambient lighting according to claim 6, characterized in that, The gesture recognition algorithm module uses a three-dimensional convolutional neural network model; the speech semantic understanding algorithm module uses a natural language processing model.

8. The control method for three-dimensional interactive automotive ambient lighting according to claim 1, characterized in that, The dynamic programming and generation of three-dimensional lighting effect schemes includes: The lighting effect library management module stores preset lighting modes, color schemes, dynamic trajectories, and particle effects; the dynamic lighting effect generation module dynamically generates a three-dimensional lighting effect sequence containing parameters such as color, brightness, direction, diffusion angle, spatial position changes, and temporal changes, based on the context representation vector and the precise interaction intentions of the occupants, combined with the lighting effect library; the lighting effect parameter optimization module adjusts the parameters of the dynamic lighting effect generation module based on user feedback.

9. A control system for a three-dimensional interactive automotive ambient lighting system, characterized in that, include: The multimodal perception unit is used to collect real-time data on the in-vehicle and external environment, vehicle operating status, occupant physiological behavior, and occupant interaction commands. The data fusion and context analysis unit is used to preprocess, extract features, fuse multiple scales, identify contexts and predict trends of the multimodal data collected by the multimodal perception unit to generate a representation vector of the current in-vehicle context. The interaction intent recognition unit is used to analyze the occupant's gesture input, voice input, and eye tracking information to identify the occupant's precise interaction intent; A 3D light field generation engine is used to dynamically plan and generate a 3D lighting effect scheme based on the representation vector of the current in-vehicle environment and the precise interaction intention of the occupant. The 3D lighting effect scheme includes color parameters, brightness parameters, dynamic trajectory parameters, spatial distribution parameters, and temporal change parameters of each lighting execution unit. A distributed light control execution network is used to receive the 3D lighting effect scheme and precisely drive multiple independent lighting execution units to realize the 3D lighting effect scheme in the in-vehicle space.

10. The control system for the three-dimensional interactive automotive ambient lighting according to claim 9, characterized in that, The multimodal sensing unit includes: An ambient light sensor array is used to acquire real-time data on ambient light intensity and color temperature inside and outside the vehicle; an in-vehicle vision sensor system is used to capture occupants' facial expressions, body posture, gestures, and gaze direction; an in-vehicle voice recognition module is used to capture occupants' voice commands and emotional characteristics; a vehicle status sensor interface is used to receive data from the vehicle controller area network on vehicle speed, steering angle, throttle opening, brake pedal pressure, gear position, wiper status, and door open / close status; an external environment sensor interface is used to receive data on external temperature, humidity, rainfall intensity, geographical location, and time; and biophysiological sensors are used to acquire data on occupants' skin conductance, heart rate, and respiratory rate.

Images