Vehicle tail light emotional interaction method and system based on multi-source information fusion and vehicle

Abstract

The application provides a vehicle tail light emotion interaction method and system based on multi-source information fusion and a vehicle, and relates to the technical field of intelligent networked automobile human-computer interaction. The method comprises the following steps: acquiring environment data, space-time data, vehicle data, user biological feature data and outside-vehicle visual scene data; determining whether the interaction triggering condition is met according to the vehicle data; inputting the environment, space-time, biological feature and outside-vehicle scene data into an emotion calculation model when the condition is met, and outputting a continuous emotion vector in a Valence-Arousal two-dimensional emotion coordinate system; inputting the vector and a scene color vector and a semantic label into a pattern generation engine, dynamically generating a tail light display pattern instruction and driving the tail light screen to display. The application realizes the upgrading of the vehicle tail light from the function indication to the situation self-adaptive emotion expression through multi-source situation perception and emotion-driven pattern generation, and improves the intelligent and personalized level of human-vehicle interaction.

Description

Technical Field

[0001] This invention relates to the field of human-machine interaction technology for intelligent connected vehicles, and in particular to a method, system, and vehicle for emotional interaction of vehicle taillights based on multi-source information fusion. Background Technology

[0002] In recent years, with the development of intelligent connected vehicle technology, the demand for interaction between vehicles and pedestrians, other vehicles, and the environment has been increasing. Traditional vehicle taillights only have basic lighting functions such as turning, braking, and position indication. They are single-function, static, and cannot convey more information beyond the driver's intentions. Existing technologies have solutions for displaying preset patterns or animations through taillights, but their displayed content is fixed and lacks the ability to perceive and respond to dynamic situations. They can only execute pre-programmed lighting effects in limited scenarios and cannot provide personalized and adaptive dynamic expressions based on the specific spatiotemporal environment of the vehicle, the user's emotional state, and the semantics of the surrounding environment. Summary of the Invention

[0003] To address the technical problems mentioned above, this invention provides a vehicle taillight emotion interaction method, system, and vehicle based on multi-source information fusion.

[0004] To achieve the above objectives, a first aspect of the present invention provides a vehicle taillight emotion interaction method based on multi-source information fusion, comprising: Acquire multi-source information, including environmental data, spatiotemporal data, vehicle data, user biometric data, and external visual scene data; Determine whether the preset interaction trigger conditions are met based on the vehicle data; When the interaction triggering condition is met, the environmental data, the spatiotemporal data, the user biometric data, and the vehicle exterior visual scene data are input into the emotion computing model. After being fused and processed by the emotion computing model, a continuous emotion vector is output. The continuous emotion vector is an emotion coordinate point representing the current comprehensive situation in the Valence-Arousal two-dimensional emotion coordinate system. The continuous emotion vector and the environmental color vector and scene semantic label in the vehicle exterior visual scene data are input into the pattern generation engine, and the pattern generation engine dynamically generates taillight display pattern instructions. The pattern command is sent to the taillight display controller to drive the taillight screen to display the emotional pattern.

[0005] Furthermore, the pattern generation engine also obtains the user's personalized preference profile as input, and uses the continuous emotion vector, the environmental color vector, the scene semantic label, and the preference vector in the preference profile as conditional inputs to generate the pattern instruction.

[0006] Furthermore, the sentiment computing model outputs the continuous sentiment vector in the following manner: The sentiment components of each input modality are calculated separately, and the sentiment components of each modality are represented as two-dimensional vectors. ,in, For Valence component, For Arousal components; Evaluate the confidence level of each input modality. The fusion weights are calculated based on the confidence levels of each modality. ; Based on the sentiment components of each modality and the fusion weights, a weighted summation method is used to calculate the fused sentiment vector. ,in, The fused emotion vector This refers to the continuous emotion vector.

[0007] Furthermore, the confidence level Based on sensor signal quality score The result obtained by mapping using the Sigmoid function is:

[0008] in, For sensitivity parameters, For bias parameters, Assess the quality of sensor signals; The fusion weight The confidence scores for each mode are obtained by applying Softmax normalization with a temperature coefficient: ,in, This is the temperature coefficient.

[0009] Furthermore, the pattern generation engine adopts a conditional generation model based on continuous emotion vectors, using the continuous emotion coordinates and the environmental color vector as conditional inputs to dynamically generate the visual elements, color schemes, dynamic sequences, and rhythm of change of the pattern. The pattern generation engine also includes an ambient light adaptive color mapping module, which dynamically adjusts the hue, saturation, and brightness of the generated pattern based on the ambient color vector and the current ambient brightness.

[0010] Furthermore, the external visual scene data is acquired and processed in the following ways: Real-time image frames are captured by a rear-facing camera outside the vehicle. The real-time image frames are classified into scenes, and scene semantic labels are output. The real-time image frames are subjected to pixel-level clustering to extract the main color tone set, and the ambient color vector is calculated by weighting the pixel proportions of each cluster. ,in, The pixel percentage of the k-th cluster. Let k be the cluster center; The scene semantic tags and the environment color vectors serve as additional inputs to the pattern generation engine.

[0011] Furthermore, it also includes steps for updating the user's personalized preference model: Constructing context feature vectors The context feature vector is composed of time features, location features, and emotional coordinate features; Establish a situation-preference conditional probability model ,in, For preferred style categories, These are model parameters; When updating model parameters, apply time-decay weights to historical interaction records. ,in, The attenuation rate, For the current time, The time of the k-th historical record; By minimizing the weighted loss function Update the model parameters.

[0012] Furthermore, for new users, a proactive preference detection mechanism displays several candidate styles on the vehicle's infotainment screen for the user to choose from, and the system then selects the preferred style based on the user's choice. Initialize the prior distribution of the preference model:

[0013] in, The total number of candidate styles, This is the smoothing coefficient.

[0014] Furthermore, the environmental data includes at least real-time weather conditions, temperature, air quality index, and public holiday information obtained through vehicle networking services; the spatiotemporal data includes at least precise time, date, season information, and high-precision geographical location obtained through the Global Navigation Satellite System; the vehicle data includes at least vehicle speed, gear position, door status, and vehicle mode obtained through the vehicle bus; and the user biometric data includes at least a driver's facial image obtained through an in-vehicle camera and in-vehicle voice signals obtained through a microphone. The interaction triggering conditions include at least the following: the vehicle is stationary or at extremely low speed for a duration exceeding a first preset threshold, and the vehicle is located in a public road area where light interaction is permitted.

[0015] Furthermore, when the vehicle data indicates that the interaction triggering condition is no longer met, the emotional pattern on the taillight screen is controlled to fade out smoothly and return to the normal lighting mode. During the display of the emotional pattern, when the vehicle speed signal in the vehicle data is detected to trigger emergency braking, the emotional pattern is immediately replaced by a full-screen red warning.

[0016] A second aspect of the present invention provides a vehicle taillight emotion interaction system based on multi-source information fusion, comprising: The multi-source information acquisition module includes a vehicle network communication unit, a positioning and timing unit, an on-board bus interface, an in-vehicle camera and microphone array facing the driver, and an external rear-facing camera, used to collect environmental data, spatiotemporal data, vehicle data, user biometric data, and external visual scene data. The central processing and decision-making module includes a processor and a memory. The memory stores an emotion computing model and a pattern generation engine. The emotion computing model is used to fuse the environmental data, the spatiotemporal data, the user's biometric data, and the vehicle's external visual scene data to output a continuous emotion vector. The continuous emotion vector is an emotion coordinate point representing the current comprehensive situation in a Valence-Arousal two-dimensional emotion coordinate system. The pattern generation engine is used to dynamically generate taillight display pattern instructions based on the continuous emotion vector and the environmental color vector and scene semantic tags in the vehicle's external visual scene data. The intelligent taillight display module includes at least two high-resolution, independently pixel-controlled LED dot matrix displays integrated on both sides of the rear of the vehicle and their display driver controller, used to receive the pattern instructions and display emotional patterns; The system control bus is used to transmit data and control commands between the multi-source information acquisition module, the central processing and decision-making module, and the intelligent taillight display module.

[0017] Furthermore, the memory also stores a user's personalized preference profile, and the pattern generation engine also obtains the user's personalized preference profile as input, and uses the continuous emotion vector, the environmental color vector, the scene semantic label and the preference vector in the preference profile as conditional inputs to generate the pattern instruction.

[0018] Furthermore, the emotion computing model includes: The continuous emotion space modeling module uses the Valence-Arousal two-dimensional emotion coordinate system to perform continuous spatial modeling of emotions, and outputs continuous emotion coordinate points in the two-dimensional emotion space. The multimodal confidence fusion module is used to evaluate the data quality and confidence of each input modality in real time. When the confidence of a certain modality is lower than a preset threshold, the fusion weight of other modalities is automatically increased. The visual scene semantic understanding module is used to classify scenes and extract the main color tone from the images captured by the rear-facing camera outside the vehicle, and output scene semantic labels and environmental color vectors.

[0019] Furthermore, the pattern generation engine includes: The conditional generation model uses the continuous emotional coordinates and the environmental color vector as conditional inputs to dynamically generate the visual elements, color scheme, dynamic sequence and rhythm of the pattern, and supports smooth interpolation of adjacent emotional states in the Valence-Arousal emotional space. An ambient light adaptive color mapping module is used to dynamically adjust the hue, saturation, and brightness of the generated pattern based on the ambient color vector and the current ambient brightness.

[0020] Furthermore, the system also includes an environmental adaptive adjustment mechanism: by monitoring the intensity of light outside the vehicle through an ambient light sensor, the brightness of the taillight display screen is automatically adjusted; by monitoring the distance between pedestrians and vehicles behind the vehicle through a rear-facing radar or the rear-facing camera outside the vehicle, the brightness of the pattern is automatically reduced or switched to static mode when the distance is less than a second preset threshold. The emotion computing model processes the user's biometric data offline at the vehicle end, and only outputs the desensitized continuous emotion vector. The original image and audio data are discarded immediately after real-time analysis.

[0021] A third aspect of the present invention provides a vehicle including the vehicle taillight emotion interaction system based on multi-source information fusion provided in the second aspect of the present invention.

[0022] Compared with existing technologies, the present invention provides a vehicle taillight emotional interaction method, system, and vehicle based on multi-source information fusion. By acquiring multi-source information including environmental data, spatiotemporal data, vehicle data, user biometric data, and external visual scene data, and after fusion processing by an emotional computing model, it outputs a continuous emotional vector in the Valence-Arousal two-dimensional emotional coordinate system. Then, using this emotional vector, environmental color vector, and scene semantic tags as conditions, it inputs a pattern generation engine to dynamically generate taillight pattern instructions and drive the display. This achieves a technological leap from static preset to dynamic generation of taillight display content, and from single function to context-adaptive expression. It enables vehicle taillights to generate personalized patterns with emotional semantics in real time according to the current comprehensive context, significantly improving the richness of interaction between the vehicle and the external environment and the user experience. Attached Figure Description

[0023] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0024] Figure 1 The flowchart is provided for the vehicle taillight emotion interaction method based on multi-source information fusion in Embodiment 1 of the present invention.

[0025] Figure 2 This is a flowchart of the multimodal confidence fusion processing provided in Embodiment 1 of the present invention.

[0026] Figure 3 The flowchart illustrates the mechanism for updating and cold-starting a user personalized preference model, as provided in Embodiment 1 of the present invention.

[0027] Figure 4 This is an architecture diagram of a vehicle taillight emotion interaction system based on multi-source information fusion, provided in Embodiment 2 of the present invention. Detailed Implementation

[0028] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the invention. In the description of the embodiments of the present invention, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment". The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0029] Before describing specific embodiments of the present invention, the key terms involved in the present invention will first be explained: 1. Two-Dimensional Emotion Coordinate System: Originating from the theory of emotional dimensions in psychology, proposed by psychologist Russell. The Valence axis represents the positivity or negativity of the emotion, typically ranging from negative (negative / unpleasant) to positive (positive / pleasant); the Arousal axis represents the intensity or degree of activation of the emotion, ranging from low activation (calm / sleep) to high activation (excitement / alertness). This coordinate system can map complex human emotional states into a two-dimensional continuous space, expressing more nuanced and richer emotional transitions compared to discrete emotion labels (such as "happy" or "sad"). In this invention, the continuous emotion vector output by the system is a coordinate point in this coordinate system, used to guide the generation of subsequent patterns.

[0030] 2. Spherical Linear Interpolation (Slerp): A method for smooth interpolation on a unit sphere, first proposed by Shoemake in computer graphics for quaternion interpolation to achieve smooth rotational animation. Unlike ordinary linear interpolation (Lerp), Slerp ensures that the interpolation path follows the spherical arc, thus maintaining the unit length characteristic of the interpolation result. In this invention, Slerp is used to smoothly transition the emotional coordinates between adjacent time steps, ensuring the continuity and naturalness of the pattern animation when emotional states change, and avoiding pattern jumps caused by abrupt changes in emotional coordinates.

[0031] 3. Conditional Generative Model: This type of generative model introduces additional conditional information (such as labels, vectors, and text descriptions) during the generation process. Common types include Conditional Generative Adversarial Networks (cGANs), Conditional Variational Autoencoders (cVAEs), and diffusion-based conditional generative models. Compared to unconditional generative models, conditional generative models can control the attributes of the generated result based on the input conditional information, achieving on-demand generation. In this invention, the conditional generative model uses continuous emotion vectors, environmental color vectors, and user preference vectors as conditional inputs to dynamically generate matching taillight patterns, making the generated content context-adaptive and personalized.

[0032] 4. Internet of Vehicles (IoV): This refers to a communication network system that uses vehicles as nodes and facilitates information exchange between vehicles, between vehicles and infrastructure, between vehicles and networks, and between vehicles and pedestrians through wireless communication technologies such as V2X (Vehicle-to-Everything). IoV can provide vehicles with environmental information far exceeding the range of onboard sensors, including real-time weather, traffic conditions, and road events. In this invention, the system acquires real-time weather conditions, temperature, air quality index, and public holiday information through the IoV communication unit, serving as an important source of environmental data for affective computing.

[0033] 5. Low-latency Communication Bus: This refers to a communication bus in the vehicle network that possesses deterministic latency and real-time transmission capabilities, such as Controller Area Network (CAN), Controller Area Network Flexible Data Rate (CANFD), and Automotive Ethernet. These buses ensure that sensor data and control commands are transmitted within a specified time window, forming the foundation of the vehicle's real-time control system. In this invention, the system control bus adopts a low-latency vehicle communication bus to ensure that the end-to-end latency from multi-source information acquisition to pattern command issuance meets real-time requirements.

[0034] 6. Pulse Width Modulation (PWM) Dimming: A common technique for controlling LED brightness by adjusting the duty cycle of a pulse signal. PWM dimming has advantages such as a wide dimming range, good color reproduction, and high control precision, and is widely used in automotive lighting systems. In this invention, the brightness adjustment of the taillight display screen is achieved through PWM dimming. Combined with ambient light sensor data, the display screen brightness is adaptively adjusted to adapt to the visual effect under different lighting conditions.

[0035] Example 1 This embodiment provides a vehicle taillight emotion interaction method based on multi-source information fusion, including: S1. Obtain multi-source information, including environmental data, spatiotemporal data, vehicle data, user biometric data, and external visual scene data.

[0036] Specifically, the environmental data includes at least real-time weather conditions, temperature, air quality index, and public holiday information obtained through vehicle-to-everything (V2X) services. Weather conditions include different meteorological conditions such as sunny, cloudy, overcast, rainy, snowy, and foggy. These conditions directly affect the lighting characteristics and color tendencies of the environment (e.g., overcast skies tend to be cool gray, sunny skies tend to be warm yellow), thus affecting the color selection and brightness settings of the taillight pattern. Temperature information can be used to infer the impact of seasonal changes on user emotions (e.g., warm weather is usually associated with positive emotions), while the air quality index reflects the transparency of the environment. Under smog or high pollution conditions, the color contrast and brightness of the taillight pattern need to be adjusted accordingly to ensure visibility. Public holiday information (e.g., Spring Festival, National Day, etc.) provides a thematic semantic background for pattern generation. The system can automatically incorporate relevant holiday elements (e.g., festive red tones, holiday icons, etc.) during the holidays to enhance the timeliness and cultural resonance of emotional expression. The spatiotemporal data includes at least precise time, date, seasonal information, and high-precision geographic location obtained through the Global Navigation Satellite System (GNSS). Precise time (hours, minutes) is used to infer the user's current stage of daily activity (such as commuting, lunch break, evening leisure time), with different time periods corresponding to different typical emotional tendencies and behavioral patterns. Date and seasonal information provide contextual background on a longer time scale (such as spring revival, summer vitality, autumn tranquility, winter introversion), and these natural rhythm changes also affect human emotional states. High-precision geographic location can obtain point of interest (POI) types (such as commercial areas, residential areas, parks, school surroundings, etc.) through reverse geocoding. Different places have different semantic attributes and emotional atmospheres, providing rich scene context for affective computing. The vehicle data includes at least vehicle speed, gear position, door status, and vehicle mode obtained through the vehicle bus (such as CAN bus or vehicle Ethernet). Vehicle speed signals are used to determine the vehicle's motion status (stationary, low speed, medium speed, high speed) and are one of the core inputs for triggering interaction. Gear position signals (P / R / N / D) help determine if the vehicle is in a parking / interactive state. Door status (open / closed) can be used to determine if the user is about to get in or out of the vehicle, thus predicting the start or end of the interaction in advance. Vehicle modes (such as Sport, Comfort, Eco, Off-road, etc.) reflect the current operating characteristics of the vehicle, and the system can adjust the dynamism and color style of the taillight patterns accordingly to ensure the interactive experience is consistent with the overall vehicle character. The user biometric data includes at least the driver's facial image obtained through the in-vehicle camera and the in-vehicle voice signal obtained through the microphone.Facial images are captured via a DMS camera, and computer vision technology is used to analyze the driver's facial expression features (such as eyebrow and eye shape, mouth corner curvature, facial muscle tension, etc.) to infer facial emotional tendencies (such as pleasure, calmness, surprise, tension, fatigue, etc.). Voice signals are captured via a microphone array, and information such as the occupants' tone of voice, speech rate, volume, and background music style is analyzed to help determine the user's emotional state and overall atmosphere (such as relaxed and cheerful, calm and soothing, tense and hurried, etc.). The external visual scene data includes at least real-time image frames captured by an external rear-facing camera, used to extract the dominant color tone, color mood, and scene semantics of the vehicle's surrounding environment. The external rear-facing camera is typically installed at the rear of the vehicle, has a wide field of view, and can capture visual information such as roads, buildings, sky, vegetation, other vehicles, and pedestrians behind the vehicle. By analyzing and processing the real-time image frames, the distribution of the dominant color tone and scene category (such as city streets, natural landscapes, tunnels, nighttime roads, etc.) can be obtained. This information is of significant guiding importance for generating taillight patterns that are coordinated with the surrounding visual environment.

[0037] S2. Determine whether the preset interaction trigger conditions are met based on the vehicle data.

[0038] Specifically, such as Figure 1As shown, the system first determines whether the vehicle is parked or moving at low speed. The interaction triggering conditions include at least: the vehicle is stationary or moving at extremely low speed for a duration exceeding a first preset threshold, and the vehicle is located in a public road area where light interaction is permitted. The stationary or extremely low speed state is determined by the vehicle speed signal; a speed less than or equal to a preset speed threshold (e.g., 2 km / h) is considered extremely low speed. A duration exceeding the first preset threshold (e.g., 10 to 15 seconds) indicates the vehicle is in a temporary stopped state (e.g., waiting at a red light, temporarily stopping to yield, queuing, etc.), rather than a brief deceleration or imminent start. This judgment logic aims to avoid triggering the display of emotional patterns during normal vehicle movement, thereby ensuring driving safety. The determination of public road areas is achieved through high-precision positioning combined with geofencing technology. The system has a built-in road attribute database that can identify whether the current vehicle location is in a public road area where light interaction is permitted (e.g., urban roads, scenic roads, parking lots, etc.). In scenarios such as private roads, highways, and tunnels, the system automatically prohibits interaction to avoid interference or misunderstanding for traffic participants. In addition, the system also uses the gear position in the vehicle data for auxiliary judgment. When the gear is in P (Park) or N (Neutral) and the electronic parking brake is activated, it indicates that the driver intends to keep the vehicle stationary, and the system's confidence in the interaction trigger condition is higher. When all the above sub-conditions are met, the system determines that the interaction trigger condition is met and enters the next stage of emotional interaction; if any sub-condition is not met (such as a sudden increase in vehicle speed, insufficient duration, or location outside the allowed area), the system remains in standby listening state and does not trigger the taillight emotional interaction.

[0039] S3. When the interaction triggering condition is met, the environmental data, the spatiotemporal data, the user biometric data, and the vehicle exterior visual scene data are input into the emotion computing model. After being fused and processed by the emotion computing model, a continuous emotion vector is output. The continuous emotion vector is the emotion coordinate point representing the current comprehensive situation in the Valence-Arousal two-dimensional emotion coordinate system.

[0040] In this step, such as Figure 2 As shown, the sentiment computing model outputs the continuous sentiment vector in the following manner: (1) Calculation of sentiment components for each modality: After inputting multi-source raw data, the system divides it into three processing branches: structured environment and spatiotemporal data, user biometric data, and vehicle exterior visual scene data.

[0041] For structured environmental data (weather, temperature, holidays, etc.) and spatiotemporal data (time, date, location, etc.), the system uses a rule engine based on expert knowledge to perform sentiment mapping and output environmental sentiment components. The rule engine has a built-in sentiment mapping rule library that has been validated through psychological experiments and user surveys. For example, on a sunny and warm weekend morning, when located in a park or recreational area, the environmental sentiment component is mapped to the first quadrant of the Valence-Arousal coordinate system (positive valence, moderate activation), corresponding to the emotional regions of "pleasure" and "relaxation"; while on a rainy and cold Monday morning rush hour, when located in a congested commercial area, the environmental sentiment component is mapped to the third quadrant (negative valence, low activation) or the fourth quadrant (negative valence, high activation), corresponding to the emotional regions of "depression" and "irritability".

[0042] For user biometric data (facial images and voice signals), the system employs a lightweight neural network model for analysis, outputting the user's emotional component. The facial expression recognition network takes face images captured by a DMS camera as input, performs feature extraction via a convolutional neural network and classification via fully connected layers, outputting the mapping coordinates of facial expressions in the Valence-Arousal space. The voice emotion recognition network takes acoustic features in the voice signal (such as fundamental frequency, energy, speech rate, formants, etc.) as input, analyzes them through a recurrent neural network or Transformer architecture, and outputs the two-dimensional coordinates corresponding to the voice emotion. The outputs of the two sub-networks are weighted and fused to obtain the comprehensive emotional component of the user's biometric modality.

[0043] For external visual scene data, the system uses a lightweight visual scene semantic understanding module for processing. This module is based on MobileNet or a similar high-efficiency convolutional neural network architecture and runs in real time on the vehicle's NPU. It outputs scene category labels (such as "city-sunny day", "nature-cloudy day", "night-street", etc.) and environmental color vectors. Then, the scene category is converted into the corresponding scene sentiment component through a scene-sentiment mapping table.

[0044] Assume the system has a total of The input modality, the first The sentiment components of each modality output are represented as a two-dimensional vector:

[0045] in, For Valence component, The two components together constitute the coordinates of this modality in the Valence-Arousal two-dimensional emotional space.

[0046] (2) Confidence assessment: After obtaining the emotional components of each modality, the system evaluates the signal quality of each modality. Confidence of each modality Sensor signal quality scoring The result obtained by mapping using the Sigmoid function is:

[0047] in: For the first Sensor signal quality score for each modality; for the camera modality, Determined by image sharpness score (e.g., Laplacian variance normalized value); for vehicle-to-everything (V2X) modes, Determined by the normalized value of the network signal strength; for the rule engine modality, Always 1 (structured data is always reliable); For the first Learnable sensitivity parameters for each modality control the response speed of confidence to changes in signal quality; For the first Learnable bias parameters for each modality control the baseline value of the confidence level; To output the confidence level.

[0048] (3) Calculation of normalized weights: After calculating the confidence scores for each mode using the Sigmoid function, the system performs Softmax normalization with a temperature coefficient on the confidence scores to obtain the fusion weights.

[0049] in, For temperature coefficient; when The time weights are concentrated on the mode with the highest confidence (hard selection), when The time weights tend to be evenly distributed (soft fusion); the typical value range is... ; For the first The fusion weights of each modality, and satisfying .

[0050] (4) Calculation of fused emotion vectors: After weight calculation, the system fuses the data using a weighted summation method, outputting Valence-Arousal two-dimensional continuous sentiment coordinates. The final fused sentiment vector... It is obtained by weighted summation of the sentiment components of each modality:

[0051] in: The final Valence score represents the positive or negative emotional valence of the overall context. The final Arousal value represents the degree of emotional activation in the overall context; That is, continuous emotional coordinates in the Valence-Arousal two-dimensional emotional space, due to Satisfy normalization constraints The weighted summation result naturally remains at Within the range, no additional truncation is required.

[0052] (5) Smooth transition of emotional state: When the system is in continuous time steps and When a change in sentiment coordinates is detected, spherical linear interpolation (Slerp) is used to smoothly transition between adjacent sentiment coordinates, avoiding abrupt pattern changes.

[0053] in: for Emotional coordinates of a moment As the emotional coordinates for the next moment; The time step for updating the emotional state, measured in seconds (s), typically takes the value of [value missing]. ; Indicates the current frame at time step Relative position within; Output The intermediate sentiment coordinates are obtained after interpolation. Since spherical linear interpolation is closed for points on a unit sphere, the output naturally remains within the range of the interpolated points. Within the range.

[0054] The sentiment computing model includes: a rule engine for processing structured environment and spatiotemporal data, a lightweight neural network model for analyzing unstructured user biometric data, and a lightweight visual scene semantic understanding module for processing vehicle-exterior visual scene data. The rule engine performs reasoning based on deterministic rules, offering fast response and strong interpretability, making it suitable for rapid processing of structured data. The lightweight neural network model is designed based on efficient network architectures such as MobileNetV3 and EfficientNet-Lite, and after pre-training and quantization compression, it is deployed on the vehicle's NPU, enabling facial expression recognition and speech emotion recognition under low latency. The visual scene semantic understanding module is also based on a lightweight CNN architecture, combining scene classification and dominant color extraction subtasks for multi-task learning, providing rich visual semantic information while ensuring real-time performance. The sentiment computing model also includes a multimodal confidence fusion mechanism: it evaluates the quality and confidence of each sensor input data in real time, and automatically increases the weights of other modalities when the confidence of a certain modality sensor falls below a preset threshold, ensuring the robustness of the fusion result. Even in extreme cases where sensors partially fail (e.g., the camera is glared by strong light or the microphone is interfered with by wind noise) or fail completely (e.g., the vehicle network signal is interrupted), this fusion mechanism can still output reasonable emotion vectors based on trusted modalities, ensuring that the basic functions of the system are not affected.

[0055] The external visual scene data is acquired and processed in the following ways: Real-time image frames are captured by an external rear-facing camera. These cameras typically employ high dynamic range (HDR) sensors, enabling them to capture acceptable image quality even under complex lighting conditions such as backlighting and nighttime illumination. The camera's frame rate is usually set to 15 fps to 30 fps, ensuring real-time performance without excessively consuming computing resources and bus bandwidth.

[0056] The real-time image frames are classified into scenes, and scene semantic labels are output. The scene classification uses a lightweight convolutional neural network, which, after being trained on a large amount of labeled data, can identify a variety of typical scene categories, including but not limited to: city daytime streets, city nighttime streets, suburban roads, highways, tunnels, natural landscapes (parks / mountains / coasts), parking lots, residential areas, etc. The scene semantic labels not only serve as one of the input conditions for the pattern generation engine, but also provide emotional cues at the visual scene level for the sentiment computing model (for example, natural landscape scenes are usually associated with positive valence, and tunnel scenes are usually associated with low activation and neutral to negative valence).

[0057] The real-time image frames are subjected to pixel-level clustering to extract the dominant color tone set, and the ambient color vector is calculated by weighting the pixel proportions of each cluster.

[0058] in, For the first The percentage of pixels in each cluster, For the first Cluster centers; The scene semantic tags and the environment color vectors serve as additional inputs to the pattern generation engine.

[0059] Preferably, the lightweight visual scene semantic understanding module performs scene classification (natural scene / urban scene / night scene, etc.) and main color extraction on the rear-view camera image outside the vehicle, and outputs environmental color vector and scene semantic label as additional input for the generation of continuous emotion vector.

[0060] S4. Input the continuous emotion vector and the environmental color vector and scene semantic label in the vehicle exterior visual scene data into the pattern generation engine, and dynamically generate the taillight display pattern instruction through the pattern generation engine.

[0061] Specifically, the pattern generation engine also obtains the user's personalized preference profile as input, and uses the continuous emotion vector, the environmental color vector, the scene semantic tags, and the preference vector in the preference profile as conditional inputs to generate the pattern instruction. The pattern generation engine adopts a conditional generation model based on continuous emotion vectors, using the continuous emotion coordinates and the environmental color vector as conditional inputs to dynamically generate the visual elements, color scheme, dynamic sequence, and change rhythm of the pattern; the pattern generation engine also includes an ambient light adaptive color mapping module, which dynamically adjusts the hue, saturation, and brightness of the generated pattern according to the environmental color vector and the current ambient brightness.

[0062] Specifically, such as Figure 1 As shown, the pattern generation engine also obtains the user's personalized preference profile as input, and uses the continuous emotion vector, the environmental color vector, the scene semantic label, and the preference vector in the preference profile as conditional inputs to generate the pattern instruction. The ambient light adaptive color mapping module achieves color adaptation in the following way: by mapping the ambient color vector... Convert from RGB space to HSV space to obtain ambient hue. saturation and brightness : Let the original color tone output by the pattern generation model be... Adjust it to a target color that harmonizes with the ambient color using the following color shift formula. : ,in: The original color tone output by the pattern generation model, in radians (rad). The ambient hue is expressed in radians (rad). The hue difference is expressed in radians (rad), and the sine function takes radians as its independent variable. For color blending intensity coefficient; when The color tone of the pattern is not affected by the environment. The maximum shift in pattern hue towards ambient hue; To Modulo operation, guaranteeing output Within the legal color range.

[0063] The formula for adjusting pattern brightness is:

[0064] in, For ambient brightness; The original brightness value output by the pattern generation model; As a weight for ambient brightness; The larger the value, the closer the pattern brightness is to the ambient brightness; a typical value is [value missing]. ; This is the lower limit of brightness, typically set to a value of [value missing]. (Corresponding to the lowest visible brightness threshold); This represents the upper limit of brightness, with a typical value of [value missing]. (Corresponding to the upper limit of anti-glare); This is a truncation function that constrains the output to a safe brightness range; This is the final output brightness.

[0065] S5. Send the pattern instruction to the taillight display controller to drive the taillight screen to display the emotional pattern.

[0066] In this step, such as Figure 1As shown, after receiving pattern instructions from the central processing module, the taillight display controller parses the instructions into pixel-level display data and writes the data to each pixel of the LED dot matrix display screen through the drive circuit. The taillight screen then displays a dynamic visual pattern. During the pattern display, the system continuously monitors whether the trigger conditions are still met. When the vehicle data indicates that the interaction trigger conditions are no longer met (such as the vehicle speed exceeding a preset threshold, the duration condition not being met, or the vehicle leaving the allowed interaction area), the system controls the emotional pattern on the taillight screen to smoothly fade out and return to the normal lighting mode. Smooth fading out means that the brightness of the pattern gradually decreases to zero within a set transition time (usually 0.5 to 2 seconds), while the taillights return to the normal lighting state required by regulations (such as red position lights or daytime running lights). This process ensures the smoothness and safety of the interaction state transition, avoiding the abruptness caused by sudden extinguishing or misunderstanding of vehicles behind. During the display of the emotional pattern, if the vehicle speed signal in the vehicle data is detected to trigger emergency braking (i.e., the vehicle speed drops significantly in a short period of time, exceeding a preset emergency braking threshold), the emotional pattern is immediately replaced by a full-screen red warning. This design follows the "safety first" principle, ensuring that in any situation that may endanger driving safety, the taillights can quickly return to a highly recognizable safety warning state, meeting regulatory requirements for taillight braking signals. The response time of the full-screen red warning is strictly controlled to the millisecond level, ensuring that the presentation of the safety signal is not delayed due to the display of the emotional pattern.

[0067] Furthermore, the method also includes a user personalized preference model update step. For example... Figure 3 As shown, the system first determines whether the user is a new user or whether there is insufficient historical data: For new users or those with insufficient historical data, the system initiates a cold start initialization process. This involves displaying several candidate styles on the vehicle's infotainment screen for the user to choose from, obtaining the user's selected theme style, and initializing the prior distribution of the preference model. Specifically, this is achieved through an active preference detection mechanism, displaying several candidate styles on the vehicle's screen for the user to choose from, and then determining the user's preferred style based on their selection. Initialize the prior distribution of the preference model:

[0068] in: The total number of candidate styles (e.g.) ), where is a positive integer; Index of user-selected preferred styles; For indicator functions, when The value is 1 if it is true, and 0 otherwise. This is the smoothing coefficient, typically taking the value of... This is used to preserve the exploration probability of other styles (the initial probability of each non-selected style is 0). ); Let the initial prior probabilities of each style satisfy the following conditions: .

[0069] For users with existing historical data, the system reads the user's historical preference profile and constructs a contextual feature vector. The context feature vector is composed of time features, location features, and emotional coordinate features:

[0070] in: The time feature vector is encoded using sine and cosine periodic codes. For the current hour (unit: hour), This indicates the current day of the week (0 = Monday, 6 = Sunday). For location semantic embedding vectors, The embedding dimension is determined by the number of location categories and the embedding layer design, with a typical value of [value missing]. Location categories include commercial areas, residential areas, parks / nature areas, roads / transportation areas, etc., and the corresponding embedding vectors are obtained by looking up a table. The current emotional coordinates. Overall context vector dimensions. ,when hour .

[0071] Based on contextual feature vectors, the system establishes a context-preference conditional probability model:

[0072] in, For preferred style categories, For the total number of styles, These are the model parameters. The MLP (Multilayer Perceptron) consists of an input layer, two hidden layers (128 neurons each), and an output layer, employing ReLU activation and Dropout regularization to prevent overfitting. The model uses contextual feature vectors... For input, output a The probability distribution represents the probability of a user's preference for different style categories in the current context. The system then drives the taillights to display the corresponding style pattern and collects user interaction feedback (including pattern selection, switching operations, and dwell time). When updating model parameters, a time-decay weight is applied to historical interaction records.

[0073] in: This is the current time, expressed in days. For the first The time of each historical record, in days; This is the time difference, expressed in days. The attenuation rate; More recent records have higher weight.

[0074] The time decay design is based on the reasonable assumption that user preferences are time-varying, meaning that users' aesthetic preferences change slowly over time, and recent preferences reflect current preferences better than long-term preferences. Finally, the system updates the model parameters by minimizing the weighted loss function.

[0075] in, For users in the first Actual preference selection (real label) in the second interaction. For the first The contextual feature vectors corresponding to each interaction are used. The loss function is the Weighted Negative Log-Likelihood Loss, which is optimized and updated using gradient descent (such as the Adam optimizer). The model is incrementally updated after each interaction, eliminating the need for retraining and ensuring that the model can adapt to changes in preferences online.

[0076] Furthermore, the emotion computing model processes the user's biometric data offline in the vehicle, outputting only the de-identified continuous emotion vector. The original image and audio data are discarded immediately after real-time analysis. This design adheres to the principle of minimizing privacy protection, ensuring that the user's biometric data (facial images, voice recordings) does not leave the vehicle and is not stored locally or uploaded to the cloud, fundamentally preventing the risk of leakage of sensitive user information.

[0077] In a specific application scenario of this embodiment: A car equipped with the system of this invention is driving towards a suburban park on Saturday morning and is waiting at a red light at an intersection. Information collection and scene triggering: Vehicle network data acquisition: Date "Saturday", Weather "Sunny", Temperature "22°C", Holiday information "None"; Location data: GPS time "10:00", Location "Main road leading to the park"; Vehicle data: Speed ​​"0km / h", Electronic parking brake activated, Continuous stationary time >15 seconds; User data: DMS camera detects the driver's relaxed expression and upturned lips; Microphone detects upbeat music playing inside the car; External visual scene data: The rear-facing camera captures the current intersection image, the visual scene semantic understanding module outputs the scene semantic label "City - Sunny", extracts the main environmental colors as sky blue and light green, and generates an environmental color vector [0.55, 0.72, 0.85] (normalized RGB representation). System judgment: The vehicle is stationary on a public road, satisfying the interaction triggering conditions.

[0078] Information fusion and continuous sentiment coordinate generation: Rule engine processing: Date (weekend) + Weather (sunny) + Time (morning) + Location (leading to the leisure area) Environmental emotional components: Valence = +0.7, Arousal = +0.4; ML model processing: facial expression (relaxed and happy) + audio background (upbeat music). User sentiment components: Valence = +0.8, Arousal = +0.5; Visual scene module processing: Scene semantics "city - sunny day" + environmental color vector $\rightarrow$ Scene sentiment components: Valence = +0.6, Arousal = +0.3; Multimodal confidence fusion: All three inputs have high confidence (clear camera images, normal vehicle network signal), weights are equally distributed, and the fusion output continuous sentiment coordinates are: Valence = +0.72, Arousal = +0.42, corresponding to the "relaxed and pleasant" region in the sentiment space.

[0079] Dynamic generation of emotionalized patterns: The system queries the user's multi-contextual preference profile. In the context of "weekend morning - city - relaxed and pleasant interval", the user's historical preference is a simple and natural style, and the time decay weight shows that the recent preference is stable; the conditional generation model uses emotional coordinates. Based on environmental color vectors and user preference vectors, a pattern scheme is dynamically generated: the left taillight screen generates a fluid animation of "lightly moving clouds," with the animation rhythm corresponding to the Arousal value (medium speed); the right taillight screen generates a dynamic silhouette of "tree shadows and flying birds"; the ambient light adaptive color mapping module maps the main color tone of the pattern to sky blue (echoing the sky color) and light warm yellow (harmonizing with the sunlight color tone) according to the environmental color vector, avoiding conflicts with the environmental colors. Pattern display and termination: the scheme is sent to the taillights, the display screen lights up, presenting the above vivid pattern; after waiting for about 40 seconds, the green light turns on, and the driver starts moving; the system detects the increase in vehicle speed and controls the taillight pattern to fade out smoothly within 1 second, then returns to the red daytime running light display.

[0080] Example 2 like Figure 4 This embodiment provides a vehicle taillight emotion interaction system based on multi-source information fusion, including: The multi-source information acquisition module includes a vehicle-to-everything (V2X) communication unit, a positioning and timing unit, an onboard bus interface, an in-vehicle camera and microphone array facing the driver, and an external rear-facing camera. It is used to collect environmental data, spatiotemporal data, vehicle data, user biometric data, and external visual scene data. The V2X communication unit uses a 5G-V2X or LTE-V2X communication module, enabling low-latency communication with roadside units (RSUs) and cloud servers to acquire real-time environmental and traffic information. The positioning and timing unit uses a multi-band GNSS receiver (supporting joint positioning with GPS, BeiDou, GLONASS, Galileo, and other systems), combined with an inertial measurement unit (IMU) for dead reckoning, maintaining a certain level of positioning accuracy even in areas with GNSS signal obstruction, such as under viaducts and tunnels. The onboard bus interface supports CAN and CAN bus interfaces. The system utilizes multiple protocols, including FD and automotive Ethernet, to adapt to the bus architecture of different vehicle models, acquiring signals such as vehicle speed, gear position, door status, and vehicle mode from the vehicle network. The in-vehicle camera uses a wide dynamic range (WDR) infrared camera, which can clearly capture the driver's facial image under strong backlight conditions during the day and low light conditions at night. The microphone array adopts a dual-microphone or quad-microphone array design with beamforming and noise suppression functions, which can effectively extract in-vehicle voice signals against the background of driving wind noise and road noise. The exterior rear-facing camera uses a high-resolution (at least 720p) HDR camera, installed above the license plate or at the tailgate handle, with a field of view of not less than 120°, which can cover a wide field of view behind the vehicle.

[0081] The central processing and decision-making module includes a processor and a memory. The memory stores an emotion computing model and a pattern generation engine. The emotion computing model fuses environmental data, spatiotemporal data, user biometric data, and external visual scene data to output a continuous emotion vector. This continuous emotion vector represents the emotion coordinates of the current comprehensive situation in a Valence-Arousal two-dimensional emotion coordinate system. The pattern generation engine dynamically generates taillight display pattern instructions based on the continuous emotion vector and the environmental color vector and scene semantic tags in the external visual scene data. The core processor of the central processing and decision-making module uses an automotive-grade SoC chip, integrating a multi-core ARM Cortex-A series CPU, GPU, and NPU, providing sufficient computing power for the real-time operation of the emotion computing model and the pattern generation engine. The memory includes LPDDR4 / 5 RAM and UFS / eMMC storage chips for storing model parameters, user preference profiles, and temporary data. This module also features a hardware security isolation area (TEE) for storing encryption keys for user preference profiles, preventing unauthorized access to sensitive data.

[0082] The intelligent taillight display module includes at least two high-resolution, independently pixel-controlled LED dot matrix displays integrated on both sides of the rear of the vehicle, along with their display driver controllers. These controllers receive pattern commands and display emotionally-oriented patterns. Each LED dot matrix display utilizes Mini-LED or Micro-LED technology, offering advantages such as high brightness, high contrast, wide color gamut, and long lifespan. The pixel density is no less than 100 PPI, ensuring clear and detailed pattern display. The display surface is covered with high-transmittance protective glass, possessing an IP6K9K dustproof and waterproof rating, capable of withstanding high-pressure water jets used in car washes and harsh weather conditions. Each display has at least 128×64 pixels, supports full-color display (24-bit color depth), and a refresh rate of no less than 60Hz, ensuring smooth dynamic patterns. The display driver controller incorporates a frame buffer, a color calibration table (LUT), and a PWM dimming control unit, enabling independent control of the brightness and color of each pixel for refined image display.

[0083] The system control bus is used to transmit data and control commands between the multi-source information acquisition module, the central processing and decision-making module, and the intelligent taillight display module. The system control bus uses an in-vehicle Ethernet backbone network with a transmission rate of no less than 100 Mbps, which can meet the bandwidth requirements of high-definition image data and pattern command data. For time-sensitive control commands (such as the full-screen red warning command during emergency braking), the priority arbitration mechanism of the CAN bus ensures that they are processed within microseconds of latency. Time synchronization between modules is achieved through IEEE 802.1AS or a similar Precision Time Protocol (PTP), ensuring that multi-source data is fused under a unified time base and avoiding inconsistencies in perception caused by time deviations.

[0084] Specifically, the memory also stores a user's personalized preference profile. The pattern generation engine uses this profile as input and generates the pattern instruction using the continuous emotion vector, the environmental color vector, the scene semantic label, and the preference vector in the profile as conditional inputs. The user's personalized preference profile is stored in a secure area of ​​the memory in encrypted form and includes the probability distribution of the user's style preferences under different contextual features, a summary of historical interaction records (only anonymized contextual features and style selections are saved, not the original biometric data), and model parameters. This profile supports multi-user configuration; different drivers can load their own preference profiles through facial recognition or account switching to achieve a personalized experience of "one person, one style."

[0085] Specifically, the emotion computing model includes: The continuous emotion space modeling module uses the Valence-Arousal two-dimensional emotion coordinate system to model emotions in a continuous space, outputting continuous emotion coordinate points within this two-dimensional emotion space. This module maintains the runtime state of the emotion coordinates and updates it periodically based on the multimodal fusion results. It also handles the historical caching of emotion coordinates (storing the emotion coordinate sequence of the most recent N time steps) for subsequent emotion trajectory analysis and temporal context reference for pattern generation. The output interface of the continuous emotion space modeling module is a standardized two-dimensional floating-point vector, facilitating direct use by downstream pattern generation engines.

[0086] The multimodal confidence fusion module is used to evaluate the data quality and confidence of each input modality in real time. When the confidence of a certain modality falls below a preset threshold, the fusion weight of other modalities is automatically increased. This module continuously monitors the data arrival rate and signal quality indicators of each input source, dynamically calculates the confidence value and fusion weight of each modality, and ensures that the fusion strategy can adaptively adjust as data quality changes. This module also has a built-in fault diagnosis function. When the confidence of a certain modality remains below the failure threshold, the system generates a diagnostic log and triggers a prompt message on the dashboard, reminding the user to check the relevant sensors (such as cleaning camera lenses, checking vehicle network antenna connections, etc.).

[0087] The visual scene semantic understanding module is used to classify scenes and extract dominant colors from images captured by the vehicle's external rear-facing camera, outputting scene semantic labels and environmental color vectors. This module is based on a lightweight deep neural network, employing a multi-task learning architecture. It shares the underlying feature extraction network and branches out at the top level for scene classification and dominant color extraction. The scene classification branch outputs the probability distribution of scene categories, while the dominant color extraction branch outputs the environmental color vector. Inference for this module is performed on the vehicle's NPU, with a single-frame processing latency of no more than 50 ms, meeting real-time processing requirements.

[0088] Specifically, the pattern generation engine includes: The conditional generation model, using the continuous emotional coordinates and the environmental color vector as conditional inputs, dynamically generates visual elements, color schemes, dynamic sequences, and rhythmic changes of patterns, and supports smooth interpolation of adjacent emotional states in the Valence-Arousal emotional space. The conditional generation model employs a conditional diffusion model or a conditional GAN ​​architecture. During the training phase, it uses a large amount of pattern data labeled with emotional and environmental tags, enabling the model to learn the mapping relationship from conditional vectors to pattern content. During the inference phase, the model performs sampling and iterative denoising in the latent space based on the current input conditional vector (diffusion model) or generator forward inference (GAN), outputting pattern content that matches the conditions. The pattern output by the model is represented in a multi-layered image format, including a foreground layer, a background layer, and a dynamic element layer, facilitating layered rendering and dynamic combination by the subsequent display control module. The conditional generation model supports trajectory interpolation in the emotional space; that is, when the emotional coordinates change continuously over time, the model can generate a series of transitional patterns corresponding to intermediate emotional states. These patterns visually present smooth morphological evolution and color gradations, rather than discrete jumps.

[0089] An ambient light adaptive color mapping module is used to dynamically adjust the hue, saturation, and brightness of the generated pattern based on the ambient color vector and the current ambient brightness. This module receives the ambient color vector from the visual scene semantic understanding module and the ambient brightness value from the ambient light sensor (typically mounted behind the windshield or on the exterior rearview mirror housing). It then performs color post-processing on the original pattern output by the conditional generation model according to the aforementioned hue shift formula and brightness adjustment formula. This module also supports a manual adjustment mode, allowing users to adjust the color blending intensity coefficient via the vehicle's infotainment system interface. and ambient brightness weight The preference value enables personalized control over the degree of color adaptation in the pattern.

[0090] Specifically, the system also includes an environmental adaptability adjustment mechanism: by monitoring the external light intensity through an ambient light sensor, the brightness of the taillight display screen is automatically adjusted to ensure that the pattern is clearly visible under different lighting conditions without glare. This mechanism is also linked to the vehicle's external lighting control strategy; when the vehicle's headlights or daytime running lights are in different states, the base brightness of the taillight display screen is adjusted accordingly, always maintaining coordination with the overall vehicle lighting system. The system monitors the distance between pedestrians and vehicles behind the vehicle using rear-facing radar or an external rear-facing camera. When the distance is less than a second preset threshold, the pattern brightness is automatically reduced or the system switches to a static mode to prevent overly bright dynamic patterns from causing visual interference to approaching pedestrians or other vehicle drivers. The rear-facing radar uses ultrasonic or millimeter-wave radar, achieving centimeter-level ranging accuracy. It is fused with visual detection results to improve the reliability and robustness of ranging.

[0091] The emotion computing model processes the user's biometric data offline in the vehicle, outputting only the desensitized continuous emotion vector. The original image and audio data are discarded immediately after real-time analysis. Specifically, after image frames captured by the in-vehicle camera enter the NPU for inference, the inference result (emotion component) is passed to the fusion module, while the original image frame data is overwritten or released simultaneously with the inference completion, without being written to storage media. Similarly, audio data captured by the microphone is discarded immediately after the speech emotion recognition model completes inference. The system implements a "one-time processing" design for data flow at the software architecture level, ensuring from the source that privacy data leaves no trace in the system.

[0092] Example 3 This invention provides a vehicle in embodiment three, including the vehicle taillight emotional interaction system based on multi-source information fusion described in embodiment two. The vehicle can be a pure electric vehicle, a hybrid electric vehicle, or a traditional gasoline-powered vehicle, as long as it has the corresponding electrical architecture and sensor configuration for installation. This system is integrated into the vehicle as an optional or standard feature before it leaves the factory, or it can be added as an aftermarket product. For factory-integrated solutions, the system's modules are deeply integrated with the vehicle network, enabling the acquisition of richer vehicle signals and synergy with other vehicle functions (such as linking with the navigation system to obtain route information, and linking with the entertainment system to obtain emotional cues related to music playback). For aftermarket solutions, the system can acquire some vehicle signals through the OBD interface and independently perform emotional calculations and pattern generation, exhibiting good compatibility and scalability. After the system is installed, the vehicle's taillights not only retain all the basic lighting functions required by regulations (turn signals, brake signals, position indicators, etc.), but also add emotional interaction functions based on multi-source information fusion, giving the vehicle richer expressive power and emotional warmth when interacting with the external environment.

[0093] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A vehicle taillight emotion interaction method based on multi-source information fusion, characterized in that, include: Acquire multi-source information, including environmental data, spatiotemporal data, vehicle data, user biometric data, and external visual scene data; Determine whether the preset interaction trigger conditions are met based on the vehicle data; When the interaction triggering condition is met, the environmental data, the spatiotemporal data, the user biometric data, and the vehicle exterior visual scene data are input into the emotion computing model. After being fused and processed by the emotion computing model, a continuous emotion vector is output. The continuous emotion vector is an emotion coordinate point representing the current comprehensive situation in the Valence-Arousal two-dimensional emotion coordinate system. The continuous emotion vector and the environmental color vector and scene semantic label in the vehicle exterior visual scene data are input into the pattern generation engine, and the pattern generation engine dynamically generates taillight display pattern instructions. The pattern command is sent to the taillight display controller to drive the taillight screen to display the emotional pattern.

2. The method according to claim 1, characterized in that, The pattern generation engine also obtains the user's personalized preference profile as input, and uses the continuous emotion vector, the environmental color vector, the scene semantic label, and the preference vector in the preference profile as conditional inputs to generate the pattern instruction.

3. The method according to claim 1, characterized in that, The sentiment computation model outputs the continuous sentiment vector in the following manner: The sentiment components of each input modality are calculated separately, and the sentiment components of each modality are represented as two-dimensional vectors. ,in, For Valence component, For Arousal components; Evaluate the confidence level of each input modality. The fusion weights are calculated based on the confidence levels of each modality. ; Based on the sentiment components of each modality and the fusion weights, a weighted summation method is used to calculate the fused sentiment vector. ,in, The fused emotion vector This refers to the continuous emotion vector.

4. The method according to claim 3, characterized in that, The confidence level Based on sensor signal quality score The result obtained by mapping using the Sigmoid function is: in, For sensitivity parameters, For bias parameters, Assess the quality of sensor signals; The fusion weight The confidence scores for each mode are obtained by applying Softmax normalization with a temperature coefficient: ,in, This is the temperature coefficient.

5. The method according to claim 1, characterized in that, The pattern generation engine adopts a conditional generation model based on continuous emotion vectors, using the continuous emotion coordinates and the environmental color vector as conditional inputs to dynamically generate the visual elements, color schemes, dynamic sequences, and rhythm of change of the pattern. The pattern generation engine also includes an ambient light adaptive color mapping module, which dynamically adjusts the hue, saturation, and brightness of the generated pattern based on the ambient color vector and the current ambient brightness.

6. The method according to claim 1, characterized in that, The external visual scene data is acquired and processed in the following ways: Real-time image frames are captured by a rear-facing camera outside the vehicle. The real-time image frames are classified into scenes, and scene semantic labels are output. The real-time image frames are subjected to pixel-level clustering to extract the main color tone set, and the ambient color vector is calculated by weighting the pixel proportions of each cluster. ,in, The pixel percentage of the k-th cluster. Let k be the cluster center; The scene semantic tags and the environment color vectors serve as additional inputs to the pattern generation engine.

7. The method according to claim 2, characterized in that, It also includes the user personalized preference model update steps: Constructing context feature vectors The context feature vector is composed of time features, location features, and emotional coordinate features; Establish a situation-preference conditional probability model ,in, For preferred style categories, These are model parameters; When updating model parameters, apply time-decay weights to historical interaction records. ,in, The attenuation rate, For the current time, The time of the k-th historical record; By minimizing the weighted loss function Update the model parameters.

8. The method according to claim 7, characterized in that, For new users, a proactive preference detection mechanism displays several candidate styles on the in-vehicle screen for the user to choose from, and the system then selects the preferred style based on the user's choice. Initialize the prior distribution of the preference model: in, The total number of candidate styles, This is the smoothing coefficient.

9. The method according to claim 1, characterized in that, The environmental data includes at least real-time weather conditions, temperature, air quality index, and public holiday information obtained through vehicle networking services; the spatiotemporal data includes at least precise time, date, season information, and high-precision geographical location obtained through the Global Navigation Satellite System; the vehicle data includes at least vehicle speed, gear position, door status, and vehicle mode obtained through the vehicle bus; and the user biometric data includes at least a driver's facial image obtained through an in-vehicle camera and in-vehicle voice signals obtained through a microphone. The interaction triggering conditions include at least the following: the vehicle is stationary or at extremely low speed for a duration exceeding a first preset threshold, and the vehicle is located in a public road area where light interaction is permitted.

10. The method according to claim 1, characterized in that, When the vehicle data indicates that the interaction triggering condition is no longer met, the emotional pattern on the taillight screen is controlled to fade out smoothly and return to the normal lighting mode. During the display of the emotional pattern, when the vehicle speed signal in the vehicle data is detected to trigger emergency braking, the emotional pattern is immediately replaced by a full-screen red warning.

11. A vehicle taillight emotion interaction system based on multi-source information fusion, characterized in that, include: The multi-source information acquisition module includes a vehicle network communication unit, a positioning and timing unit, an on-board bus interface, an in-vehicle camera and microphone array facing the driver, and an external rear-facing camera, used to collect environmental data, spatiotemporal data, vehicle data, user biometric data, and external visual scene data. The central processing and decision-making module includes a processor and a memory, wherein the memory stores the emotion computing model and the pattern generation engine; The emotion computing model is used to fuse the environmental data, the spatiotemporal data, the user biometric data, and the vehicle exterior visual scene data to output a continuous emotion vector. The continuous emotion vector is an emotion coordinate point representing the current comprehensive situation in the Valence-Arousal two-dimensional emotion coordinate system. The pattern generation engine is used to dynamically generate taillight display pattern instructions based on the continuous emotion vector and the environmental color vector and scene semantic tags in the vehicle exterior visual scene data. The intelligent taillight display module includes at least two high-resolution, independently pixel-controlled LED dot matrix displays integrated on both sides of the rear of the vehicle and their display driver controller, used to receive the pattern instructions and display emotional patterns; The system control bus is used to transmit data and control commands between the multi-source information acquisition module, the central processing and decision-making module, and the intelligent taillight display module.

12. The system according to claim 11, characterized in that, The memory also stores a user's personalized preference profile. The pattern generation engine also obtains the user's personalized preference profile as input and uses the continuous emotion vector, the environmental color vector, the scene semantic label, and the preference vector in the preference profile as conditional inputs to generate the pattern instruction.

13. The system according to claim 11, characterized in that, The sentiment computing model includes: The continuous emotion space modeling module uses the Valence-Arousal two-dimensional emotion coordinate system to perform continuous spatial modeling of emotions, and outputs continuous emotion coordinate points in the two-dimensional emotion space. The multimodal confidence fusion module is used to evaluate the data quality and confidence of each input modality in real time. When the confidence of a certain modality is lower than a preset threshold, the fusion weight of other modalities is automatically increased. The visual scene semantic understanding module is used to classify scenes and extract the main color tone from the images captured by the rear-facing camera outside the vehicle, and output scene semantic labels and environmental color vectors.

14. The system according to claim 11, characterized in that, The pattern generation engine includes: The conditional generation model uses the continuous emotional coordinates and the environmental color vector as conditional inputs to dynamically generate the visual elements, color scheme, dynamic sequence and rhythm of the pattern, and supports smooth interpolation of adjacent emotional states in the Valence-Arousal emotional space. An ambient light adaptive color mapping module is used to dynamically adjust the hue, saturation, and brightness of the generated pattern based on the ambient color vector and the current ambient brightness.

15. The system according to claim 11, characterized in that, The system also includes an environmental adaptive adjustment mechanism: it monitors the intensity of light outside the vehicle through an ambient light sensor and automatically adjusts the brightness of the taillight display screen; it monitors the distance between pedestrians and vehicles behind the vehicle through a rear-facing radar or an external rear-facing camera, and automatically reduces the pattern brightness or switches to static mode when the distance is less than a second preset threshold. The emotion computing model processes the user's biometric data offline at the vehicle end, and only outputs the desensitized continuous emotion vector. The original image and audio data are discarded immediately after real-time analysis.

16. A vehicle, characterized in that, Including the vehicle taillight emotion interaction system based on multi-source information fusion as described in any one of claims 11 to 15.

Images