Cabin atmosphere lamp control method and device and storage medium

Abstract

The invention discloses a cabin atmosphere lamp control method and device and a storage medium, and the method comprises the steps: carrying out the feature extraction of an obtained vehicle working data set, and obtaining vehicle working features, which comprise a music feature, a passenger feature and a vehicle working condition feature; determining a fusion rule according to the music features, the passenger features and the vehicle working condition features, and fusing the extracted features based on the fusion rule to obtain fusion features; inputting the fusion features into a vehicle driving atmosphere type identification model, and generating a light control instruction; and based on the light control instruction, controlling an atmosphere lamp in the vehicle cabin to work according to the first working mode parameter indicated by the light control instruction. The fusion features are obtained based on multi-modal feature fusion, and the fusion rule is selected according to the extracted features, so that the light control instruction is more in line with the real driving atmosphere, and the cabin atmosphere lamp is controlled more intelligently and more in line with the real driving atmosphere.

Description

Technical Field

[0001] This application relates to the field of vehicle control, specifically to a method, device, and storage medium for controlling cabin ambient lighting. Background Technology

[0002] As a key element of the driving experience, automotive ambient lighting control technology has seen significant development in recent years.

[0003] The existing technology allows users to manually select colors and adjust brightness via the central control screen or physical buttons. Specifically, it adjusts the PWM (Pulse Width Modulation) signal input to the RGB LED beads based on user button operations, controlling the duty cycle of the three primary colors of the RGB LEDs to mix different colors. However, the control logic of ambient lighting through button selection and brightness adjustment is relatively simple and cannot cope with complex driving scenarios. Summary of the Invention

[0004] This application provides a method, device, and storage medium for controlling cabin ambient lighting, which can be used for intelligent control of ambient lighting to adapt to different driving atmospheres. The technical solution is as follows: Firstly, a method for controlling cabin ambient lighting is provided, the method comprising: Obtain the vehicle's operational data set; Feature extraction is performed on the vehicle operating data set to obtain vehicle operating features, which include: music features, occupant features, and vehicle operating condition features. A fusion rule is determined based on the music features, occupant features, and vehicle operating condition features, and the music features, occupant features, and vehicle operating condition features are fused based on the fusion rule to obtain fused features; The fused features are input into the vehicle driving atmosphere type recognition model to generate lighting control commands; Based on the lighting control command, the ambient lights in the vehicle cabin are controlled to operate according to the first operating mode parameters indicated by the lighting control command. The lighting control command is used to indicate at least one of the RGB values, illumination time, and illumination brightness of the ambient lights when they are operating.

[0005] Secondly, a cabin ambient lighting control device is provided, the device comprising: The acquisition module is used to acquire a set of vehicle operating data. The feature extraction module is used to extract features from the vehicle operating data set to obtain vehicle operating features, which include: music features, occupant features and vehicle operating condition features. a feature fusion module configured to determine a fusion rule based on the music feature, the occupant feature and the vehicle working condition feature, and fuse the music feature, the occupant feature and the vehicle working condition feature based on the fusion rule to obtain a fusion feature; an atmosphere recognition module configured to input the fusion feature into a vehicle driving atmosphere type recognition model to generate a light control instruction; an instruction generation module configured to control an atmosphere lamp in a vehicle cabin to work in a first working mode parameter indicated by the light control instruction based on the light control instruction, the light control instruction being used to indicate at least one of an RGB value, a light emitting time and a light emitting brightness of the atmosphere lamp when working.

[0006] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program / instruction. The computer program stored in the computer readable storage medium is loaded and executed by a processor to implement the steps of the cabin atmosphere lamp control method according to any one of the preceding aspects.

[0007] In a fourth aspect, a computer program product is provided, and the computer program product includes a computer program / instruction. The computer program / instruction is executed by a processor to implement the steps of the cabin atmosphere lamp control method according to any one of the preceding aspects.

[0008] The technical solutions provided in the present application bring at least the following beneficial effects: The present application selects a corresponding fusion rule based on the music feature, the occupant feature and the vehicle working condition feature extracted from a vehicle working data set, and fuses the music feature, the occupant feature and the vehicle working condition feature based on the fusion rule to obtain a fusion feature. Since the fusion feature is obtained based on multi-modal feature fusion, and the fusion rule is selected according to the extracted features, the fusion feature can fully and truly represent different driving atmospheres, so that the working parameters indicated by the light control instruction recognized by the subsequent vehicle driving atmosphere type recognition model are more in line with the real situation, thereby enabling the cabin atmosphere lamp to be controlled more intelligently and more in line with the real driving atmosphere based on the light control instruction. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0010] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application; Figure 2is a cockpit ambient light control method flowchart provided by an embodiment of the present application. Figure 3 is a structural schematic diagram of a cockpit ambient light control device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0011] To make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0012] Reference is made to Figure 1 which shows a schematic diagram of a method implementation environment provided by an embodiment of the present application. The implementation environment can include: an ECU (Electronic Control Unit, cockpit domain controller) 11, a BCM (Body control module, vehicle body controller) 12, an ADAS (Advanced Driver-Assistance Systems, advanced driving assistance system) 13, an IVI (In-Vehicle Infotainment, information entertainment system) 14, a sensor unit 15, an LED control unit 16 and an LED ambient light 17.

[0013] The cockpit domain controller ECU serves as the brain of the system, using a high-performance vehicle-grade SoC chip, such as a processor based on an ARM Cortex-M7 core (with a frequency not less than 480 MHz), or an NXP S32K344 of the same level. The ECU has at least 2MB of Flash and 512KB of SRAM to run complex fusion algorithms and store context model libraries.

[0014] The vehicle body controller BCM is responsible for managing and controlling the vehicle body electrical equipment, such as vehicle lights, vehicle windows, door locks, wipers, rearview mirrors, etc. The vehicle body controller BCM is in communication connection with the cockpit domain controller ECU through a CAN bus, to send the vehicle speed, driving mode to the cockpit domain controller ECU, wherein the driving mode includes the economy mode, the comfort / standard mode, the sports mode.

[0015] The advanced driving assistance system ADAS is used to improve driving safety and comfort. The advanced driving assistance system ADAS is in communication connection with the cockpit domain controller ECU through a CAN bus, to send collision warning information or navigation prompt information to the cockpit domain controller ECU.

[0016] The information entertainment system IVI is an "intelligent interaction center" in the vehicle, integrating entertainment, navigation, communication and other functions to improve the driving experience. The information entertainment system IVI is in communication connection with the cockpit domain controller ECU through a CAN bus, to send navigation information, music data to the cockpit domain controller ECU.

[0017] The sensor unit is in communication connection with the cabin domain controller ECU through a CAN bus to send the collected sensor data to the cabin domain controller ECU. The sensor unit includes at least one of an illumination sensor, a capacitive / pressure sensor, a temperature and humidity sensor, a camera, and a microphone array. The illumination sensor is used to monitor the ambient illumination and color temperature in the cabin. Optionally, the illumination sensor is installed near the vehicle instrument panel or interior rearview mirror, and the data acquisition frequency is 1 Hz. The capacitive / pressure sensor is used to monitor the heart rate (HR) and respiratory rate (RR) of the occupants. Optionally, the capacitive / pressure sensor is integrated into the seat / steering wheel, and the data acquisition frequency is 5 Hz. The temperature and humidity sensor is used to monitor the temperature and humidity in the cabin. Optionally, the temperature and humidity sensor has a data acquisition frequency of 0.5 Hz. The camera is used to acquire image data. Optionally, the camera includes an in-cabin wide-angle camera and an out-of-cabin camera. The in-cabin wide-angle camera is used to collect in-cabin image data and is installed above the A-pillar or interior rearview mirror, uses infrared fill light, and has an image acquisition frequency of 15 fps. The out-of-cabin camera is used to acquire out-of-cabin image data. The microphone array is used to acquire sound data.

[0018] In one possible implementation, the sensor unit further includes an in-cabin millimeter wave radar sensor and an out-of-cabin millimeter wave radar sensor. The in-cabin millimeter wave radar sensor is used to monitor the heart rate (HR) and respiratory rate (RR) of the driver and passengers, and the out-of-cabin millimeter wave radar sensor is used to measure the distance information between the out-of-cabin vehicle and the current vehicle. Optionally, the millimeter wave radar sensor has a data acquisition frequency of 5 Hz.

[0019] The LED control unit is in communication connection with the cabin domain controller ECU through a LIN (Local Interconnect Network) bus, and the LED control unit is electrically connected to the LED. The LED control unit is used to receive instructions sent by the cabin domain controller ECU and control the RGB value, light-emitting time, and light-emitting brightness of the LED atmosphere lamp based on the instructions. The LED control unit is a distributed LED control unit, which includes at least one LED control subunit. The types of LED control subunits include a door panel LED control subunit, an instrument panel LED control subunit, and a footwell LED control subunit. Each LED control subunit includes an MCU (Micro Controller Unit) and an LED driver. For example, the MCU is a TCPL01x series, and the LED driver is an AL5887Q series. Each LED control subunit has a corresponding LED atmosphere lamp.

[0020] The LED atmosphere lamp has independently addressable intelligent LED lamp beads, that is, the lamp beads integrate a control chip IC, so that the color and brightness of the lamp beads can be controlled individually, thereby realizing different dynamic effects.

[0021] In the embodiment of the application, the cabin domain controller ECU obtains a vehicle working data set from a sensor unit, a body controller BCM, an advanced driving assistance system ADAS and an infotainment system IVI, generates a light control instruction based on the vehicle working data set, and sends the light control instruction to an LED control unit, so that the LED control unit controls the LED atmosphere lamp to work at the working parameters indicated by the light control instruction, including RGB values, light-emitting time and light-emitting brightness.

[0022] In some embodiments, the working parameters further include address information of the LED control subunit, so that after the light control instruction is broadcast to the LED control subunit, the LED control subunit corresponding to the address information executes the light control instruction.

[0023] Based on the above Figure 1 The embodiment of the application provides a cabin atmosphere lamp control method, as shown in the method applied to the cabin domain controller ECU, the method comprises steps 101-105. Figure 2

[0024] Step 101, obtaining a vehicle working data set.

[0025] The vehicle working data set includes data related to vehicle working conditions sent by the body controller BCM, data related to vehicle working conditions sent by the advanced driving assistance system ADAS, data related to vehicle working conditions sent by the infotainment system IVI, and data related to vehicle working conditions sent by the sensor unit.

[0026] Specifically, the body controller BCM sends vehicle speed and driving mode to the cabin domain controller ECU, the advanced driving assistance system ADAS sends collision warning information or navigation prompt information to the cabin domain controller ECU, the infotainment system IVI sends navigation information and music data to the cabin domain controller ECU, and the sensor unit sends multi-modal sensor data to the cabin domain controller ECU.

[0027] In some embodiments, the multi-modal sensor data includes ambient illuminance and color temperature collected by an illumination sensor, capacitance / pressure collected by a user pressing a capacitance / pressure sensor, temperature and humidity collected by a temperature and humidity sensor, image data collected by a camera, sound data collected by a microphone array, and radar wave data collected by a millimeter wave radar sensor.

[0028] ​At step 102, feature extraction is performed on the vehicle working data set to obtain vehicle working features, including music features, passenger features, and vehicle working condition features.

[0029] The cabin domain controller ECU stores a feature extraction model to perform feature extraction on the vehicle working data set by the model to obtain the vehicle working features.

[0030] Specifically, music data and sound data in the vehicle working data set are subjected to feature extraction to obtain music features, including rhythm features, frequency features, type features, energy features, and spectral distribution features of music played by the vehicle.

[0031] The multi-modal sensor data in the vehicle working data set are subjected to feature extraction to obtain passenger features.

[0032] The image data, radar wave data, vehicle speed information, driving mode information, collision warning information or navigation prompt information, and navigation information in the vehicle working data set are subjected to feature extraction to obtain vehicle working condition features, including vehicle speed features, driving mode features, navigation features, and complexity features of the vehicle working condition.

[0033] In some embodiments, the vehicle working condition features further include illumination features obtained by extracting the ambient illuminance and color temperature monitored by the illumination sensor.

[0034] The embodiments of the present application introduce illumination features, so that the ambient light is also considered when intelligently adjusting the brightness and working mode of the atmosphere lamp, thereby ensuring the visual effect, minimizing energy consumption, and avoiding interference with driving safety.

[0035] In some embodiments, the multi-modal sensor data in the vehicle working data set are subjected to feature extraction to obtain passenger features, including: The multi-modal sensor data are subjected to feature extraction to obtain initial passenger features. The initial passenger features are converted into passenger state labels based on a membership function. The passenger features are obtained based on the passenger state labels and the initial passenger features.

[0036] The initial passenger features include an identity feature, an emotion feature, an expression feature, an age feature, a number of people feature, a seat distribution feature, a fatigue feature, an attention feature, a heart rate feature, and a breathing frequency feature. The identity feature, the emotion feature, the expression feature, the age feature, the number of people feature, the sleep feature, the fatigue feature, and the attention feature are obtained by analyzing camera image data. The fatigue feature is used to represent whether the driver is driving while tired, and the attention feature is used to represent whether the driver is inattentive. The heart rate feature and the breathing frequency feature are obtained by analyzing the capacitance / pressure of a capacitance / pressure sensor. The passenger state label includes an emotion state label, an expression state label, an age state label, a fatigue state label, an attention state label, a heart rate state label, and a breathing frequency state label. The passenger feature includes the initial passenger feature and the passenger state label.

[0037] According to the embodiments of the present application, the passenger features, in particular the physiological features such as the heart rate feature, the breathing feature, and the behavior pattern, are extracted, so that the emotion and physiological state of the user can be more accurately evaluated.

[0038] In some embodiments, the initial passenger features further include a conversation atmosphere feature, which is determined by analyzing the voice tone in the sound data. The conversation atmosphere feature is used to represent the conversation atmosphere type in the vehicle, and the conversation atmosphere type includes one or more of a hot conversation atmosphere, a relaxed conversation atmosphere, a tense conversation atmosphere, and a happy conversation atmosphere.

[0039] For example, a pre-trained deep learning model (such as MobileNet-SSD for face detection and EfficientNet for expression classification) is used to analyze the image data, and the identity feature, the emotion feature, the expression feature, the age feature, the number of people feature, the sleep feature, the fatigue feature, and the attention feature are extracted.

[0040] In some embodiments, the membership function is a passenger physiological reference index. Before the initial passenger features are converted into the passenger state label based on the membership function, the following steps are included: Identity verification is performed according to the identity feature in the initial passenger features; If the identity verification is passed, the passenger physiological reference index is obtained.

[0041] Since the user's mood, fatigue level, age, etc. are difficult to accurately quantify, the embodiments of the present application define a membership function through fuzzy control (Fuzzy Control), and based on the fuzzy logic controller and the membership function, convert the difficult-to-quantify features in the initial occupant features into a fuzzy set (state label), so that the control logic is closer to the human way of thinking. Fuzzy control is an intelligent control strategy that simulates human fuzzy thinking and decision-making methods. Unlike traditional control methods based on precise mathematical models (such as PID control), fuzzy control does not require precise mathematical equations for the system, but rather uses the experience and knowledge of human experts to construct fuzzy rules. The membership function is a concept in fuzzy control, used to describe the degree to which an element belongs to a certain fuzzy set.

[0042] For example, based on the identity feature in the initial occupant features, the occupant physiological reference index is obtained, the heart rate feature in the initial occupant features is compared with the heart rate index in the occupant physiological reference index, and the heart rate state label is determined, so as to convert the heart rate feature of the occupant feature into the heart rate state label. The heart rate state label includes "normal heart rate", "slightly high heart rate" and "very high heart rate".

[0043] For example, the age feature in the initial occupant features is compared with the age index in the occupant physiological reference index, and the age state label is determined. The age state label represents the age range of the occupant, and the age range includes the age range of children, the age range of teenagers, the age range of middle-aged people and the age range of the elderly.

[0044] For example, the heart rate feature and the respiratory rate feature are converted into the emotion state label through the membership function. The emotion state label is used to represent the emotion type and level of the user. For example, a heart rate of 100 beats per minute and a respiratory rate of 30 times per minute are converted into a "slightly nervous" emotion state label or a "very nervous" emotion state label of the user.

[0045] It can be understood that the membership function can also be configured for the initial occupant features in the embodiments of the present application. The membership function can also be configured for the music features and the vehicle working condition features to convert the features in the music features and the vehicle working condition features into quantifiable data. For example, the light feature in the vehicle working condition feature is quantified into an environmental brightness level and an environmental color temperature level, and the environmental brightness level and the environmental color temperature level are coupled into the vehicle working condition feature.

[0046] Step 103, determining a fusion rule according to the music features, the occupant features and the vehicle working condition features, and fusing the music features, the occupant features and the vehicle working condition features based on the fusion rule to obtain the fused features.

[0047] In some embodiments, determining the fusion rule according to the music features, the occupant features and the vehicle working condition features includes: In a case where the occupant feature represents that the driver is in a tense state, a fatigue state, or a state of inattention, a first fusion rule is determined, and a fusion weight corresponding to the occupant feature in the first fusion rule is not less than a first threshold.

[0048] In a case where the driver is in a tense state, a fatigue state, or a state of inattention, a safety accident often occurs due to the driver being in a tense state, a fatigue state, or a state of inattention. Therefore, in a case where the occupant feature represents that the driver is in a tense state, a fatigue state, or a state of inattention, the safety priority principle is executed in the embodiment of the application, the fusion weight of the occupant feature is determined to be the highest, and the fusion weights of the music feature and the vehicle working condition feature are reduced.

[0049] In a case where the driver is in a fatigue state or a state of inattention, the fusion weight corresponding to the occupant feature in the first fusion rule is not less than the first threshold, so that the weight of the occupant feature can be guaranteed. Therefore, the music feature, the occupant feature, and the vehicle working condition feature are fused based on the first fusion rule to obtain a fusion feature, so that the subsequently generated light control instruction is mainly for a case where the driver is in a fatigue state or a state of inattention or for the mood of the driver. Preferably, the first threshold is less than 0.8 and greater than or equal to 0.5.

[0050] In a case where the vehicle working condition feature represents that the road condition in which the vehicle is located has a complexity greater than a first complexity threshold, a second fusion rule is determined, and a fusion weight of the vehicle working condition feature in the second fusion rule is not less than a second threshold.

[0051] In a case where the road condition is complex, the fusion weight of the vehicle working condition feature is not less than the second threshold, so that the weight of the vehicle working condition feature can be guaranteed. Therefore, the music feature, the occupant feature, and the vehicle working condition feature are fused based on the second fusion rule to obtain a fusion feature, so that the subsequently generated light control instruction is mainly for prompting the driver that the road condition is complex or for reducing the interference of the atmosphere lamp on the driving of the driver. Exemplarily, the second threshold is greater than or equal to 0.5.

[0052] In a case where the vehicle working condition feature represents that the road condition in which the vehicle is located has a complexity greater than a second complexity threshold, a third fusion rule is determined, and a fusion weight of the vehicle working condition feature in the third fusion rule is not less than a third threshold, the third threshold is greater than the second threshold, and the second complexity threshold is greater than the first complexity threshold.

[0053] In a case where the road condition complexity is greater than the second complexity threshold, it indicates that the road condition is more complex, and therefore, in a case where the vehicle working condition feature represents that the road condition complexity where the vehicle is located is greater than the second complexity threshold, the fusion weight of the vehicle working condition feature in the third fusion rule is not less than the third threshold, which can further improve the fusion weight of the vehicle working condition feature, so that the music feature, the passenger feature and the vehicle working condition feature are fused based on the third fusion rule to obtain the fusion feature, so that the subsequently generated light control instruction can further prompt the driver that the road condition is complex or reduce the interference of the atmosphere lamp on the driving of the driver. Exemplarily, the third threshold is greater than or equal to 0.8.

[0054] In some embodiments, determining the fusion rule according to the music feature, the passenger feature and the vehicle working condition feature further includes: In a case where the passenger feature represents that the driver is in a normal state and the vehicle working condition feature represents that the road condition complexity where the vehicle is located is less than the first complexity threshold, a preset fusion rule is determined, so that the music feature, the passenger feature and the vehicle working condition feature are fused based on the preset fusion rule to obtain the fusion feature.

[0055] By setting the preset fusion rule, the feature fusion under the normal working state of the vehicle is met. The fusion weight corresponding to the passenger feature in the preset fusion rule is not less than the fourth threshold, and the vehicle working condition feature is not less than the fifth threshold, wherein the fourth threshold is greater than 0.3 and the fifth threshold is greater than 0.3.

[0056] In yet some embodiments, determining the fusion rule according to the music feature, the passenger feature and the vehicle working condition feature includes: A first safety level corresponding to the passenger feature and a second safety level corresponding to the vehicle working condition feature are determined. If the first safety level is greater than or equal to the second safety level, a fourth fusion rule is determined, in which the fusion feature of the passenger feature is greater than or equal to the fusion feature of the vehicle working condition feature. If the first safety level is less than the second safety level, a fifth fusion rule is determined, in which the fusion feature of the passenger feature is less than the fusion feature of the vehicle working condition feature.

[0057] Since the passenger feature and the vehicle working condition feature are two different types of features, in order to further control the atmosphere lamp, in the embodiments of the present application, the passenger feature and the vehicle working condition feature are both converted into safety levels, the first safety level corresponding to the passenger feature and the second safety level corresponding to the vehicle working condition feature are compared to determine the fusion rule.

[0058] For example, in complex vehicle conditions, if the driver's driving skills are low, the determined first safety level is greater than or equal to the second safety level, and the fusion characteristics of occupant features are greater than or equal to the fusion characteristics of vehicle operating conditions. Subsequent generated lighting control commands primarily target situations where the driver is fatigued, inattentive, or in a state of mood. Conversely, in complex vehicle conditions, because the driver's driving skills are high, the driver can calmly handle the current situation. The driver's and occupant's emotional and facial features indicate a calm state of mind, the determined first safety level is lower than the second safety level, and the fusion characteristics of occupant features are lower than the fusion characteristics of vehicle operating conditions. This results in subsequent generated lighting control commands that focus more on alerting the driver to complex road conditions or reducing the interference of ambient lighting on the driver's driving.

[0059] Step 104: Input the fused features into the vehicle driving atmosphere type recognition model to generate lighting control commands.

[0060] In some embodiments, the vehicle driving atmosphere type recognition model identifies the fused features and directly obtains the lighting control commands corresponding to the fused features.

[0061] In some other embodiments, the fused features are input into a vehicle driving atmosphere type recognition model to generate lighting control commands, including: The fused features are input into the vehicle driving atmosphere type recognition model to obtain the vehicle driving atmosphere type; Based on the correspondence table between vehicle driving atmosphere type and lighting control command, determine the lighting control command corresponding to the vehicle driving atmosphere type.

[0062] By identifying and fusing features through a vehicle driving atmosphere type recognition model, the vehicle driving atmosphere type is obtained. Then, based on the correspondence table between the vehicle driving atmosphere type and the lighting control command, the lighting control command corresponding to the vehicle driving atmosphere type is determined, which can avoid the problem of the fusion features being difficult to quantify accurately.

[0063] Step 105: Based on the lighting control command, control the ambient lights in the vehicle cabin to operate according to the first working mode parameters indicated by the lighting control command. The first working mode parameters include at least one of RGB value, illumination time, and illumination brightness.

[0064] The lighting control instructions include lighting control sub-instructions. The lighting control sub-instructions indicate the LED control sub-unit address information and operating mode parameters, which include at least one of the following: RGB value, illumination time, and illumination brightness.

[0065] After generating the lighting control command, the cockpit domain controller ECU sends the lighting control sub-commands in the lighting control command to the target LED control sub-unit with the corresponding address information among multiple LED control sub-units, based on the address information of the lighting control command. The target LED control sub-unit then executes the lighting control sub-commands to control the operating parameters of the target LED ambient light corresponding to the target LED control sub-unit.

[0066] In some embodiments, the cockpit domain controller ECU also broadcasts lighting control commands to each LED control subunit, causing the target LED control subunit with corresponding address information among the multiple LED control subunits to execute the lighting control sub-commands to control the operating parameters of the target LED ambient light corresponding to the target LED control subunit. In some embodiments, after controlling the ambient light in the cockpit to operate according to the first operating mode parameters indicated by the lighting control commands, the method further includes: Received user input control commands; The ambient lighting is controlled by user commands to operate according to the second operating mode parameters indicated by the user commands.

[0067] Considering the complexity of vehicle conditions, the lighting control commands output by the vehicle driving atmosphere type recognition model may not be able to meet user needs. In this embodiment, after the ambient lights in the cabin are controlled to operate with the first working mode parameters indicated by the lighting control command, if a user control command for modifying the ambient light working mode parameters is received, the ambient lights are controlled to operate with the second working mode parameters indicated by the user control command based on the user control command, thereby facilitating the user to flexibly modify the working parameters of the ambient lights and select their preferred working mode.

[0068] In some embodiments, the cockpit domain controller ECU stores each received user control command to build a set of user history control commands.

[0069] In some embodiments, the method further includes: After the ambient lights in the vehicle cabin are controlled to operate according to the first working mode parameters indicated by the lighting control command, if a user control command is received within a preset time, the user's historical control command set is obtained. In the user's historical control command set, identify the target control command that is similar to the user's control command and the number of target control commands. The target control command is used to control the ambient light to switch from the third working mode parameter to the fourth working mode parameter. The third working mode parameter of the ambient light corresponding to the target control command is similar to the first working mode parameter, and the fourth working mode parameter of the ambient light corresponding to the target control command is similar to the second working mode parameter. Specifically, the similarity between the third and first working mode parameters is determined by comparing their RGB values, emission time, and brightness with those of the first working mode parameters. Similarly, the similarity between the fourth and second working mode parameters is determined by comparing their RGB values, emission time, and brightness with those of the second working mode parameters.

[0070] In one possible implementation, the third working mode parameter is converted into a third working mode parameter vector and the first working mode parameter is converted into a first working mode parameter vector through normalization. The Euclidean distance between the third and first working mode parameter vectors is determined. If the Euclidean distance between the third and first working mode parameter vectors is less than a distance threshold, then the third working mode parameter is similar to the first working mode parameter; conversely, if the Euclidean distance is greater than or equal to the distance threshold, then the third working mode parameter is dissimilar to the first working mode parameter. Similarly, the fourth working mode parameter is converted into a fourth working mode parameter vector and the second working mode parameter is converted into a second working mode parameter vector through normalization. The Euclidean distance between the fourth and second working mode parameter vectors is determined. If the Euclidean distance between the fourth and second working mode parameter vectors is less than a distance threshold, then the fourth working mode parameter is similar to the second working mode parameter; conversely, if the Euclidean distance is greater than or equal to the distance threshold, then the fourth working mode parameter is dissimilar to the second working mode parameter.

[0071] In this embodiment, within a preset time after the ambient lights are controlled to operate with the first operating mode parameter based on vehicle operating data, if the user modifies the operating mode parameter of the ambient lights through user control commands, the user's modification of the ambient light operating parameter will be recorded each time. If the user modifies the operating parameter of a certain ambient light a large number of times, the parameters of the vehicle driving atmosphere type recognition model will be adjusted, or the first operating mode parameter in the relational table will be replaced with the second operating mode parameter, or the fusion weight in the fusion rule will be modified to change the operating parameter indicated by the light control command output by the vehicle driving atmosphere type recognition model. This adaptively modifies the model's recognition capability, the mapping relationship in the relational table, or the weight allocation logic in the fusion rule, thereby gradually learning and adapting to the preferences of a specific user.

[0072] In this embodiment, vehicle operating data is obtained by receiving data from multiple sources. Based on the music features, occupant features, and vehicle operating condition features extracted from the vehicle operating data set, corresponding fusion rules are selected. Based on the fusion rules, the music features, occupant features, and vehicle operating condition features are fused to obtain fused features. Since the fused features are obtained based on multimodal feature fusion and the fusion rules are selected according to the extracted features, the fused features can fully and realistically represent different driving atmospheres. By effectively fusing heterogeneous data from multiple sources and intelligently adjusting the weight allocation, a lighting control command that best matches the current comprehensive situation is output. Thus, the cabin ambient lighting is controlled more intelligently and in a more realistic driving atmosphere based on the lighting control command.

[0073] In some embodiments, vehicle operating data is acquired in real time to obtain a vehicle operating data sequence. This sequence is then extracted to obtain a vehicle operating feature sequence, which is input into a vehicle driving atmosphere type recognition model to obtain lighting control commands. In other words, an end-to-end vehicle driving atmosphere type recognition model is trained to map time-series data from multimodal sensors to lighting control parameters without determining fusion rules. This end-to-end vehicle driving atmosphere type recognition model can be trained with a large amount of driving data, enabling more complex nonlinear relationship modeling and more accurate predictions.

[0074] In some embodiments, the sensor unit further includes a BCI (Brain-Machine Interface). In more futuristic visions, non-invasive electroencephalogram (EEG) sensors (such as EEG electrodes integrated in a headrest) can be integrated to directly acquire the driver's brain activity, thereby enabling a more fundamental and accurate assessment of states such as fatigue, distraction, and emotions.

[0075] In some embodiments, the sensor unit further includes an ambient odor sensor, such as an electronic nose sensor, for detecting in-vehicle air quality (such as CO2 concentration and VOCs), and for alerting the system by using a specific light color (such as green) when the air quality deteriorates, and for linking the air conditioning system.

[0076] In some embodiments, the cockpit domain controller (ECE) also communicates with a cloud server to synchronize the user's personalized preference model to the cloud. When the user changes vehicles or another family member drives, personalized configurations can be downloaded from the cloud. Simultaneously, manufacturers can continuously update and optimize fusion rules and vehicle driving atmosphere type recognition models through OTA (Over-The-Air) technology to continuously improve the user experience.

[0077] In some embodiments, vehicle operating condition characteristics include facility characteristics, which are obtained by extracting urban infrastructure operational information, acquired using V2X (Vehicle to Everything) technology. Because vehicle operating condition characteristics include facility characteristics, LED ambient lighting can be controlled more intelligently. For example, by identifying congested road sections in the city via V2X, the ambient lighting can be adjusted to a soothing mode in advance when the vehicle approaches a known congested section, helping the driver to calm down.

[0078] The present application will be described below through an embodiment: For scenario one: A family of three is returning from a weekend trip and is driving on the highway in the evening.

[0079] The cockpit domain controller ECU acquires the vehicle operating data set in scenario one at time T1; and performs feature extraction on the vehicle operating data set to obtain vehicle operating features. The vehicle's operational characteristics in Scenario 1 include: illumination characteristics, indicating that the illumination sensor collects 80 lux of light; occupancy and seating distribution characteristics, indicating that there are 3 people in the vehicle: the driver, the front passenger, and the rear passenger; age status labels, indicating that the driver and front passenger are both middle-aged, and the rear passenger is a child; emotional and facial expression characteristics, indicating that the driver is focused, and the mother and child are happy; heart rate characteristics, indicating that the driver's heart rate is 75 bpm; heart rate status label, indicating that the driver's heart rate is normal; respiratory rate characteristics, indicating that the driver breathes 16 times / minute; respiratory rate status label, indicating that the driver's respiratory rate is normal; conversation atmosphere characteristics, indicating that the conversation atmosphere is relaxed and cheerful; and vehicle speed, driving mode, navigation, and vehicle condition complexity characteristics, indicating that the vehicle is traveling at 100 km / h on a highway in "Comfort" mode, and the navigation shows that there are still 30 kilometers to the destination. Musical characteristics: The music type is a lively children's song with a BPM (Beat Per Minute) of 130.

[0080] Since vehicle operating characteristics represent the vehicle driving on the highway, emotional and facial features represent the driver being focused, conversation atmosphere features represent a relaxed and cheerful conversation atmosphere, and the type of music has a low correlation with the current highway driving situation, the preset fusion rules are determined to be used, namely, the fusion weight of occupant features is 0.6, the fusion weight of vehicle operating characteristics is 0.3, and the fusion weight of music features is 0.1.

[0081] Based on preset fusion rules, music features, occupant features, and vehicle operating condition features are fused to obtain fused features. These fused features are then input into the vehicle driving atmosphere type recognition model to generate lighting control commands. The RGB values ​​indicated by the lighting control commands are (255, 140, 0) to present a soft amber color, with a luminous brightness of 50%. The LED ambient lights on the dashboard and front door panels are statically lit, while the LED ambient lights on the rear door panels and roof adopt a slowly changing dynamic effect, such as flashing at a very low frequency, to cater to children's preferences and avoid affecting driving.

[0082] The lighting control command is broadcast to all LED control subunits, enabling each subunit to receive and parse the command. Based on the command, each subunit controls its assigned ambient LED to smoothly adjust to the specified operating parameters within 500ms.

[0083] If an emergency occurs at time T2 and the vehicle speed drops from 100km / h to 30km / h within 2 seconds, the ADAS system will issue a forward collision warning. The capacitive / pressure sensor will detect that the driver's heart rate instantly spikes to 120bpm, and the camera will capture the driver's expression of fear and tension.

[0084] The cockpit domain controller (ECU) acquires the vehicle operating data set at time T2 and generates vehicle operating characteristics based on this data set. A second fusion rule is determined based on music features, occupant features, and vehicle operating condition features. In this second fusion rule, the fusion weights for vehicle operating condition features and occupant features are both 0.5, while the fusion weight for music features is 0. That is, the weight of entertainment-related music features instantly drops to 0, and the fusion weights for vehicle operating condition features and occupant features are both 0.5. Based on the second fusion rule, the music features, occupant features, and vehicle operating condition features are fused to obtain fused features. These fused features are then input into the vehicle driving atmosphere type recognition model, which outputs a lighting control command. This command instructs all LED ambient lights to instantly switch to a high-brightness red and flash rapidly at a frequency of 2Hz, coordinating with other alarm systems, such as audible alarms, to create the strongest possible warning to the driver until the vehicle returns to a safe state.

[0085] In another scenario, if the driver exhibits signs of fatigue (such as decreased blinking frequency, changes in head posture, and decreased heart rate variability), the LED ambient lights will be controlled to alternately display high color temperature blue-white light and red light to remind the driver of fatigue driving. Compared to a single sound alarm, this multi-sensory stimulation can more effectively wake up the driver and is expected to reduce the rate of visual distraction caused by fatigue by 10%-15%.

[0086] In another scenario, if the driver becomes fatigued, the system will automatically switch to a refreshing cool-toned light, creating a warm and inviting atmosphere when the user is relaxing. In yet another scenario, when the ADAS system issues a collision warning or navigation prompt, it will control the LED ambient lights in the edge of the driver's field of vision to flash in a striking color, guiding the driver's surrounding vision, shortening reaction time, and improving driving safety.

[0087] In another scenario, based on data from an ambient light sensor, the brightness of LED ambient lights is intelligently adjusted, significantly optimizing energy efficiency. For example, the ambient lights automatically decrease or even turn off during the day when there is sufficient light, and smoothly increase in brightness when entering a tunnel or underground parking garage. Compared to systems that are always on or require manual adjustment, the average power consumption of the ambient lighting system is expected to be reduced by 30%-50% under comprehensive urban conditions.

[0088] See Figure 3 This application provides a cabin ambient lighting control device 20, which includes: Module 201 is used to acquire a set of vehicle working data. Feature extraction module 202 is used to extract features from the vehicle working data set to obtain vehicle working features, which include: music features, occupant features and vehicle working condition features. The feature fusion module 203 is used to determine the fusion rules based on music features, occupant features and vehicle operating condition features, and to fuse the music features, occupant features and vehicle operating condition features based on the fusion rules to obtain fused features; The instruction generation module 204 is used to input the fused features into the vehicle driving atmosphere type recognition model to generate lighting control instructions; The control module 205 is used to control the ambient lights in the vehicle cabin to operate according to the first working mode parameters indicated by the lighting control command, which is used to indicate at least one of the RGB values, illumination time, and illumination brightness of the ambient lights when they are working.

[0089] In one possible implementation, the feature fusion module 203 includes: The first feature fusion submodule is used to determine the first fusion rule when the occupant feature indicates that the driver is in a state of fatigue or inattentiveness. The fusion weight corresponding to the occupant feature in the first fusion rule is not less than the first threshold. The second feature fusion submodule is used to determine the second fusion rule when the complexity of the road conditions represented by the vehicle operating condition features is greater than the first complexity threshold. The fusion weight of the vehicle operating condition features in the second fusion rule is not less than the second threshold. The third feature fusion submodule is used to determine the third fusion rule when the complexity of the road conditions represented by the vehicle operating condition features is greater than the second complexity threshold. In the third fusion rule, the fusion weight of the vehicle operating condition features is not less than the third threshold, and the third threshold is greater than the second threshold.

[0090] In one possible implementation, the instruction generation module 204 is specifically used for: The fused features are input into the vehicle driving atmosphere type recognition model to obtain the vehicle driving atmosphere type; Based on the correspondence table between vehicle driving atmosphere type and lighting control command, determine the lighting control command corresponding to the vehicle driving atmosphere type.

[0091] In one possible implementation, device 20 further includes: The first module is used to receive user input commands. The second module is used to control the ambient light to operate according to the second working mode parameters indicated by the user's control commands.

[0092] In one possible implementation, device 20 further includes: The third module is used to obtain the user's historical control command set if the user control command is received within a preset time after the ambient light in the vehicle cabin is operated according to the first working mode parameters indicated by the light control command. The fourth module is used to determine the target control command and the number of target control commands that are similar to the user's control command in the user's historical control command set. The target control command is used to control the ambient light to switch from the third working mode parameter to the fourth working mode parameter. The third working mode parameter of the ambient light corresponding to the target control command is similar to the first working mode parameter, and the fourth working mode parameter of the ambient light corresponding to the target control command is similar to the second working mode parameter. The fifth module is used to modify the fusion weight in the current fusion rule, replace the first working mode parameter in the relationship correspondence table with the second working mode parameter, or modify the model parameters of the vehicle driving atmosphere type recognition model when the number of target control commands exceeds a preset threshold.

[0093] This device extracts music features, occupant features, and vehicle operating condition features from the vehicle's operating data set, selects corresponding fusion rules, and then fuses these features based on the fusion rules to obtain fused features. Since the fused features are obtained based on multimodal feature fusion and the fusion rules are selected according to the extracted features, the fused features can fully and realistically represent different driving atmospheres. This makes the working parameters indicated by the lighting control commands identified by the subsequent vehicle driving atmosphere type recognition model more consistent with the real situation, thereby enabling more intelligent and realistic control of the cabin ambient lighting based on the lighting control commands.

[0094] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0095] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-described cabin ambient lighting control methods.

[0096] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0097] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the above-described cabin ambient lighting control methods.

[0098] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the environmental images of the vehicle's surroundings, road preprocessing and recognition results, road matching results, first duration, second duration, and control strategy involved in this application were all obtained with full authorization.

[0099] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0100] It should be noted that the terms "first," "second," etc. (if applicable) in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0101] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for controlling cabin ambient lighting, characterized in that, The method includes: Obtain the vehicle's operational data set; Feature extraction is performed on the vehicle operating data set to obtain vehicle operating features, which include: music features, occupant features, and vehicle operating condition features. A fusion rule is determined based on the music features, occupant features, and vehicle operating condition features, and the music features, occupant features, and vehicle operating condition features are fused based on the fusion rule to obtain fused features; The fused features are input into the vehicle driving atmosphere type recognition model to generate lighting control commands; Based on the lighting control command, the ambient lights in the vehicle cabin are controlled to operate according to the first working mode parameters indicated by the lighting control command. The first working mode parameters include at least one of RGB value, illumination time, and illumination brightness.

2. The method according to claim 1, characterized in that, The step of determining the fusion rule based on the music features, the occupant features, and the vehicle operating condition features includes: When the occupant features indicate that the driver is in a state of tension, fatigue, or inattention, a first fusion rule is determined, wherein the fusion weight corresponding to the occupant features in the first fusion rule is not less than a first threshold. If the complexity of the road conditions represented by the vehicle operating condition features is greater than a first complexity threshold, a second fusion rule is determined, wherein the fusion weight of the vehicle operating condition features in the second fusion rule is not less than the second threshold. If the complexity of the road conditions represented by the vehicle operating condition features is greater than the second complexity threshold, a third fusion rule is determined. In the third fusion rule, the fusion weight of the vehicle operating condition features is not less than the third threshold, and the third threshold is greater than the second threshold.

3. The method according to claim 1, characterized in that, The step of inputting the fused features into the vehicle driving atmosphere type recognition model to generate lighting control commands includes: The fused features are input into the vehicle driving atmosphere type recognition model to obtain the vehicle driving atmosphere type; Based on the correspondence table between vehicle driving atmosphere type and lighting control command, determine the lighting control command corresponding to the vehicle driving atmosphere type.

4. The method according to claim 3, characterized in that, After controlling the ambient lighting in the cockpit to operate according to the first operating mode parameters indicated by the lighting control command, the method further includes: Receive user input commands; Based on the user control command, the ambient light is controlled to operate according to the second operating mode parameters indicated by the user control command.

5. The method according to claim 4, characterized in that, The method further includes: After controlling the ambient light to operate according to the first working mode parameters indicated by the light control command, if the user control command is received within a preset time, the user's historical control command set is obtained. In the set of user history control commands, a target control command similar to the user control command and the number of the target control commands are determined. The target control command is used to control the ambient light to switch from a third working mode parameter to a fourth working mode parameter. The third working mode parameter corresponding to the target control command is similar to the first working mode parameter, and the fourth working mode parameter corresponding to the target control command is similar to the second working mode parameter. If the number of target control commands exceeds a preset threshold, modify the fusion weight in the current fusion rule, replace the first working mode parameter in the relationship correspondence table with the second working mode parameter, or modify the model parameters of the vehicle driving atmosphere type recognition model.

6. A cabin ambient lighting control device, characterized in that, The device includes: The acquisition module is used to acquire a set of vehicle operating data. The feature extraction module is used to extract features from the vehicle operating data set to obtain vehicle operating features, which include: music features, occupant features and vehicle operating condition features. The feature fusion module is used to determine fusion rules based on the music features, the occupant features, and the vehicle operating condition features, and to fuse the music features, the occupant features, and the vehicle operating condition features based on the fusion rules to obtain fused features; The instruction generation module is used to input the fused features into the vehicle driving atmosphere type recognition model to generate lighting control instructions; The control module is used to control the ambient lights in the vehicle cabin to operate according to the first working mode parameters indicated by the lighting control command, wherein the lighting control command is used to indicate at least one of the RGB value, illumination time and illumination brightness of the ambient lights when they are working.

7. The apparatus according to claim 6, characterized in that, The feature fusion module includes: The first feature fusion submodule is used to determine a first fusion rule when the occupant feature indicates that the driver is fatigued or inattentive, wherein the fusion weight corresponding to the occupant feature in the first fusion rule is not less than a first threshold. The second feature fusion submodule is used to determine a second fusion rule when the complexity of the road conditions represented by the vehicle operating condition feature is greater than a first complexity threshold, wherein the fusion weight of the vehicle operating condition feature in the second fusion rule is not less than a second threshold. The third feature fusion submodule is used to determine a third fusion rule when the complexity of the road conditions represented by the vehicle operating condition feature is greater than the second complexity threshold. The fusion weight of the vehicle operating condition feature in the third fusion rule is not less than the third threshold, and the third threshold is greater than the second threshold.

8. The apparatus according to claim 6, characterized in that, The instruction generation module is specifically used for: The fused features are input into the vehicle driving atmosphere type recognition model to obtain the vehicle driving atmosphere type; Based on the correspondence table between vehicle driving atmosphere type and lighting control command, determine the lighting control command corresponding to the vehicle driving atmosphere type.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the steps of the method according to any one of claims 1-5.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-5.

Images