Vehicle intelligent control method and system based on multi-passenger emotion recognition and vehicle

Abstract

The application discloses a kind of vehicle intelligent control method, system and vehicle based on multiple passenger emotion recognition, comprising: based on in-vehicle environment perception data, vehicle driving state data and the passenger state data of the ith passenger, the emotion label of the ith passenger is identified and corresponding emotion intensity;Based on the emotion label of the ith passenger, the expected environment parameter set of the ith passenger is determined;Based on whether there is emotional demand conflict between n passengers, determine the globally optimal compromise environment parameter set or determine the globally optimal environment parameter set;Again based on whether the vehicle supports the partition adjustment of seat position, determine whether the demand environment parameter set of the ith passenger will be compensated;Based on the above environment parameter set, corresponding function control instruction is generated;Based on function control instruction, vehicle intelligent control is carried out.The application can maximize the emotional and comfort needs of different passengers in limited in-vehicle space, and optimize the ride experience.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a vehicle intelligent control method, system, and vehicle based on multi-occupant emotion recognition. Background Technology

[0002] In the field of autonomous driving, current technologies have enabled vehicles to perform autonomous navigation, decision-making, and environmental perception. These fundamental technologies support the stable operation of autonomous driving systems under various road conditions. Environmental perception technology collects external information through devices such as radar, cameras, and sensors, while autonomous navigation and decision-making technologies rely on complex algorithms to process this information, ensuring safe vehicle operation. Furthermore, Human-Machine Interaction (HCI) technology also plays a crucial role in autonomous driving systems, enabling effective communication between humans and machines through graphical user interfaces (GUIs), speech recognition, and speech synthesis.

[0003] While existing technologies have made significant progress in the aforementioned areas, such as accurately identifying occupants' emotional states and appropriately adjusting vehicle environmental parameters accordingly, some key shortcomings and deficiencies remain. For instance, current autonomous driving systems typically assume only one occupant or that all occupants have consistent emotional states when considering their emotional states and psychological needs. However, in real-world shared mobility, family travel, or multi-occupant commuting scenarios, different occupants (such as the driver in the takeover position versus rear passengers, or rear passengers of different ages) often have drastically different emotional states and psychological needs. For example, one occupant might be anxious due to time constraints and desire a fast-moving vehicle with upbeat music, while another might be tired and stressed and desire a quiet environment and a smooth ride. Current technologies lack effective conflict identification and resolution mechanisms when facing such conflicts in the emotional states of multiple occupants, failing to balance the differentiated needs of different occupants. This could lead to a sharp decline in the experience for some occupants and even trigger interpersonal conflicts within the vehicle. Therefore, how to intelligently handle the conflicting emotional needs of multiple passengers is a technical bottleneck that urgently needs to be overcome in the field of autonomous driving emotional interaction. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this application is to provide a vehicle intelligent control method, system and vehicle based on multi-occupant emotion recognition, so as to maximize the satisfaction of the emotional and comfort needs of different occupants in a limited vehicle space and optimize the riding experience.

[0005] Firstly, this application provides a vehicle intelligent control method based on multi-occupant emotion recognition, including:

[0006] Based on in-vehicle environmental perception data, vehicle driving status data, and the occupant status data of the i-th occupant, the emotion label and corresponding emotion intensity of the i-th occupant are identified. Here, i takes any integer from 1 to n, and n represents the total number of occupants in the vehicle.

[0007] Based on the emotion label of the i-th occupant, the expected set of environmental parameters for the i-th occupant is determined. The elements in the set of environmental parameters (i.e., environmental parameters) include air conditioning temperature, in-vehicle lighting intensity, music type, music volume, and driving style (i.e., vehicle driving style).

[0008] If there is an emotional conflict among n occupants, the optimal compromise environmental parameter set is determined based on the expected environmental parameter set of each occupant. Then, it is determined whether the vehicle supports zoned adjustment based on seat position and whether the i-th occupant meets the compensation trigger condition. If so, the environmental parameter zoned compensation for the i-th occupant is performed to obtain the required environmental parameter set of the i-th occupant after zoned compensation.

[0009] If there is no conflict of emotional needs among the n passengers, then the globally optimal set of environmental parameters is determined based on the expected set of environmental parameters for each passenger.

[0010] Based on the globally optimal compromise environmental parameter set, or the environmental parameter set required by each occupant, or the globally optimal environmental parameter set, corresponding functional control instructions are generated.

[0011] Intelligent vehicle control is performed based on the aforementioned function control commands.

[0012] Based on the identified emotion tags of each occupant, personalized desired environmental parameters are matched, covering multiple in-vehicle control dimensions such as air conditioning temperature, lighting, audio-visual systems, and driving style. This comprehensively meets the personalized needs of occupants for comfort, relaxation, and calmness under different emotions. A conflict-based logic for occupant emotional needs is added. When conflicting needs exist, the system first calculates the globally optimal compromise set of environmental parameters for the entire vehicle, taking into account the basic experience of all occupants. If the vehicle supports zoned seat adjustment and the occupant meets the compensation trigger conditions, then targeted zone parameter compensation is executed, balancing the unified control of the entire vehicle with personalized compensation for each individual, significantly reducing the discomfort caused by conflicts. When there are no conflicting emotional needs, a unified globally optimal set of environmental parameters is directly generated, simplifying the calculation logic, shortening the command generation response time, and improving the response speed of in-vehicle comfort adjustments. Based on whether there is a conflict and whether zone compensation is used, corresponding standard function control commands are output, uniformly driving the intelligent control of the vehicle's air conditioning, lighting, audio-visual systems, and powertrain chassis, achieving a closed-loop linkage from emotion recognition to in-vehicle hardware execution.

[0013] Optionally, methods for identifying the emotion label and corresponding emotion intensity of the i-th occupant include:

[0014] Acquire in-vehicle environment perception data, vehicle driving status data, and occupant status data of the i-th occupant.

[0015] The in-vehicle environment perception data, vehicle driving status data, and occupant status data of the i-th occupant are combined to form a comprehensive time-series dataset of the i-th occupant.

[0016] The comprehensive time-series dataset of the i-th passenger is input into a pre-trained emotion recognition model, and the comprehensive analysis outputs the emotion label and corresponding emotion intensity of the i-th passenger.

[0017] By combining in-vehicle environmental perception data, vehicle driving status data, and the occupant status data of the i-th occupant, a comprehensive time-series dataset for the i-th occupant is formed. This dataset retains the continuous characteristics of data changes over time and can capture dynamic changes such as gradual and instantaneous fluctuations in occupant emotions. Based on a pre-trained emotion recognition model, the comprehensive time-series dataset is uniformly extrapolated and calculated, automatically outputting standardized emotion labels and quantified emotion intensity. This achieves simultaneous qualitative judgment of emotion type and quantitative representation of emotion intensity. Using individual, independent time-series datasets as model input, the data for each occupant does not interfere with each other, allowing for parallel analysis of the emotions of all occupants in the vehicle. This accurately distinguishes the differentiated emotional states of occupants in different seats, providing precise quantitative input for subsequent personalized in-vehicle control based on occupant characteristics.

[0018] Optionally, the in-vehicle environment perception data includes in-vehicle temperature time series, humidity time series, and light intensity time series; the vehicle driving status data is vehicle speed time series; and the occupant status data of the i-th occupant includes the i-th occupant's voice data time series, facial expression data time series, and heart rate variability data time series. This multi-dimensional time series data comprehensively covers the in-vehicle environment, driving conditions, occupant voice, facial features, and physiological characteristics. The multi-source information corroborates each other, avoiding the problems of susceptibility to interference and high misjudgment rates associated with single-signal emotion recognition, and enriching the basis for emotion judgment.

[0019] The method for comprehensively analyzing and outputting the emotion label and corresponding emotion intensity of the i-th passenger includes:

[0020] A feature vector set is extracted from the comprehensive time-series dataset of the i-th occupant. This feature vector set contains seven feature vectors: in-vehicle temperature, in-vehicle humidity, in-vehicle light intensity, vehicle speed, voice data, facial expression data, and heart rate variability.

[0021] By processing all feature vectors in the feature vector set using LSTM, seven output vectors encoding the emotional state of the i-th passenger are obtained.

[0022] The seven output vectors are fused into an output matrix for predicting the probability of emotional states.

[0023] Based on the output matrix, the probabilities of various emotional states of the i-th passenger are predicted using a fully connected layer and a softmax activation function.

[0024] The emotional state with the highest probability is taken as the emotional label of the i-th passenger, and the highest probability (i.e. the probability corresponding to the emotional label of the i-th passenger) is taken as the corresponding emotional intensity.

[0025] Feature vectors are extracted independently from seven data sources, preserving the temporal variation features of temperature, humidity, light intensity, vehicle speed, voice, face, and heart rate variability, thus avoiding the loss of effective information due to the mixing of different feature types. An LSTM network is used to temporally encode the seven feature vectors, fully exploring the correlation patterns of data evolution over time and accurately capturing dynamic processes such as the slow accumulation and sudden changes in emotions, improving the effectiveness of temporal feature extraction. The output vectors of the seven LSTMs are fused to construct a unified (predictive) output matrix, achieving deep coupling of environmental, driving, and passenger physiological behavior features, and comprehensively integrating multi-dimensional implicit emotional correlation information. By relying on fully connected layers and a softmax activation function to output normalized probability distributions of various emotions, quantitative probability calculation of emotional states is achieved, with simultaneous qualitative and quantitative output. The emotional state corresponding to the highest probability is selected as the emotion label, and the highest probability value represents the emotion intensity, unifying the quantitative judgment standard.

[0026] Optionally, the desired set of environmental parameters for the i-th occupant can be determined as follows:

[0027] Based on the emotion tag of the i-th occupant, a pre-defined table mapping emotion tags to desired environmental parameter sets is consulted to obtain the desired environmental parameter set for the i-th occupant. This table lookup and matching is performed automatically, eliminating the need for complex real-time iterative calculations. The vehicle controller can quickly retrieve target parameters, reducing the latency of generating vehicle environment control commands and improving response speed.

[0028] Optionally, the method to determine whether there is a conflict of emotional needs among the n passengers is as follows:

[0029] The emotion labels of n occupants and their corresponding emotion intensities are combined to form a multi-occupant emotion mapping matrix.

[0030] Traverse the emotion tag pairs of any two passengers in the multi-passenger emotion mapping matrix. If there is a conflict as defined in the preset conflict rule base, it is determined that there is an emotional need conflict among the n passengers; otherwise, it is determined that there is no emotional need conflict among the n passengers.

[0031] A standardized multi-occupant emotion mapping matrix is ​​constructed by integrating the emotion tags and intensity of all occupants. This unified collection of quantitative emotion information for all occupants enables structured storage of multi-occupant emotion data, facilitating batch traversal and comparison, and improving the efficiency of conflict judgment and handling. A verification logic that iterates through the pairwise emotion tag pairs within the multi-occupant emotion mapping matrix fully covers the comparison relationships of the needs of any two occupants in the vehicle, eliminating any potential conflict issues and avoiding the problem of missed judgments of local conflicts. Based on a pre-set conflict rule base as the judgment benchmark, standardized rules define the environmental parameter requirement conflict scenarios corresponding to various emotions, ensuring unified conflict judgment standards and reproducible results.

[0032] Optionally, methods for determining the globally optimal set of environmental parameters based on the expected set of environmental parameters for each occupant include:

[0033] Determine the global optimal air conditioning temperature Global optimal in-vehicle lighting intensity Global optimal music volume .

[0034] Determine the globally optimal music type and globally optimal driving style .

[0035] Set the global optimal air conditioning temperature Global optimal in-vehicle lighting intensity Globally optimal music type Global optimal music volume Global optimal driving style Combined, the globally optimal set of environmental parameters is formed. .

[0036] Determine the global optimal air conditioning temperature Global optimal in-vehicle lighting intensity Global optimal music volume The methods include:

[0037] Will , , Multiply to obtain the global adjustment base weight for the i-th passenger. .in, This represents the intensity of the emotion corresponding to the emotion label of the i-th passenger. This represents the position weight factor of the i-th passenger. This represents the activity state factor of the i-th crew member.

[0038] Based on the global adjustment base weights of n crew members, calculate the normalized weight coefficient of the i-th crew member. .

[0039] based on right , , By performing weighted summation for each of the n passengers, the globally optimal theoretical air conditioning temperature can be obtained. Global optimal theoretical in-vehicle illumination intensity Global optimal theoretical music volume ;in, This represents the desired air conditioning temperature for the i-th passenger. This represents the desired interior light intensity for the i-th occupant. This represents the desired music volume for the i-th passenger.

[0040] right , , By performing amplitude limiting, the globally optimal air conditioning temperature is obtained. Global optimal in-vehicle lighting intensity Global optimal music volume .

[0041] The weight normalization process is completed by summing the global adjustment base weights of all crew members, resulting in the dimensionless normalized weight coefficient of the i-th crew member. This ensures a unified weighted calculation benchmark for multiple occupants. Based on normalized weighting coefficients, the desired temperature, lighting, and volume for each occupant are weighted and summed to output theoretically optimal global parameters. This automatically balances the differentiated comfort needs of multiple occupants, yielding a control benchmark value that considers all occupants. A limiting function is introduced to constrain the theoretical parameters, setting a maximum single adjustment difference centered on the current actual in-vehicle state. This limits the single adjustment range of air conditioning, lighting, and volume, preventing sudden parameter changes from causing abrupt changes in driving and riding sensations and frequent large-scale movements of hardware actuators, thus reducing the impact on in-vehicle comfort and equipment wear.

[0042] Determine the globally optimal music type The methods include:

[0043] based on , By performing a weighted summation of the data from the n crew members, we can obtain the j-th music genre. Weighted votes Where j takes any integer from 1 to r, and r represents the total number of music genres. This represents the desired music type for the i-th passenger. Indicates reaction and The indicator function of the relationship, when the desired music type of the i-th occupant is the j-th music type. (Right now )hour, When the desired music type of the i-th passenger is not the j-th music type. (Right now )hour, .

[0044] The music genre with the highest weighted votes is selected as the globally optimal music genre. .

[0045] via indicator function It accurately distinguishes whether the passenger's desired music type matches the currently counted music type. If they match, the desired music type is included in the weighted score; otherwise, it is not counted. This allows for efficient weighted score aggregation of various music types, with simple computational logic and low onboard computing power consumption. It iterates through all r music types and calculates weighted votes for each, fully covering all available music types without missing any candidate solutions. The music type with the highest score is selected as the globally optimal music type, achieving a weighted balance decision on music needs for multiple passengers and effectively resolving conflicts in music type preferences among different passengers.

[0046] Determine the globally optimal driving style The methods include:

[0047] The desired driving style of the i-th occupant Normalized weighting coefficients of the i-th crew member Combine them to form the i-th tuple.

[0048] Sort the n tuples in ascending order of their normalized weight coefficients to form a style weight sequence.

[0049] Starting from the first tuple in the style weight sequence, normalized weight coefficients are accumulated to obtain the accumulated weight corresponding to the k-th tuple. , .

[0050] The cumulative weight The desired driving style at the position where the value first exceeds 0.5 is taken as the globally optimal driving style. .

[0051] Each passenger's desired driving style and normalized weight coefficient are encapsulated as independent tuples. All tuples are then arranged in ascending order of weight to construct a style weight sequence, statistically analyzed sequentially from low to high weight. This prioritizes passenger needs and prevents high-weight passenger demands from being interfered with by lower-weight data. The cumulative weight of each segment of the sequence is calculated, with the first occurrence of a cumulative weight exceeding 0.5 serving as the decision threshold. A majority-weight judgment mechanism ensures that the selected driving style represents more than half of the weighted demands within the vehicle, taking into account the driving and riding preferences of most passengers. Based on the cumulative weight, the globally optimal driving style is selected, prioritizing the driving needs of passengers with high emotional intensity, important positions, and active behaviors, effectively balancing the conflicts between different driving styles such as aggressive, stable, and relaxed.

[0052] Optionally, methods for determining the globally optimal compromise set of environmental parameters based on the expected set of environmental parameters for each occupant include:

[0053] Determine the globally optimal compromise air conditioning temperature Global optimal compromise in-vehicle light intensity Global optimal compromise music volume .

[0054] Determine the globally optimal compromise music type and globally optimal compromise driving style .

[0055] The globally optimal compromise air conditioning temperature Global optimal compromise in-vehicle light intensity Globally optimal compromise music type Global optimal compromise music volume Global optimal compromise driving style Combined, the set of globally optimal compromise environmental parameters is formed. .

[0056] Determine the globally optimal compromise air conditioning temperature Global optimal compromise in-vehicle light intensity Global optimal compromise music volume The methods include:

[0057] Will , , , Multiplying these together yields the global compromise adjustment base weight for the i-th passenger. ;in, This represents the intensity of the emotion corresponding to the emotion label of the i-th passenger. This represents the emotion label factor of the i-th passenger. This represents the position weight factor of the i-th passenger. This represents the passenger type factor of the i-th passenger.

[0058] Based on the global adjustment base weights of n crew members, calculate the normalized compromise weight coefficient of the i-th crew member. .

[0059] based on right , , By performing weighted summation for each of the n passengers, the globally optimal compromise theoretical air conditioning temperature can be obtained. Global optimal compromise theory of in-vehicle light intensity Global optimal compromise theory for music volume ;in, This represents the desired air conditioning temperature for the i-th passenger. This represents the desired interior light intensity for the i-th occupant. This represents the desired music volume for the i-th passenger.

[0060] right , , By performing amplitude limiting, the globally optimal compromise air conditioning temperature is obtained. Global optimal compromise in-vehicle light intensity Global optimal compromise music volume .

[0061] Based on a normalized compromise weighting coefficient, the desired temperature, lighting, and volume for each occupant are weighted and summed to output compromise theoretical environmental parameters. This approach balances and integrates the individualized demands of all occupants when their emotional needs conflict, outputting a compromise adjustment baseline value that considers all occupants. A clip limiting function is used to constrain the maximum adjustment range in a single instance, centered on the current actual state inside the vehicle. This limits sudden changes in temperature, lighting, and volume, preventing discomfort caused by drastic changes in the in-vehicle environment and reducing wear and tear from frequent large-scale adjustments by the onboard actuators.

[0062] Determine the globally optimal compromise music type The methods include:

[0063] based on , Perform a weighted summation of the data from the n crew members to determine the choice of the j-th music genre. Groups that are not moderate Where j takes any integer from 1 to r, and r represents the total number of music genres. This represents the desired music type for the i-th passenger. express and Emotional distance.

[0064] The music type with the smallest group discomfort is selected as the globally optimal compromise music type. .

[0065] The system integrates a weighted average of normalized compromise coefficients specific to conflict scenarios to calculate group discomfort. This approach assigns a higher percentage of impact to passengers with high negative emotional intensity, high location priority, or special passenger types. Compromise decisions prioritize reducing discomfort for core passengers and adapt to conflicting emotional needs of multiple passengers. It iterates through all r-type music candidate types, calculating the overall discomfort cost for each, covering all possible solutions and avoiding omission of the optimal compromise music type. The music type corresponding to the minimum group discomfort is selected as the compromise music type, aiming to achieve conflict equilibrium with the lowest overall emotional loss across the entire vehicle.

[0066] Determine the globally optimal compromise driving style The methods include:

[0067] The desired driving style of the i-th occupant Normalized compromise weighting coefficients with the i-th passenger Combine them to form the i-th tuple.

[0068] Sort the n tuples in ascending order of their normalized compromise weight coefficients to form a style compromise weight sequence.

[0069] Starting from the first tuple in the style compromise weight sequence, the normalized compromise weight coefficients are accumulated to obtain the cumulative compromise weight corresponding to the k-th tuple. , .

[0070] The compromise cumulative weight The desired driving style at the position where the value first exceeds 0.5 is taken as the globally optimal compromise driving style. .

[0071] Each occupant's desired driving style and a normalized compromise weight coefficient specific to the conflict scenario are encapsulated as independent tuples. All tuples are arranged in ascending order of compromise weight to construct a sequence, sequentially aggregating occupant demands from low-weight to high-weight, ensuring that the emotional demands of high-priority occupants dominate the cumulative statistics. The compromise cumulative weight is obtained by successively accumulating the weights within the weight sequence. A threshold is set when the compromise cumulative weight first exceeds 0.5, ensuring that the majority of weighted demands dominate the decision-making process. This filters out driving styles that can accommodate the preferences of most weighted occupants in the vehicle, achieving a balanced compromise in multi-occupant driving demand conflict scenarios.

[0072] Optionally, if the emotional intensity corresponding to the emotional label of the i-th passenger is greater than a preset emotional intensity threshold, and meets any one of conditions one through four, then the i-th passenger is determined to meet the compensation trigger condition.

[0073] Condition 1 is: , , This indicates the preset temperature deviation threshold;

[0074] Condition two is: , , This indicates the preset light intensity deviation threshold;

[0075] Condition three is: , , This indicates the preset music volume deviation threshold;

[0076] Condition four is: .

[0077] A two-tiered judgment threshold is set up. First, occupants with significant emotional discomfort are screened based on an emotional intensity threshold. Only occupants with negative emotions reaching a certain level are eligible for compensation, avoiding meaningless zoning compensation for occupants with stable emotions, reducing frequent operation of the in-vehicle zoning adjustment hardware, and lowering energy consumption and structural wear. Four deviation judgment conditions are set from four dimensions of the in-vehicle environment: temperature, light, volume, and music type. These conditions quantify the difference between the occupant's expected environmental parameters and the global environmental parameters of the compromise: if any of the following conditions are met, such as temperature deviation exceeding the threshold, light deviation exceeding the threshold, volume deviation exceeding the threshold, or the compromise music not matching the occupant's preferred music style, it is determined that the occupant's comfort needs have not been met by the global compromise solution. The judgment logic covers all core dimensions of driving and riding comfort, ensuring no comfort needs conflicts are missed. The difference is calculated... , , The system intuitively quantifies the gap between an individual's expectations and the compromise parameters of the entire vehicle, and uses independent preset thresholds for each dimension to achieve standardized quantitative judgment.

[0078] Optionally, the method for performing environmental parameter partition compensation for the i-th occupant and obtaining the set of required environmental parameters for the i-th occupant after partition compensation includes:

[0079] like Based on , , Determine the required air conditioning temperature for the i-th passenger. ;like Then the required air conditioning temperature for the i-th passenger will be determined. ;in, This indicates the preset temperature compensation coefficient.

[0080] like This will make the required light intensity for the i-th occupant... ;like This will make the required light intensity for the i-th occupant... .

[0081] like Then make based on , , Determine the music volume required by the i-th passenger. ;like Then the music volume required by the i-th passenger will be determined. ;in, This indicates the preset volume compensation coefficient.

[0082] The music type required by the i-th occupant The driving style required by the i-th occupant .

[0083] The desired air conditioning temperature for the i-th passenger. Required light intensity Music type required Required music volume Driving style requirements Combined, these parameters form the set of required environmental parameters for the i-th occupant after the partition compensation. .

[0084] Differentiated one-way compensation logic is set for temperature, light, and volume. Compensation adjustment is only activated when there is a large difference between the occupant's personal desired environmental parameters and the global optimal compromise environmental parameters. This avoids excessive contrast in the environmental zones of the whole vehicle caused by indiscriminate two-way compensation, and takes into account both the consistency of the basic compromise environment of the whole vehicle and the individual personalized comfort needs of each person.

[0085] Temperature and volume are compensated with independent coefficients. , Gradient compensation is implemented to achieve smooth gradient compensation, preventing sudden changes in zone temperature and volume from causing secondary discomfort to other occupants, while also reducing wear and tear caused by frequent and large adjustments in the air conditioning and audio systems. In terms of lighting, when there are deviations in demand, the lighting intensity is directly matched to the occupants' ideal lighting level. Independent adjustment of lighting zones minimizes interference with other occupants, maximizing the mitigation of negative emotions caused by lighting discomfort. Music genres are directly adopted based on the occupants' desired style, and independent audio-visual output for each zone completely resolves genre conflicts. A unified driving style is adopted, following a globally optimal compromise driving style. Since chassis driving control cannot be executed independently by seat zone, a unified vehicle driving strategy is implemented to avoid conflicts between multiple driving commands that could disrupt chassis control logic, ensuring driving safety and stability.

[0086] Optionally, based on the location of the i-th passenger, a preset correspondence table between location and location weight factor is consulted to obtain the location weight factor of the i-th passenger. Based on the activity behavior state of the i-th occupant, a pre-defined correspondence table between activity behavior states and activity state factors is consulted to obtain the activity state factor of the i-th occupant. Based on the emotion label of the i-th passenger, a pre-defined correspondence table of emotion labels and emotion label factors is consulted to obtain the emotion label factor of the i-th passenger. Based on the occupant type of the i-th occupant, query the preset correspondence table between occupant types and occupant type factors to obtain the occupant type factor of the i-th occupant. Independent mapping tables are established for four key influencing dimensions: occupant location, occupant activity behavior, emotional labels, and occupant type. Each dimension factor can be configured independently without interfering with each other, which facilitates iterative optimization of individual mapping rules based on vehicle type and driver / passenger characteristics, and makes system expansion and maintenance convenient.

[0087] Secondly, this application provides a vehicle intelligent control system based on multi-occupant emotion recognition, including a memory and a processor. The memory stores a computer program, and the processor is configured to execute the computer program to implement the above-mentioned vehicle intelligent control method based on multi-occupant emotion recognition.

[0088] Thirdly, this application provides a vehicle that includes the aforementioned vehicle intelligent control system based on multi-occupant emotion recognition.

[0089] This application establishes an emotion mapping matrix based on occupant position (seat position), utilizes multimodal fusion technology to identify the individual emotions of different occupants in the vehicle, and introduces a conflict resolution strategy based on a preset conflict rule base. When a conflict in emotional needs among occupants is detected, the system can make decisions based on safety priority, majority voting, or the preferences of the disadvantaged group, and, with the support of vehicle hardware, implement zoned environmental adjustment. This allows the system to maximize the satisfaction of the emotional and comfort needs of different occupants within the limited in-vehicle space, thereby optimizing the riding experience. Attached Figure Description

[0090] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments of this application will be described below.

[0091] Figure 1 This is a schematic diagram of the vehicle in an embodiment of this application.

[0092] Figure 2 This is a flowchart of the vehicle intelligent control method based on multi-occupant emotion recognition in the embodiments of this application.

[0093] Figure 3 This is a flowchart illustrating the method for identifying the emotion label and corresponding emotion intensity of the i-th passenger in this application embodiment.

[0094] Figure 4 This is a flowchart illustrating the method for comprehensively analyzing and outputting the emotion label and corresponding emotion intensity of the i-th passenger in this embodiment of the application.

[0095] Figure 5 This is a flowchart illustrating the method for determining the globally optimal set of environmental parameters in this application embodiment.

[0096] Figure 6 This is a flowchart illustrating the method for determining the globally optimal compromise set of environmental parameters in this application embodiment. Detailed Implementation

[0097] The embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The detailed descriptions and drawings of the following embodiments are used to exemplify the principles of this application, but should not be used to limit the scope of this application; that is, this application is not limited to the described embodiments. Figure 1 As shown, Figure 1 This is a schematic diagram of a vehicle in an embodiment of this application. The vehicle may be, but is not limited to, a pure electric vehicle (PEV / BEV), a hybrid electric vehicle (HEV), a range-extended electric vehicle (REEV), a plug-in hybrid electric vehicle (PHEV), or a new energy vehicle.

[0098] like Figure 1 As shown, the vehicle in this embodiment includes a vehicle intelligent control system based on multi-occupant emotion recognition. The vehicle intelligent control system based on multi-occupant emotion recognition in this embodiment includes a memory and a processor. The memory stores a computer program, and the processor is configured to execute the computer program to implement the vehicle intelligent control method based on multi-occupant emotion recognition in this embodiment.

[0099] like Figure 2 As shown, the vehicle intelligent control method based on multi-occupant emotion recognition in this application embodiment includes the following steps:

[0100] S1. Based on in-vehicle environmental perception data, vehicle driving status data, and the occupant status data of the i-th occupant, identify the emotion label and corresponding emotion intensity of the i-th occupant, and then execute S2. Here, i takes any integer from 1 to n, and n represents the total number of occupants in the vehicle. For example, n = 4 occupants.

[0101] like Figure 3As shown, in one possible embodiment, the method for identifying the emotion label and corresponding emotion intensity of the i-th passenger includes the following steps:

[0102] S101. Obtain in-vehicle environment perception data, vehicle driving status data, and occupant status data of the i-th occupant.

[0103] In one possible embodiment, the in-vehicle environment perception data includes in-vehicle temperature time series, humidity time series, and light intensity time series; the vehicle driving status data is the vehicle speed time series; the occupant status data of the i-th occupant includes the i-th occupant's voice data time series, facial expression data time series, and heart rate variability data time series.

[0104] As an example, the timing of collecting the voice data of the i-th occupant using an audio input device. :

[0105] .

[0106] in, Indicates in The voice sample of the i-th occupant and its timestamp collected at each moment.

[0107] As an example, the timing of capturing the facial expression data of the i-th occupant using a video camera. :

[0108] .

[0109] in, Indicates in The facial expression image of the i-th occupant captured at each moment, along with its timestamp.

[0110] As an example, a heart rate variability monitor is used to comprehensively assess the emotional state of passengers and collect time-series heart rate variability data. :

[0111] .

[0112] in, Indicates in The heart rate variability value of the i-th passenger measured at time t and its timestamp.

[0113] As an example, environmental sensors are used to collect in-vehicle environmental perception data, including in-vehicle temperature time series. Humidity time series Light intensity time series .

[0114] ;

[0115] ;

[0116] ;

[0117] in, , , They represent in The system continuously collects data on the vehicle's interior temperature and timestamp, humidity and timestamp, and light intensity and timestamp.

[0118] As an example, obtain the vehicle speed timing sequence. :

[0119] ;

[0120] in, Indicates in The vehicle speed and its timestamp are collected at all times.

[0121] S102. Combine the in-vehicle environment perception data, vehicle driving status data, and the occupant status data of the i-th occupant to form a comprehensive time-series dataset of the i-th occupant.

[0122] As an example, the i-th occupant integrated time-series dataset :

[0123] .

[0124] S103. Input the comprehensive time-series dataset of the i-th occupant into the pre-trained emotion recognition model, and comprehensively analyze and output the emotion label and corresponding emotion intensity of the i-th occupant. Among them, the system's predefined set of emotion states includes 6 emotions, namely happiness, sadness, tension, relaxation, anxiety, and optimism.

[0125] It simultaneously collects three types of multi-source perception data: in-vehicle environment, vehicle driving, and individual occupant status. It fully covers three types of emotional influencing factors: external environment, driving disturbance, and occupant's own physiological behavior, eliminating the shortcomings of single-dimensional data recognition, which is one-sided and has low accuracy.

[0126] like Figure 4 As shown, in one possible embodiment, the method for comprehensively analyzing and outputting the emotion label and corresponding emotion intensity of the i-th passenger includes the following steps:

[0127] S1031, from the comprehensive time-series dataset of the i-th occupant (i.e. The feature vector set is extracted from the data. This feature vector set contains seven feature vectors: in-vehicle temperature, in-vehicle humidity, in-vehicle light intensity, vehicle speed, voice data, facial expression data, and heart rate variability. Each feature vector represents a component of the comprehensive time-series dataset. Different data sources within.

[0128] S1032. By processing all feature vectors in the feature vector set using LSTM, seven output vectors encoding the emotional state of the i-th passenger are obtained. Processing all feature vectors in the feature vector set using LSTM can capture temporal dependencies.

[0129] S1033. Fuse the 7 output vectors into an output matrix for predicting the probability of emotional state.

[0130] As an example, the seven output vectors are fused into a unified sentiment feature representation, namely the output matrix H:

[0131] ;

[0132] in, This represents the weight matrix of the fusion layer. This represents the bias term of the fusion layer. , These are the parameters in the emotion recognition model, obtained through training. These represent the 7 output vectors at time t.

[0133] S1034. Based on the output matrix, using a fully connected layer and a softmax activation function, predict the probabilities of various emotional states for the i-th passenger. As an example, ;

[0134] Among them, P Let represent the probabilities of the six emotional states of the i-th passenger. Indicates the weights of the fully connected layer. Indicates the bias of the fully connected layer. , These are the parameters in the emotion recognition model, obtained through training.

[0135] The emotion recognition model's output layer has six neurons. The probability value of each output node represents the likelihood that the model predicts the i-th passenger is currently in that emotional state. The output is a six-dimensional vector. Each element in the equation corresponds to a probability of an emotional state.

[0136] S1035. Take the emotional state with the highest probability as the emotional label of the i-th passenger, and take the highest probability (i.e. the probability corresponding to the emotional label of the i-th passenger) as the corresponding emotional intensity.

[0137] Ultimately, an emotion mapping matrix M_emo for the n occupants (i.e., multi-occupant) in the vehicle at time t can be formed. :

[0138] .

[0139] It can also be expressed as: .

[0140] Pos_1 represents the seat position coordinates of the first occupant (e.g., driver's seat, front passenger seat, rear left, rear right). 1 represents the emotion label of the first occupant, Int_1 represents the emotion intensity corresponding to the emotion label of the first occupant, and Pos_2 represents the seat position coordinates of the second occupant (such as driver's seat, front passenger seat, rear left, rear right). 2 represents the emotion label of the second occupant, and Int_2 represents the emotion intensity corresponding to the emotion label of the second occupant. Pos_i represents the seat position coordinates of the i-th occupant (e.g., driver's seat, front passenger seat, rear left, rear right). Let i be the emotion label of the i-th passenger, and Int_i be the emotion intensity corresponding to the emotion label of the i-th passenger.

[0141] The process of temporal feature extraction, LSTM encoding, feature fusion, and probability prediction is adapted to real-time in-vehicle computing. It combines the ability to fuse multi-source information with the advantages of temporal analysis, greatly improving the accuracy and reliability of independent emotion recognition for multiple occupants, and providing standardized quantitative input for vehicle zoning and compromise environmental control.

[0142] S2. Based on the emotion label of the i-th occupant, determine the expected set of environmental parameters for the i-th occupant, and then execute S3. The elements in the environmental parameter set (i.e., environmental parameters) include air conditioning temperature, in-vehicle light intensity, music type, music volume, and driving style (i.e., vehicle driving style).

[0143] In one possible embodiment, the desired set of environmental parameters for the i-th occupant is determined as follows:

[0144] Based on the emotion label of the i-th passenger, query the preset correspondence table between emotion labels and the expected environmental parameter set to obtain the expected environmental parameter set of the i-th passenger.

[0145] As an example, for the "Relax" label, the meaning of each element in the expected environmental parameter set could be: low air conditioning temperature, soft lighting, playing relaxing music, moderate volume, and smooth driving style.

[0146] S3. Determine whether there is a conflict of emotional needs among the n passengers. If yes, execute S6; otherwise, execute S4.

[0147] In one possible implementation, the method for determining whether there is a conflict of emotional needs among the n occupants is as follows:

[0148] The emotion labels of n passengers and their corresponding emotion intensities are combined to form a multi-passenger emotion mapping matrix M_emo. .

[0149] Traversing the multi-crew emotion mapping matrix M_emo If any two occupants have the same emotional label, and a conflict exists as defined in the pre-defined conflict rule base, then an emotional need conflict exists among the n occupants; otherwise, no emotional need conflict exists. For example, if the i-th occupant's emotional label is anxiety and the (i+1)-th occupant's emotional label is relaxation, then an emotional need conflict exists (i.e., a driving style conflict: anxious occupants prefer efficiency and speed, while relaxed occupants prefer smoothness and comfort). If the desired air conditioning temperature range of the i-th occupant does not overlap with that of the (i+1)-th occupant, then an emotional need conflict exists (i.e., an air conditioning temperature setting conflict).

[0150] S4. Based on the expected set of environmental parameters for each crew member, determine the globally optimal set of environmental parameters, and then execute S5.

[0151] like Figure 5 As shown, in one possible embodiment, the method for determining the globally optimal set of environmental parameters based on the desired set of environmental parameters for each occupant includes the following steps:

[0152] S41. Determine the globally optimal air conditioning temperature T set Global optimal in-vehicle illumination intensity L set Global optimal music volume V set .

[0153] In one possible implementation, the globally optimal air conditioning temperature is determined. Global optimal in-vehicle lighting intensity Global optimal music volume The methods include:

[0154] First, using the formula: Calculate the global adjustment basis weights for the i-th passenger. .in, This represents the intensity of the emotion corresponding to the emotion label of the i-th passenger. This represents the position weight factor of the i-th passenger. This represents the activity state factor of the i-th crew member.

[0155] In one possible embodiment, based on the location of the i-th occupant, a preset correspondence table between location and location weight factor is queried to obtain the location weight factor of the i-th occupant. Based on the activity behavior state of the i-th occupant, a pre-defined correspondence table between activity behavior states and activity state factors is consulted to obtain the activity state factor of the i-th occupant. .

[0156] As an example, the position weighting factor for the occupant sitting in the driver's seat is 1.2, and the position weighting factor for the occupants sitting in other positions is 1.0.

[0157] As an example, occupant activity states are identified using in-vehicle video cameras. These activity states can include reading, sleeping, and talking. For instance, the activity state factor for sleeping is 1.5 (high environmental sensitivity), while the activity state factor for talking is 0.8.

[0158] Secondly, based on the global adjustment base weights of the n crew members, the normalized weight coefficient of the i-th crew member is calculated. The specific calculation formula is as follows: .

[0159] Then, based on right , , By performing weighted summation for each of the n passengers, the globally optimal theoretical air conditioning temperature can be obtained. Global optimal theoretical in-vehicle illumination intensity Global optimal theoretical music volume .in, This represents the desired air conditioning temperature for the i-th passenger. This represents the desired interior light intensity for the i-th occupant. Let represent the desired music volume for the i-th passenger. As an example, using the formula: , , Calculate the globally optimal theoretical air conditioning temperature Global optimal theoretical in-vehicle illumination intensity Global optimal theoretical music volume .

[0160] Finally, for , , By performing amplitude limiting, the globally optimal air conditioning temperature is obtained. Global optimal in-vehicle lighting intensity Global optimal music volume .

[0161] As an example, using the formula: , , Calculate the globally optimal air conditioning temperature Global optimal in-vehicle lighting intensity Global optimal music volume .in, Represents the amplitude limiting function. This indicates the current actual temperature inside the vehicle. This indicates the current actual light intensity inside the vehicle. Indicates the current actual music volume. This indicates the preset upper limit for a single temperature adjustment. This indicates the preset upper limit for adjusting the light intensity per cycle. This indicates the preset maximum volume adjustment limit for a single music playback. Indicates will Limited to and Between, that is ,but ;like ,but ;like ,but .

[0162] The system calculates the global adjustment base weight of an individual passenger based on three factors: emotional intensity, passenger position weight, and passenger activity status. Passengers with high emotional intensity, in key positions, and active are automatically given higher weights. The calculation prioritizes the needs of passengers with higher levels of discomfort and those who are more critical to the driver and passenger, and the weight allocation is in line with the actual driving and riding priorities in the vehicle.

[0163] S42. Determine the globally optimal music type. .

[0164] In one possible implementation, the globally optimal music type is determined. The methods include:

[0165] based on , By performing a weighted summation of the data from the n crew members, we can obtain the j-th music genre. Weighted votes Where j takes any integer from 1 to r, and r represents the total number of music genres. This represents the desired music type for the i-th passenger. Indicates reaction and The indicator function of the relationship, when the desired music type of the i-th occupant is the j-th music type. (Right now )hour, When the desired music type of the i-th passenger is not the j-th music type. (Right now )hour, The specific calculation formula is as follows: .

[0166] Then, select the music genre with the highest weighted votes as the globally optimal music genre. .

[0167] Based on normalized weight coefficients A weighted voting calculation logic is constructed, which incorporates priority information such as the intensity of the passenger's emotions, the passenger's position, and the activity status into the voting score. Passengers with stronger emotions and higher priority are given higher decision-making weight, and the compromise result is more in line with the comfort needs of the core drivers and passengers in the vehicle.

[0168] In the event of a tie, the principle of minimum emotional conflict distance is introduced as arbitration: the music genre with the smallest sum of total emotional distances to the expected environmental parameters of all occupants is selected. Emotional distance The values ​​are predefined musical sentiment similarity matrix values. Indicates music genre With music genre The emotional distance (which can be obtained by querying the music emotional similarity matrix value, equivalent to looking up a table) is as follows: for example, the emotional distance between classical and light music is 0.3, and the emotional distance between classical and heavy metal is 0.9.

[0169] S43. Determine the globally optimal driving style .

[0170] In one possible implementation, the globally optimal driving style is determined. The methods include:

[0171] First, the desired driving style of the i-th occupant. Normalized weighting coefficients of the i-th crew member Combine them to form the i-th tuple.

[0172] Secondly, the n tuples are sorted in ascending order of normalized weight coefficients to form a style weight sequence.

[0173] Then, starting from the first tuple in the style weight sequence, the normalized weight coefficients are accumulated to obtain the accumulated weight corresponding to the k-th tuple. Specifically, using the formula: Calculate the cumulative weight corresponding to the k-th tuple. , This represents the normalized weight coefficient in the b-th tuple.

[0174] Finally, the cumulative weights will be... The desired driving style at the position where the value first exceeds 0.5 is taken as the globally optimal driving style. .

[0175] Each passenger's desired driving style and normalized weight coefficient are encapsulated into independent tuples, which centrally carry individual needs and priority data. The data structure is well-organized, which facilitates batch sorting and cumulative calculation. The unified weight standard mentioned above is reused, and the multi-dimensional control logic of the whole vehicle is unified.

[0176] As an example, driving styles can be Very Smooth, Smooth, Standard, Sport, and Efficient / Fast. Available driving styles are predefined.

[0177] S44, will , , , , Combined, a globally optimal set of environmental parameters is formed. .

[0178] S5. Generate corresponding function control instructions based on the global optimal set of environment parameters, and then execute S11.

[0179] For example, setting the car's air conditioning temperature to... Adjust the interior lighting intensity to Set the music type to Turn the music volume up to This makes the overall driving style of the vehicle... .

[0180] S6. Based on the expected environmental parameter set for each occupant, determine the globally optimal compromise environmental parameter set, and then execute S7.

[0181] like Figure 6 As shown, in one possible embodiment, the method for determining the globally optimal compromise environmental parameter set based on the desired environmental parameter set for each occupant includes:

[0182] S61. Determine the globally optimal compromise air conditioning temperature. Global optimal compromise in-vehicle light intensity Global optimal compromise music volume .

[0183] In one possible implementation, the globally optimal compromise air conditioning temperature is determined. Global optimal compromise in-vehicle light intensity Global optimal compromise music volume The methods include:

[0184] First, using the formula: Calculate the global compromise adjustment base weight for the i-th passenger. .in, This represents the intensity of the emotion corresponding to the emotion label of the i-th passenger. This represents the emotion label factor of the i-th passenger. This represents the position weight factor of the i-th passenger. This represents the passenger type factor of the i-th passenger.

[0185] In one possible embodiment, based on the emotion label of the i-th passenger, a preset correspondence table of emotion labels and emotion label factors is queried to obtain the emotion label factor of the i-th passenger. For example, the anxiety label factor is 1.5, and the relaxation label factor is 0.8. Based on the occupant type of the i-th occupant, the system queries a pre-defined table of correspondence between occupant types and occupant type factors to obtain the occupant type factor of the i-th occupant. .

[0186] As an example, occupant type (such as child or elderly) is identified using an in-vehicle video camera. For instance, the occupant type factor for a child is 1.3, and the occupant type factor for an elderly person is 1.5.

[0187] Secondly, based on the global adjustment base weights of the n crew members, the normalized compromise weight coefficient of the i-th crew member is calculated. The specific calculation formula is as follows: .

[0188] Then, based on right , , By performing weighted summation for each of the n passengers, the globally optimal compromise theoretical air conditioning temperature can be obtained. Global optimal compromise theory of in-vehicle light intensity Global optimal compromise theory for music volume .in, This represents the desired air conditioning temperature for the i-th passenger. This represents the desired interior light intensity for the i-th occupant. Let represent the desired music volume for the i-th passenger. As an example, using the formula: , , Calculate the globally optimal compromise theoretical air conditioning temperature Global optimal compromise theory of in-vehicle light intensity Global optimal compromise theory for music volume .

[0189] Finally, for , , By performing amplitude limiting, the globally optimal compromise air conditioning temperature is obtained. Global optimal compromise in-vehicle light intensity Global optimal compromise music volume .

[0190] As an example, using the formula: , , Calculate the globally optimal compromise air conditioning temperature Global optimal compromise in-vehicle light intensity Global optimal compromise music volume .in, Represents the amplitude limiting function. This indicates the current actual temperature inside the vehicle. This indicates the current actual light intensity inside the vehicle. Indicates the current actual music volume. This indicates the preset upper limit for a single temperature adjustment. This indicates the preset upper limit for adjusting the light intensity per cycle. This indicates the preset maximum volume adjustment limit for a single music playback.

[0191] The compromise scenario adds an emotion label factor and a passenger type factor, which, together with emotion intensity and position weight, jointly construct a global compromise adjustment base weight. Higher weights can be assigned to negative emotions and special passengers. In conflict scenarios, priority is given to comforting passengers with stronger discomfort. The compromise allocation logic is more in line with the needs of multiple conflict scenarios.

[0192] S62. Determine the globally optimal compromise music type .

[0193] In one possible implementation, the globally optimal compromise music type is determined. The methods include:

[0194] First based on , Perform a weighted summation of the data from the n crew members to determine the choice of the j-th music genre. Groups that are not moderate Where j takes any integer from 1 to r, and r represents the total number of music genres. This represents the desired music type for the i-th passenger. express and Emotional distance. The specific calculation formula is: .

[0195] Then select the music type with the smallest group discomfort as the globally optimal compromise music type. .

[0196] The Dist() function is used to quantify the degree of difference between candidate music types and passengers' ideal music types. This breaks through the limitations of binary matching judgment and can finely distinguish different levels of fit, such as similar music styles, large differences, and complete contradictions. It accurately reflects the degree of subjective emotional discomfort of passengers and improves the fineness of the assessment.

[0197] If a tie occurs, the principle of minimum emotional conflict distance is introduced as arbitration: the music type with the smallest sum of emotional distances to the expected environmental parameters of all passengers is selected.

[0198] S63. Determine the globally optimal compromise driving style .

[0199] In one possible implementation, the globally optimal compromise driving style is determined. The methods include:

[0200] First, the desired driving style of the i-th occupant. Normalized compromise weighting coefficients with the i-th passenger Combine them to form the i-th tuple.

[0201] Secondly, the n tuples are sorted in ascending order according to the normalized compromise weight coefficient to form a style compromise weight sequence.

[0202] Then, starting from the first tuple in the style compromise weight sequence, the normalized compromise weight coefficients are accumulated to obtain the compromise cumulative weight corresponding to the k-th tuple. Specifically, using the formula: Calculate the compromise cumulative weight corresponding to the k-th tuple. , This represents the normalized compromise weight coefficient in the b-th tuple.

[0203] Finally, the compromise cumulative weight will be used. The desired driving style at the position where the value first exceeds 0.5 is taken as the globally optimal compromise driving style. .

[0204] Each passenger's desired driving style and the normalized compromise weight coefficients for conflict scenarios are encapsulated into independent tuples. The data structure is regular and unified, and the weight system for conflict conditions is reused. It is consistent with the compromise calculation standards for air conditioning, lighting, and music, and the multi-dimensional conflict control logic of the whole vehicle is coherent and unified.

[0205] S64, will , , , , Combined, a globally optimal set of compromise environmental parameters is formed. .

[0206] S7. Determine whether the vehicle supports zone adjustment based on seat position and whether the i-th occupant meets the compensation trigger condition. If yes, execute S9; otherwise, execute S8.

[0207] In one possible embodiment, if the emotional intensity corresponding to the emotional tag of the i-th passenger is greater than a preset emotional intensity threshold, and meets any one of conditions one through four, then the i-th passenger is determined to meet the compensation trigger condition.

[0208] Condition 1 is: , , This indicates the preset temperature deviation threshold.

[0209] Condition two is: , , This indicates the preset light intensity deviation threshold.

[0210] Condition three is: , , This indicates the preset music volume deviation threshold.

[0211] Condition four is: .

[0212] The system distinguishes between two levels of control logic: a unified global compromise control and individual zone compensation. Zone compensation is only activated when an occupant's emotions are strong and the compromise solution cannot meet their basic comfort needs. This balances the simplicity of unified vehicle control with the humanization of personalized zone adjustment, and achieves a balance between the consistency of overall vehicle control and the effect of calming individual emotions.

[0213] As an example, zoned adjustment of seat positions means that air conditioning temperature, light intensity, music volume, and music style can be adjusted independently for passengers in different seat positions without interfering with each other.

[0214] S8. Generate corresponding function control instructions based on the globally optimal compromise environmental parameter set, and then execute S11.

[0215] For example, setting the car's air conditioning temperature to... Adjust the interior lighting intensity to Set the music type to Turn the music volume up to This makes the overall driving style of the vehicle... .

[0216] S9. Perform environmental parameter partition compensation for the i-th occupant, obtain the set of required environmental parameters for the i-th occupant after partition compensation, and then execute S10.

[0217] In one possible embodiment, the method for performing environmental parameter partition compensation for the i-th occupant and obtaining the set of required environmental parameters for the i-th occupant after partition compensation includes:

[0218] like Then the required air conditioning temperature for the i-th passenger will be determined. ;like Then the required air conditioning temperature for the i-th passenger will be determined. ;in, This indicates the preset temperature compensation coefficient. Here, the vehicle supports four-zone independent climate control.

[0219] like This will make the required light intensity for the i-th occupant... ;like This will make the required light intensity for the i-th occupant... The vehicle here supports zoned control of reading lights and ambient lighting.

[0220] like Then the music volume required by the i-th passenger will be determined. ;like Then the music volume required by the i-th passenger will be determined. ;in, This indicates the preset volume compensation coefficient. Here, the vehicle supports sound field zoning control.

[0221] like Then the music type required by the i-th passenger will be determined. ;like Then the music type required by the i-th passenger will be determined. Even if the i-th occupant requires a certain music genre... Here, the vehicle supports sound field zoning control.

[0222] Determine the driving style required by the i-th occupant. .

[0223] Will , , , , Combined, forming the set of environmental parameters required by the i-th occupant after partition compensation. .

[0224] The system integrates zone temperature, lighting, music type, volume, and driving style into a structured set of zone requirement parameters. The data format is unified, which makes it easy for the vehicle zone controller to directly parse and generate corresponding zone hardware control instructions. It has a high degree of modularity and convenient program scheduling.

[0225] S10. Generate corresponding function control instructions based on the set of environmental parameters required by each occupant, and then execute S11.

[0226] For example, adjust the air conditioning temperature at the i-th passenger's seat to... Adjust the light intensity at the i-th passenger's location to Set the music type at the i-th passenger position to Adjust the music volume at the i-th passenger's seat to This makes the overall driving style of the vehicle... .

[0227] S11. Perform intelligent vehicle control based on function control commands, and then end.

[0228] As an example, intelligent vehicle control based on function control commands can be achieved by the onboard processing unit adjusting the vehicle's air conditioning temperature, interior lighting intensity, music type and volume, and selecting a driving style (such as smoother or more dynamic) to suit the occupants' current emotional state. The main components of intelligent vehicle control include the air conditioning system, lighting system, audio system, and powertrain control system.

[0229] By integrating multi-dimensional sensory data on the in-vehicle environment, driving status, and the status of each passenger, the system independently identifies the corresponding emotion tags and intensity for each of the n passengers in the vehicle. This accurately captures the differentiated emotional states of different passengers, overcoming the limitation of uniform data collection across the entire vehicle, which cannot distinguish individual emotions. It provides a precise basis for individualized comfort control. Balancing common needs across the entire vehicle with personalized emotional demands, it offers compromise compensation in conflict scenarios and rapid unified control in non-conflict scenarios. Based on the dynamic adaptation of the in-vehicle environment and driving style to the emotions of all passengers, it effectively alleviates negative emotions and enhances overall vehicle comfort and intelligent interactive experience.

[0230] Finally, it should be noted that the application of this application is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims. Those skilled in the art can understand and implement all or part of the processes of the above embodiments, and equivalent changes made according to the claims of this application still fall within the scope of this application.

Claims

1. A vehicle intelligent control method based on multi-occupant emotion recognition, characterized in that, include: Based on in-vehicle environment perception data, vehicle driving status data, and occupant status data of the i-th occupant, the emotion label and corresponding emotion intensity of the i-th occupant are identified; where i takes all integers from 1 to n, and n represents the total number of occupants in the vehicle. Based on the emotion label of the i-th occupant, determine the set of expected environmental parameters for the i-th occupant; where the elements in the set of environmental parameters include air conditioning temperature, in-vehicle light intensity, music type, music volume, and driving style; If there is an emotional conflict among n occupants, the global optimal compromise environmental parameter set is determined based on the expected environmental parameter set of each occupant. Then, it is determined whether the vehicle supports zoned adjustment based on seat position and whether the i-th occupant meets the compensation trigger condition. If so, the environmental parameter zoned compensation for the i-th occupant is performed to obtain the required environmental parameter set of the i-th occupant after zoned compensation. If there is no conflict of emotional needs among the n passengers, then the globally optimal set of environmental parameters is determined based on the expected set of environmental parameters for each passenger. Based on the globally optimal compromise environmental parameter set, or the environmental parameter set required by each occupant, or the globally optimal environmental parameter set, generate corresponding functional control instructions; Intelligent vehicle control is performed based on the aforementioned function control commands.

2. The vehicle intelligent control method based on multi-occupant emotion recognition according to claim 1, characterized in that, Methods for identifying the emotion label and corresponding emotion intensity of the i-th passenger include: Acquire in-vehicle environment perception data, vehicle driving status data, and occupant status data of the i-th occupant; The in-vehicle environment perception data, vehicle driving status data, and occupant status data of the i-th occupant are combined to form a comprehensive time-series dataset of the i-th occupant. The comprehensive time-series dataset of the i-th passenger is input into a pre-trained emotion recognition model, and the comprehensive analysis outputs the emotion label and corresponding emotion intensity of the i-th passenger.

3. The vehicle intelligent control method based on multi-occupant emotion recognition according to claim 2, characterized in that: The in-vehicle environment perception data includes in-vehicle temperature time series, humidity time series, and light intensity time series; the vehicle driving status data is vehicle speed time series; and the occupant status data of the i-th occupant includes the i-th occupant's voice data time series, facial expression data time series, and heart rate variability data time series. The method for comprehensively analyzing and outputting the emotion label and corresponding emotion intensity of the i-th passenger includes: Extract the feature vector set from the comprehensive time-series dataset of the i-th occupant; By processing all feature vectors in the feature vector set using LSTM, seven output vectors encoding the emotional state of the i-th passenger are obtained. The seven output vectors are fused into an output matrix for predicting the probability of emotional states; Based on the output matrix, the probabilities of various emotional states of the i-th passenger are predicted using a fully connected layer and a softmax activation function. The emotional state with the highest probability is taken as the emotional label of the i-th passenger, and the highest probability is taken as the corresponding emotional intensity.

4. The vehicle intelligent control method based on multi-occupant emotion recognition according to claim 1, characterized in that: The method for determining the set of expected environmental parameters for the i-th occupant is as follows: Based on the emotion tag of the i-th passenger, query the preset correspondence table between emotion tags and the expected environmental parameter set to obtain the expected environmental parameter set of the i-th passenger. The method to determine whether there is a conflict of emotional needs among n passengers is as follows: The emotional labels of n passengers and their corresponding emotional intensities are combined to form a multi-passenger emotional mapping matrix; Traverse the emotion tag pairs of any two passengers in the multi-passenger emotion mapping matrix. If there is a conflict as defined in the preset conflict rule base, it is determined that there is an emotional need conflict among the n passengers; otherwise, it is determined that there is no emotional need conflict among the n passengers.

5. The vehicle intelligent control method based on multi-occupant emotion recognition according to claim 1, characterized in that, Methods for determining the globally optimal set of environmental parameters based on the expected set of environmental parameters for each occupant include: Determine the global optimal air conditioning temperature Global optimal in-vehicle lighting intensity Global optimal music volume ; Determine the globally optimal music type and globally optimal driving style ; Set the global optimal air conditioning temperature Global optimal in-vehicle lighting intensity Globally optimal music type Global optimal music volume Global optimal driving style Combined, the set of globally optimal environmental parameters is formed. ; Determine the global optimal air conditioning temperature Global optimal in-vehicle lighting intensity Global optimal music volume The methods include: Will , , Multiply to obtain the global adjustment base weights for the i-th crew member. ;in, This represents the intensity of the emotion corresponding to the emotion label of the i-th passenger. This represents the position weight factor of the i-th passenger. This represents the activity state factor of the i-th crew member; Based on the global adjustment base weights of n crew members, calculate the normalized weight coefficient of the i-th crew member. ; based on right , , By performing weighted summation for each of the n passengers, the globally optimal theoretical air conditioning temperature can be obtained. Global optimal theoretical in-vehicle illumination intensity Global optimal theoretical music volume ;in, This represents the desired air conditioning temperature for the i-th passenger. This represents the desired interior light intensity for the i-th occupant. This represents the desired music volume for the i-th passenger; right , , By performing amplitude limiting, the globally optimal air conditioning temperature is obtained. Global optimal in-vehicle lighting intensity Global optimal music volume ; Determine the globally optimal music type The methods include: based on , By performing a weighted summation of the data from the n crew members, we can obtain the j-th music genre. Weighted votes Where j takes any integer from 1 to r, and r represents the total number of music genres. This represents the desired music type for the i-th passenger. Indicates reaction and The indicator function of the relationship, when the desired music type of the i-th occupant is the j-th music type. hour, When the desired music type of the i-th passenger is not the j-th music type. hour, ; The music genre with the highest weighted votes is selected as the globally optimal music genre. ; Determine the globally optimal driving style The methods include: The desired driving style of the i-th occupant Normalized weighting coefficients of the i-th crew member Combine to form the i-th tuple; Sort the n tuples in ascending order of their normalized weight coefficients to form a style weight sequence; Starting from the first tuple in the style weight sequence, the normalized weight coefficients are accumulated to obtain the accumulated weight corresponding to the k-th tuple. , ; The cumulative weight The desired driving style at the position where the value first exceeds 0.5 is taken as the globally optimal driving style. .

6. The vehicle intelligent control method based on multi-occupant emotion recognition according to claim 1, characterized in that, Methods for determining the globally optimal compromise set of environmental parameters based on the expected set of environmental parameters for each occupant include: Determine the globally optimal compromise air conditioning temperature Global optimal compromise in-vehicle light intensity Global optimal compromise music volume ; Determine the globally optimal compromise music type and globally optimal compromise driving style ; The globally optimal compromise air conditioning temperature Global optimal compromise in-vehicle light intensity Globally optimal compromise music type Global optimal compromise music volume Global optimal compromise driving style Combined, the set of globally optimal compromise environmental parameters is formed. ; Determine the globally optimal compromise air conditioning temperature Global optimal compromise in-vehicle light intensity Global optimal compromise music volume The methods include: Will , , , Multiplying these together yields the global compromise adjustment base weight for the i-th passenger. ;in, This represents the intensity of the emotion corresponding to the emotion label of the i-th passenger. This represents the emotion label factor of the i-th passenger. This represents the position weight factor of the i-th passenger. The occupant type factor represents the i-th occupant; Based on the global adjustment base weights of n crew members, calculate the normalized compromise weight coefficient of the i-th crew member. ; based on right , , By performing weighted summation for each of the n passengers, the globally optimal compromise theoretical air conditioning temperature can be obtained. Global optimal compromise theory of in-vehicle light intensity Global optimal compromise theory for music volume ;in, This represents the desired air conditioning temperature for the i-th passenger. This represents the desired interior light intensity for the i-th occupant. This represents the desired music volume for the i-th passenger; right , , By performing amplitude limiting, the globally optimal compromise air conditioning temperature is obtained. Global optimal compromise in-vehicle light intensity Global optimal compromise music volume ; Determine the globally optimal compromise music type The methods include: based on , Perform a weighted summation of the data from the n crew members to determine the choice of the j-th music genre. Groups that are not moderate Where j takes any integer from 1 to r, and r represents the total number of music genres. This represents the desired music type for the i-th passenger. express and Emotional distance; The music type with the smallest group discomfort is selected as the globally optimal compromise music type. ; Determine the globally optimal compromise driving style The methods include: The desired driving style of the i-th occupant Normalized compromise weighting coefficients with the i-th passenger Combine to form the i-th tuple; Sort the n tuples in ascending order of normalized compromise weight coefficients to form a style compromise weight sequence; Starting from the first tuple in the style compromise weight sequence, the normalized compromise weight coefficients are accumulated to obtain the cumulative compromise weight corresponding to the k-th tuple. , ; The compromise cumulative weight The desired driving style at the position where the value first exceeds 0.5 is taken as the globally optimal compromise driving style. .

7. The vehicle intelligent control method based on multi-occupant emotion recognition according to claim 6, characterized in that: If the emotional intensity corresponding to the emotional label of the i-th passenger is greater than the preset emotional intensity threshold, and meets any one of conditions one through four, then the i-th passenger is determined to meet the compensation trigger condition; where, Condition 1 is: , , This indicates the preset temperature deviation threshold; Condition two is: , , This indicates the preset light intensity deviation threshold; Condition three is: , , This indicates the preset music volume deviation threshold; Condition four is: ; Methods for performing environmental parameter partition compensation for the i-th occupant and obtaining the set of required environmental parameters for the i-th occupant after partition compensation include: like Based on , , Determine the required air conditioning temperature for the i-th passenger. ;like Then the required air conditioning temperature for the i-th passenger will be determined. ;in, This indicates the preset temperature compensation coefficient; like This will make the required light intensity for the i-th occupant... ;like This will make the required light intensity for the i-th occupant... ; like Based on , , Determine the music volume required by the i-th passenger. ;like Then the music volume required by the i-th passenger will be determined. ;in, This indicates the preset volume compensation coefficient; The music type required by the i-th occupant The driving style required by the i-th occupant ; The desired air conditioning temperature for the i-th passenger. Required light intensity Music type required Required music volume Driving style requirements Combined, these parameters form the set of required environmental parameters for the i-th occupant after the partition compensation. .

8. The vehicle intelligent control method based on multi-occupant emotion recognition according to claim 6, characterized in that: Based on the location of the i-th passenger, query the preset correspondence table between location and location weight factor to obtain the location weight factor of the i-th passenger. ; Based on the activity behavior state of the i-th passenger, query the preset correspondence table between activity behavior states and activity state factors to obtain the activity state factor of the i-th passenger. ; Based on the emotion label of the i-th passenger, query the preset correspondence table between emotion labels and emotion label factors to obtain the emotion label factor of the i-th passenger. ; Based on the occupant type of the i-th occupant, query the preset correspondence table between occupant types and occupant type factors to obtain the occupant type factor of the i-th occupant. .

9. A vehicle intelligent control system based on multi-occupant emotion recognition, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: The processor is configured to execute the computer program to implement the vehicle intelligent control method based on multi-occupant emotion recognition as described in any one of claims 1 to 8.

10. A vehicle, characterized in that: This includes the vehicle intelligent control system based on multi-occupant emotion recognition as described in claim 9.

Images