A vehicle cockpit control method, system, and vehicle

Abstract

This application provides a vehicle cockpit control method, cockpit system, and vehicle, applied in the field of intelligent cockpits. The method includes: first, acquiring the user's emotional data and historical preference data for cockpit services and routes; the emotional data including emotion type and intensity; then, based on the emotional data and historical preference data, calculating the service score for each cockpit service and the route score for each alternative route; and finally, adjusting the cockpit services and determining the recommended route based on the service score for each cockpit service and the route score for each alternative route. This application not only enables the cockpit system to proactively output functional services and emotional comfort matching the user's emotional type and intensity, but also provides users with a more personalized and comfortable driving experience, enhancing the humanistic care of human-computer interaction and user satisfaction.

Description

Technical Field

[0001] This application relates to the field of intelligent cockpits, specifically to a vehicle cockpit control method, system, and vehicle. Background Technology

[0002] In-vehicle intelligent cockpit systems typically rely on voice recognition and touch feedback, or limited physiological state monitoring such as driver fatigue warnings, to trigger preset functional responses in terms of human-machine interaction. The provision of cockpit services is entirely function-oriented, lacking emotional adaptation and humanistic care, especially regarding the impact of different emotions on vehicle operation. To address these issues, existing intelligent cockpit systems are beginning to incorporate emotion recognition for in-vehicle users and adjust related equipment within the intelligent cockpit, such as lighting and music, to regulate the user's emotions.

[0003] However, when users experience emotional issues due to road conditions, simply adjusting the relevant equipment within the smart cockpit is not very effective in regulating their emotions. Therefore, how to provide emotional services that dynamically match the user's current emotions has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, this application provides a vehicle cockpit control method, system, and vehicle, which calculates the service score of each cockpit service and the route score of each alternative route based on the user's emotional data and the user's historical preference data for cockpit services and routes, adjusts the cockpit services, and determines the recommended route. This effectively solves the technical problem that when users have emotional problems caused by road conditions, simply adjusting the relevant equipment in the smart cockpit has a poor effect on regulating the user's emotions.

[0005] As a first aspect of this application, this application provides a vehicle cockpit control method, including: Acquire user emotion data and user historical preference data for cabin services and routes, wherein the emotion data includes emotion type and emotion intensity; Based on the sentiment data and the historical preference data, a service score for each cabin service and a route score for each alternative route are calculated, wherein the service score is used to characterize the degree of matching between the cabin service and the sentiment data and the historical preference data, and the route score is used to characterize the degree of matching between the alternative routes and the sentiment data and the historical preference data. Based on the service rating of each cabin service and the route rating of each alternative route, the cabin services are adjusted and a recommended route is determined.

[0006] Optionally, obtaining the user's emotional data includes: Acquire multimodal data collected by multimodal sensors, including user biometrics, user behavioral information, vehicle environmental data, and road data; Based on the multimodal data, the user's emotional data is determined.

[0007] Optionally, determining the user's emotional data based on the multimodal data includes: The multimodal data is fused and cross-validated based on an emotion recognition algorithm to generate initial emotion data, which includes initial emotion type and initial emotion intensity. When the initial emotional intensity is greater than or equal to a preset emotional intensity threshold, a preliminary emotional data verification is initiated to the user so that the user can provide the first feedback information. Based on the first feedback information, the user's preliminary emotional data is corrected to determine the user's emotional data.

[0008] Optionally, the step of calculating the service rating for each cabin service and the route rating for each alternative route based on the sentiment data and the historical preference data includes: Based on the user's emotional data, preset emotional data, and route adaptation rules, the emotional adaptation score of each alternative route is determined. Based on the user's emotional data, preset emotional data and cabin service adaptation rules, determine the emotional adaptation score for each cabin service; Based on users' historical preference data for cabin services and routes, determine the user's preference score for each cabin service and the user's preference score for each alternative route; Based on the intensity of the emotion, determine the emotion weight and the user preference weight; The service score for each cabin service is calculated based on the emotional fit score, emotional weight, user preference score for each cabin service, and user preference weight. The route score for each alternative route is calculated based on the emotional fit score, emotional weight, user preference score for each alternative route, and user preference weight.

[0009] Optionally, when there are at least two users, the multimodal data may also include user identity information and instruction information; The vehicle cockpit control method described in the first aspect above also includes: Based on the user's instruction information, the user's identity information, and the user's emotional data, determine the level of conflict of needs among users in the cockpit; Based on the user's conflict level and the preset conflict level handling rules, a first candidate set is generated and prompted to the user. The first candidate set includes instruction execution methods and cabin services.

[0010] Optionally, the user's conflict of needs includes driving style conflict, emotional conflict, and cabin service conflict. Determining the level of user conflict of needs within the cabin based on the user's instruction information, the user's identity information, and the user's emotional data includes: Based on the user's instruction information, the user's identity information, and the user's emotional data, determine whether the user has a conflict of driving style, emotional conflict, or cabin service conflict. When the user has a conflicting driving style, it is determined that the user in the cockpit has a Level 1 conflict of needs. When the user is experiencing emotional conflict, it is determined that the user in the cabin has a secondary need conflict. When a user has a cabin service conflict, it is determined that the user has a level three demand conflict within the cabin.

[0011] Optionally, generating a first candidate set based on the user's conflict level and preset conflict level processing rules includes: When there is a primary demand conflict among the users in the cockpit, a first candidate set is generated, which includes the instruction execution method corresponding to the driver. When there is a secondary demand conflict among the users in the cabin, the priority of the users is determined based on the emotional intensity and identity information of the users, and the users with the highest priority are selected to generate a first candidate set including the cabin services corresponding to the users with the highest priority. When there is a conflict of three levels of needs among the users in the cabin, a first candidate set is generated, which includes cabin services that correspond to the compromise of the users' needs.

[0012] Optionally, the vehicle cockpit control method described in the first aspect further includes: Obtain user feedback on the cabin service and the recommended route; Determine the target route based on user feedback on the recommended routes; The user's emotional data, feedback information, cabin services, and target route are uploaded to the cloud server.

[0013] As a second aspect of this application, this application provides a vehicle cockpit control system, including: Multimodal sensors and vehicle-side controllers, among which, The vehicle-mounted controller is communicatively connected to the multimodal sensors and the cloud server, respectively. The multimodal sensor is used to collect multimodal data, which includes the user's biological information, the user's behavioral information, the vehicle's environmental data, and road data. The vehicle-mounted controller is used to execute the vehicle cockpit control method described in any of the first aspects above.

[0014] As a third aspect of this application, this application provides a vehicle comprising: The vehicle cockpit control system as described in the second aspect above.

[0015] Based on the above, the vehicle cockpit control method provided in this application acquires the user's emotional data and historical preference data for cockpit services and routes. The emotional data includes emotion type and intensity. Based on the user's emotional data and historical preference data, a service score for each cockpit service and a route score for each alternative route are calculated. The service score characterizes the matching degree between the cockpit service and the emotional data and historical preference data, and the route score characterizes the matching degree between the alternative routes and the emotional data and historical preference data. Based on the scores for each cockpit service and each alternative route, the cockpit service is adjusted and a recommended route is determined. This application, by jointly making decisions with emotional data containing emotion type and intensity and the user's long-term accumulated historical preference data for cockpit services and routes, adjusts the cockpit service and determines the recommended route based on the predicted scores for each cockpit service and each alternative route. This not only enables the cockpit system to proactively output functional services and emotional comfort that match the user's emotional type and intensity, but also provides users with a more personalized and comfortable driving experience, enhancing the humanistic care of human-computer interaction and user satisfaction. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 The diagram shown is a flowchart of a vehicle cockpit control method provided in an embodiment of this application.

[0018] Figure 2 The diagram shown is a flowchart of another vehicle cockpit control method provided in an embodiment of this application.

[0019] Figure 3 The diagram shown is a flowchart of another vehicle cockpit control method provided in an embodiment of this application.

[0020] Figure 4 The diagram shown is a flowchart of another vehicle cockpit control method provided in an embodiment of this application.

[0021] Figure 5 The diagram shown is a flowchart of another vehicle cockpit control method provided in an embodiment of this application.

[0022] Figure 6 The diagram shown is a flowchart of another vehicle cockpit control method provided in an embodiment of this application.

[0023] Figure 7 The diagram shown is a structural block diagram of a vehicle cockpit control system provided in an embodiment of this application. Detailed Implementation

[0024] Unless otherwise defined, the technical or scientific terms used in the embodiments of this specification shall have the ordinary meaning understood by one of ordinary skill in the art to which this specification pertains. The terms "first," "second," and similar terms used in the embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to avoid confusion of constituent elements.

[0025] Unless the context otherwise requires, throughout this specification, "a plurality of" means "at least two," and "including" is interpreted as open-ended or encompassing, that is, "including, but not limited to." In the description of this specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this specification. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example.

[0026] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0027] Overview Currently, in-vehicle intelligent cockpit systems primarily rely on voice command recognition, touch feedback, and limited physiological state monitoring (such as driver fatigue warnings) to trigger preset functional responses in terms of human-machine interaction. This service model is entirely function-oriented and lacks emotional adaptation and humanistic care for the user's emotional state. Although existing intelligent cockpit system solutions have incorporated emotion recognition technology, capable of adjusting user emotions by modifying ambient lighting, audio, air conditioning, and other equipment within the cockpit, these solutions still have significant limitations.

[0028] Current methods for regulating emotions are limited to adjusting the cabin environment. When users experience negative emotions such as anxiety, irritability, and fatigue due to external factors like traffic congestion, traffic jams, or poor route planning, simply changing the cabin lighting or playing soothing music often fails to address the root cause. The user's fundamental need may be to find a route that allows them to relax and reduce stress, rather than simply altering the in-car atmosphere. Therefore, cabin-based adjustments cannot resolve emotional issues caused by external road conditions, resulting in limited emotional soothing effects.

[0029] Existing intelligent cockpit systems lack the technological mechanism to deeply integrate user emotional states with vehicle-road information and service decisions. Even if an intelligent cockpit system senses user anxiety, its service output is limited to adjusting the cockpit environment—a single service. It cannot incorporate emotional data as a decision-making factor into more personalized and comfortable decision-making strategies such as route planning, navigation strategy adjustment, and service requests. This results in cockpit services being disconnected from users' real emotional needs, failing to achieve an output that moves from functional response to emotional resonance.

[0030] To address the aforementioned issues and overcome the limitations of existing cockpit interior adjustments, an intelligent decision-making mechanism is needed to integrate user emotional states with vehicle-road information and service decisions. This mechanism would enable the cockpit system to dynamically perceive user emotions and proactively match route planning and service content accordingly, thereby providing truly emotionally resonant services that align with the user's current mood and actual preferences.

[0031] Based on the above concept, this application provides a vehicle cockpit control method. The vehicle cockpit control method provided by this application will be described exemplarily below with reference to the accompanying drawings.

[0032] Exemplary methods like Figure 1 As shown in the exemplary embodiment of this application, a vehicle cockpit control method is provided, applied to a vehicle controller or an independent electronic control unit in a vehicle. The method may include the following steps: S10: Obtain user sentiment data and historical user preference data for cabin services and routes.

[0033] User emotional data can be collected through preset multimodal sensors and processed by preset algorithms of the vehicle-side controller. Emotional data can include emotion type and emotional intensity. Emotion type refers to the label obtained after classifying the user's emotional state; for example, emotion types may include, but are not limited to, anxiety, fatigue, resentment, joy, irritability, depression, anticipation, and surprise. Emotional intensity is a numerical indicator used to quantify the intensity of the user's current emotion, and can be represented using a numerical range from 0 to 100.

[0034] Cockpit services refer to the functions provided by all hardware devices within the cockpit that can be adjusted by the vehicle-mounted controller to alter the user's perceived experience. Examples include ambient lighting, seat massage, audio systems, and air conditioning. Here, "route" refers to alternative routes from the current location to the destination. Historical user preference data for cockpit services and routes refers to preference information accumulated based on long-term user behavior habits. Based on this historical preference data, the number of times a user has selected or rejected different types of cockpit services and alternative routes over a past period can be obtained. This historical user preference data for cockpit services and routes can be stored in a dedicated database and retrieved directly when needed.

[0035] S20: Based on sentiment data and historical preference data, calculate the service score for each cabin service and the route score for each alternative route.

[0036] After acquiring user sentiment data and historical preference data for cabin services and routes, these two different dimensions of information—sentiment state and historical habits—can be unified into directly comparable rating values. This provides a clear and quantifiable basis for subsequent decision-making. Specifically, using user sentiment data and historical preference data for cabin services and routes, a service rating value for each cabin service and a route rating value for each alternative route are calculated. The service rating value represents the degree of matching between the cabin service and the sentiment and historical preference data, while the route rating value represents the degree of matching between the alternative routes and the sentiment and historical preference data. Both service and route rating values ​​can be represented using a numerical range from 0 to 100.

[0037] S30: Adjust cabin services and determine recommended routes based on the service score of each cabin service and the route score of each alternative route.

[0038] After calculating the service score for each cabin service and the route score for each alternative route, various methods can be used, such as ranking by score or setting a score threshold, to determine the cabin services to be provided to users and the recommended routes.

[0039] In some implementations, the service score of each cabin service and the route score of each alternative route are sorted, and at least one cabin service with the highest ranking is selected as the target cabin service for adjustment, and at least one alternative route with the highest ranking is selected as the recommended route.

[0040] In some implementations, a service rating threshold and a route rating threshold are preset. The calculated service rating value of each cabin service is compared with the service rating threshold. When a cabin service has a service rating value greater than or equal to the service rating threshold, that cabin service is selected as the target cabin service for adjustment. Similarly, the calculated route rating value of each alternative route is compared with the route rating threshold. When an alternative route has a route rating value greater than or equal to the route rating threshold, that alternative route is selected as the recommended route.

[0041] It is understandable that there may be more than one target cabin adjustment service and recommended route, and there may be multiple ways to obtain target cabin adjustment service and recommended route, which can be limited according to the actual situation.

[0042] In some embodiments of this application, by acquiring user emotion data and historical preference data for cabin services and routes (including emotion type and intensity), a service score for each cabin service and a route score for each alternative route are calculated based on the user's emotion data and historical preference data. The service score characterizes the degree of matching between the cabin service and the emotion data and historical preference data, and the route score characterizes the degree of matching between the alternative route and the emotion data and historical preference data. Based on the scores for each cabin service and each alternative route, the cabin service is adjusted and a recommended route is determined. This application, by jointly making decisions with emotion data containing emotion type and intensity and the user's long-term accumulated historical preference data for cabin services and routes, and adjusting the cabin service and determining the recommended route based on the predicted scores for each cabin service and each alternative route, not only enables the cabin system to proactively output functional services and emotional comfort that match the user's emotion type and intensity, but also provides users with a more personalized and comfortable driving experience, enhancing the humanistic care of human-computer interaction and user satisfaction.

[0043] In some embodiments of this application, obtaining the user's emotion data in S10 may include the following steps: S11: Acquire multimodal data collected by the multimodal sensor.

[0044] Multimodal data can include user biometrics, user behavioral information, vehicle environmental data, and road data. User biometrics refers to signals reflecting changes in a user's physiological state, such as, but not limited to, heart rate, heart rate variability, respiratory rate, and muscle tension. This biometrics can be obtained through flexible seat sensors and fingertip ECG sensors. Flexible seat sensors can be embedded in the seat back and cushion, while fingertip ECG sensors can be integrated into the 3 or 9 o'clock position buttons on the steering wheel. User behavioral information refers to data reflecting user behavioral characteristics, such as, but not limited to, facial micro-expression changes and eye movement characteristics. This behavioral information can be obtained through micro-expression cameras and eye-tracking modules. Micro-expression cameras can be installed above the central control screen, while eye-tracking modules can be integrated into the instrument panel.

[0045] Vehicle environmental data refers to the physical state parameters of the cabin and the surrounding environment, including but not limited to ambient noise intensity, light intensity, and traffic conditions. This data can be obtained through environmental sensors, which can be installed at the front of the vehicle and inside the vehicle. Road data refers to road-level information related to the driving route. This data can be received from roadside units via the vehicle-to-everything (V2X) wireless interface, or it can be obtained through a 4G / 5G network communication link with the urban traffic cloud platform, covering a range of up to 10 kilometers ahead. This data may include, but is not limited to, weather conditions, road construction status, and traffic congestion.

[0046] S12: Determine the user's emotional data based on multimodal data.

[0047] The vehicle-side controller needs to preprocess the aforementioned multimodal data, which may include noise reduction and feature extraction. Then, the user's emotional data, namely the emotion type and intensity, is determined through an emotion recognition algorithm.

[0048] In some embodiments of this application, step S12 involves determining the user's emotion data based on multimodal data. Specific steps may include: S121: Multimodal fusion and cross-validation of multimodal data based on emotion recognition algorithms to generate initial emotion data.

[0049] The emotion recognition algorithm is deployed on the vehicle-side controller and can be trained using a model on a cloud server. Initial emotion data can include initial emotion type and initial emotion intensity. Specifically, a Kalman filter algorithm can be used to denoise the multimodal data, filtering out interference from road bumps on the seat sensors and light changes on the camera, retaining valid signals. Based on the emotion recognition algorithm, multimodal data is fused and cross-validated to generate initial emotion data. Specifically, a convolutional neural network can be used to extract spatial features of micro-expressions and eye movements, and a long short-term memory network can be used to extract temporal features of heart rate and speech rate, which are then fused into a 128-dimensional feature vector. The recognition process can use a Bayesian network to cross-validate the preprocessed multimodal data to output initial emotion data containing the initial emotion type and initial emotion intensity.

[0050] S122: When the initial emotional intensity is greater than or equal to the preset emotional intensity threshold, initiate preliminary emotional data verification to the user so that the user can provide the first feedback information.

[0051] A preset emotional intensity threshold can be set, for example, a threshold of 60 points. After generating initial emotional data, the initial emotional intensity is compared with the preset threshold. When the vehicle controller determines that the initial emotional intensity is greater than or equal to the preset threshold, an active verification process is triggered. For example, the vehicle controller sends a verification command to the voice assistant module, which then issues a confirmation question to the user through the in-vehicle speaker, such as asking the user, "Are you feeling anxious?" The user can provide initial feedback information through voice response or touchscreen options. This initial feedback information is used to confirm or deny the user's preliminary emotional data and can include the initial emotional type and intensity, for example, an initial feedback information of anxiety of 75 points.

[0052] S123: Based on the first feedback information, revise the user's initial emotional data to determine the user's emotional data.

[0053] If the user provides positive initial feedback, the initial sentiment data is maintained and its confidence level is further increased. If the user provides negative initial feedback, the initial sentiment data is revised based on the initial feedback to determine the user's true sentiment. After determining the user's sentiment, the vehicle controller uploads the user's initial feedback information to the cloud server.

[0054] By setting an emotional intensity threshold to trigger user feedback, the initial emotional data is further corrected, and the final output is the user's emotional data. This is only triggered when the emotional intensity is high, avoiding the user's resentment caused by frequent questioning. It can also solve the defects of purely passive recognition, which is easily interfered with and prone to misjudgment. Finally, it obtains the real emotional label with high confidence, which is personally confirmed by the user.

[0055] In some embodiments of this application, such as Figure 2 As shown, in S20, based on sentiment data and historical preference data, a service rating for each cabin service and a route rating for each alternative route are calculated. Specific steps may include: S21: Based on the user's emotional data, preset emotional data, and route adaptation rules, determine the emotional adaptation score for each alternative route.

[0056] The preset emotion data and route matching rules refer to a pre-built emotion mapping table that associates different emotion types and intensity with the characteristics of different alternative routes. For example, for routes characterized by smooth road conditions, few stops and starts, and low interference, which effectively alleviate negative emotions, the emotion matching score can be set higher; while for routes characterized by frequent acceleration, deceleration, and lane changes, which exacerbate fatigue and anxiety, the emotion matching score can be set lower. The vehicle-side controller matches the characteristics of each alternative route with the preset matching rules based on the current user's emotion type and intensity, thereby deriving the emotion matching score for each alternative route.

[0057] S22: Based on the user's emotional data, preset emotional data, and the cabin service adaptation rules, determine the emotional adaptation score for each cabin service.

[0058] Similarly, the preset emotion data and cabin service matching rules associate different emotion types with the type and parameters of cabin services. For example, for extreme anxiety, soft, cool-toned ambient lighting, because the light is soft and not glaring, helps calm emotions and relieve visual fatigue, so its emotion matching score can be set higher; while for high-brightness dynamic ambient lighting, because the flickering light stimulates the senses and exacerbates irritability and fatigue, its emotion matching score can be set lower. The vehicle-side controller matches each available cabin service with the preset emotion data and cabin service matching rules based on the current user's emotion type and intensity, thus deriving an emotion matching score for each cabin service.

[0059] S23: Based on the user's historical preference data for cabin services and routes, determine the user's preference score for each cabin service and the user's preference score for each alternative route.

[0060] Specifically, rules can be pre-defined to correlate the number of times a user selects or rejects a particular cabin service and route with their preference score. For example, if a user selects a particular cabin service or route three or more times, their preference for that service or route can be categorized as high, with a score between 70 and 100. If a user has only one or two selection records, their preference can be considered medium, with a score between 40 and 60. If a user rejects a particular cabin service or route multiple times, their preference can be considered low, with a score between 0 and 30. For cabin services or routes with no history, such as new cars, the preference score can be set to a neutral value of 50 by default. Of course, rules can also be defined separately for each cabin service and preference score, and for each alternative route and preference score. This application does not impose further limitations on this; settings can be made according to the actual situation.

[0061] Based on users' historical preference data for cabin services and routes, the number of times users have selected or rejected different types of cabin services and different alternative routes over a period of time can be obtained. According to the correspondence rules between the number of times users have selected or rejected different types of cabin services and different alternative routes and their preference scores, the user's preference score for each cabin service and the user's preference score for each alternative route can be determined.

[0062] S24: Determine the emotion weight and user preference weight based on the intensity of emotion.

[0063] Emotional weight refers to the proportion of a user's emotional data in the final score calculation, while user preference weight is the proportion of a user's historical preference data in the final score calculation. In this application, the values ​​of emotional weight and user preference weight are dynamically correlated with emotional intensity. For example, when the emotional intensity is greater than or equal to 80 points, the emotional weight is 0.6 and the user preference weight is 0.4; when the emotional intensity is less than 80 points but greater than or equal to 60 points, the emotional weight is 0.5 and the user preference weight is 0.5; and when the emotional intensity is less than 60 points, the emotional weight is 0.4 and the user preference weight is 0.6. Of course, other methods can be used to determine the emotional weight and user preference weight, and this application does not impose specific limitations on them.

[0064] S25: Calculate the service score for each cabin service based on the emotional fit score, emotional weight, user preference score for each cabin service, and user preference weight.

[0065] Specifically, the service rating W for each cabin service i The calculation formula can be: W i =αXQ i +βXT i Among them, Q i The emotional fit score for each cabin service, α is the emotional weight, and T i Let α be the user's preference score for each cabin service, β be the user's preference weight, and α + β = 1.

[0066] S26: Calculate the route score for each alternative route based on the emotional fit score, emotional weight, user preference score for each alternative route, and user preference weight.

[0067] Specifically, each alternative route has a route score W. j The calculation formula can be: W j =αXQ j +βXT j Among them, Q j For each alternative route, an emotion fit score is assigned, where α is the emotion weight and T is the emotional weight. j Let α be the user's preference score for each alternative route, and β be the user's preference weight, where α + β = 1.

[0068] When there is only one user in the cabin, that is, only the driver in the driver's seat, there will be no conflict in the car regarding driving style, mood and cabin service. The service score of each cabin service and the route score of each alternative route are calculated based solely on the driver's mood data and historical preference data for cabin service and route. The cabin service is then adjusted and the recommended route is determined accordingly.

[0069] When there are at least two users in the cabin, different users may have different needs. To further resolve conflicts in needs between different users and avoid affecting the driver's mood and safe driving, this application also provides an embodiment that can resolve multi-user interaction conflicts and adapt to multi-passenger scenarios. In some embodiments of this application, such as Figure 3 As shown, the above-mentioned vehicle cockpit control method may further include the following steps: S40: Determine the level of conflict of needs among users in the cockpit based on user command information, user identity information, and user emotional data.

[0070] Demand conflict refers to a situation where commands issued simultaneously by two or more users are mutually exclusive, or their respective demands cannot be physically met simultaneously. Multimodal data also includes user identity information and command information. User identity information can include elderly people, children, young people, front passenger, driver, and rear passenger. The vehicle controller continuously monitors voice commands collected by microphones in multiple areas of the vehicle and touch commands received by the central control display unit to obtain user command information. It then uses facial recognition and height detection modules to identify each user.

[0071] The level of conflict can be classified into three levels: Level 1, Level 2, and Level 3, based on the nature and severity of the conflict. The level of conflict of user needs in the cockpit can be determined based on user instruction information, user identity information, and user emotional data.

[0072] S50: Generate a first candidate set based on the user's conflict level and the preset conflict level handling rules, and prompt the user.

[0073] The first candidate set includes command execution methods and cockpit services. The vehicle controller can invoke corresponding preset processing rules based on different conflict levels to generate differentiated recommended solutions, i.e., the first candidate set, and then prompt the user with these recommended solutions. Optionally, the prompts may include voice prompts and prompts from the central control display unit.

[0074] By determining the level of user conflict based on user command information, user identity information, and user emotional data, pre-setting rules for handling conflict levels, generating a first candidate set, and prompting the user, the cockpit system can resolve conflicts in a differentiated manner when facing complex scenarios of multi-user interaction, instead of simply executing the pilot's commands, thus avoiding a "one-size-fits-all" decision.

[0075] In some implementations, while the vehicle-side controller generates a first candidate set and prompts the user, it also uses a voice assistant module to explain the reasons and soothe the emotions of users whose needs have not been met.

[0076] For example, if the driver's anxiety level is 70, they can request to turn off the music. If the passenger's mood level is 60, they can request to play upbeat music. The vehicle controller will generate a first set of candidates, including turning off the music, and will explain to the user via voice: "The driver currently needs a quiet environment to relieve anxiety. We will play music for you shortly."

[0077] In some embodiments of this application, such as Figure 4 As shown, in S40, the user's command information, user identity information, and user emotional data are used to determine the level of conflict of needs among users in the cockpit. Specific steps may include: S41: Based on the user's instruction information, the user's identity information, and the user's emotional data, determine whether the user has a conflict of driving style, emotional conflict, or cabin service conflict.

[0078] User conflict can be categorized into driving style conflicts, emotional conflicts, and cabin service conflicts. The level of user conflict within the cabin can be matched with these three types of conflicts. Level 1 conflict can be a strong conflict, where commands are completely mutually exclusive and directly impact driving safety, such as the driver issuing a command to activate autopilot while the passenger issues a command to deactivate it. Level 2 conflict can be a moderate conflict, where commands are mutually exclusive but do not affect driving safety, such as a child user requesting an animated theme song while the driver requests news. Level 3 conflict can be a weak conflict, referring to partially conflicting needs that can be reconciled through compromise, such as a young passenger requesting the air conditioning temperature be set to 26°C while an elderly passenger in the back seat requests 24°C.

[0079] S42: When there is a conflict in driving style among users, it is determined that there is a Level 1 demand conflict among users in the cockpit.

[0080] The vehicle-side controller uses sensors such as multi-area microphones, touchscreens, and gesture recognition to detect when two or more users simultaneously issue mutually exclusive commands directly related to vehicle driving control. It then performs semantic analysis and functional classification on the conflicting commands to determine if they constitute a driving mode conflict. If it determines that the users issuing the mutually exclusive commands have a driving mode conflict, it identifies a Level 1 demand conflict among the users in the cabin.

[0081] S43: When a user experiences an emotional conflict, it is determined that there is a secondary need conflict among users in the cabin.

[0082] The vehicle-side controller uses sensors such as multi-area microphones and touch screens to detect when two or more users simultaneously issue mutually exclusive commands. It then acquires the emotional data of the users who issued the mutually exclusive commands. If it is determined that the users who issued the mutually exclusive commands are experiencing emotional conflict, it confirms that there is a secondary demand conflict among the users in the cabin.

[0083] S44: When a user has a cabin service conflict, determine that there is a Level 3 demand conflict among users in the cabin.

[0084] When the vehicle-side controller detects that two or more users simultaneously issue commands for the same cabin service but with different parameters through sensors such as multi-area microphones and touch screens, it determines that there is a cabin service conflict between the users issuing the commands and identifies a level 3 demand conflict among users in the cabin.

[0085] By matching three different types of conflicts—driving style conflict, emotional conflict, and cabin service conflict—with three levels of demand conflict—Level 1, Level 2, and Level 3—a qualitative leap has been achieved from general conflict perception to refined conflict classification and hierarchical handling. This allows the cabin system to move beyond simply following whoever's lead when dealing with multi-user interactions. Instead, it can automatically recommend the most reasonable handling path to users based on the root cause of the conflict (safety, emotions, and reconciliation), thereby improving the satisfaction of multiple users in the vehicle while ensuring safety.

[0086] In some embodiments of this application, such as Figure 5 As shown, in S50, a first candidate set is generated based on the user's conflict level and the preset conflict level handling rules. Specific steps may include: S51: When there is a Level 1 conflict of needs among users in the cockpit, generate a first candidate set including the instruction execution method corresponding to the driver.

[0087] The rules for handling pre-defined conflict levels can be preset. Optionally, these rules may include resolving conflict levels in the order of Level 1, Level 2, and Level 3. Alternatively, they may include resolving conflict levels in the order of prioritizing safe driving methods, emotional intensity, identity information, and cabin service negotiation and adaptation.

[0088] S52: When there is a conflict of secondary needs among users in the cabin, the priority of users is determined based on the intensity of their emotions and their identity information. The user with the highest priority is selected and a first candidate set is generated, which includes the cabin service corresponding to the user with the highest priority.

[0089] When a secondary conflict of needs is identified among users in the cabin, the conflict is resolved in order of emotional intensity and user identity. Based on the acquired emotional data, the system first prioritizes users according to emotional intensity. To demonstrate human consideration, the emotional intensity of users issuing mutual exclusion commands with negative emotions can be ranked, and the user with the highest emotional intensity is designated as the highest priority user. When users issuing mutual exclusion commands have the same emotional intensity, the vehicle controller uses facial recognition and height detection modules to identify each user's identity information. Then, it filters out the user with the highest identity priority according to the ranking rules of elderly, children, and youth. The final generated first candidate set includes the cabin services corresponding to the highest priority user.

[0090] S53: When there is a conflict of three levels of needs among users in the cabin, generate a first candidate set that includes cabin services that are a compromise with the user's needs.

[0091] When there is a Level 3 demand conflict among users in the cabin, the vehicle controller obtains the specific demand value or type of the user who issued the conflict, as well as the physical adjustable range corresponding to each demand, to generate a first candidate set, which includes cabin services that are compromised with the user's demand.

[0092] By subdividing conflicts into driving style conflicts, emotional conflicts, and cabin service conflicts, and corresponding to first-level, second-level, and third-level demand conflicts respectively, and establishing three handling rules for each, namely prioritizing driver instructions, prioritizing users with the highest priority based on the intensity of their emotions and their identity information, and compromising by providing the same cabin service, a clear and structured arbitration logic is provided for conflicts of different natures. This ensures driving safety while also taking into account the priority of emotional services and the compatibility of the physical environment.

[0093] In some embodiments of this application, such as Figure 6 As shown, the above-mentioned vehicle cockpit control method may further include the following steps: S60: Obtain user feedback on cabin services and recommended routes.

[0094] After the vehicle-side controller adjusts the cabin services and determines the recommended route, the voice assistant module can initiate a satisfaction survey with the user within a preset time, such as within one minute of the service being triggered, to collect the user's evaluation of the service combination and recommended route. Feedback can be user-expressed feedback via voice, such as satisfaction or dissatisfaction, or it can be rating options selected by the user through touch operation on the central control display unit.

[0095] S70: Determine the target route based on user feedback on the recommended routes.

[0096] If the user accepts the recommended route through feedback, the vehicle controller will designate the recommended route as the target route and initiate navigation guidance; if the user rejects the recommended route through feedback and manually selects another alternative route, the vehicle controller will designate the user's selected alternative route as the target route and record this selection to update the user's historical route preference data.

[0097] S80: Uploads user emotion data, feedback information, cabin services, and target routes to the cloud server.

[0098] The vehicle-mounted controller can package the emotion data used for this decision, the feedback information provided by the user, the final combination of cabin services to be executed, and the final selected target route, and upload them to the cloud server through the communication module.

[0099] The uploaded data can be used as samples for iterative training of large cloud models to optimize the parameters of the sensory recognition algorithm and various adaptation rules, so that the updated algorithms and rules can provide more accurate services and route recommendations.

[0100] By calculating the service score for each cabin service and the route score for each alternative route based on emotional data and historical preference data, the cabin service is adjusted and the recommended route is determined. User feedback on the cabin service and recommended route is obtained to determine the final combination of cabin services to be executed and the final selected target route. Finally, the user's emotional data, feedback information, cabin service, and target route are uploaded to the cloud server, so that the proposed solution forms a self-learning closed loop. The cabin system can learn and improve through each service and each user feedback, enabling it to achieve output from functional response to emotional resonance, and allowing the cabin system to provide more emotional services that are more in line with the user's current emotions and actual preferences.

[0101] To strictly ensure the security of sensitive privacy data such as user biometrics during model iteration using feedback data and cloud computing, and to resolve the contradiction between the privacy-related inability to directly upload raw multimodal data and the need to optimize model algorithm performance, in a preferred embodiment, this method may further include the following privacy protection and model iteration measures. In some embodiments, the above-mentioned vehicle cockpit control method may further include the following steps: S90: Encrypts and stores multimodal data collected by multimodal sensors in the vehicle-side trusted execution environment.

[0102] Raw, sensitive multimodal data such as electrocardiogram data, micro-expression data, and muscle tension data are directly written into the built-in trusted execution environment security zone within the vehicle-side controller. The data is encrypted using the AES-256 encryption algorithm, and the encryption key is stored only locally on the vehicle side and is never uploaded to a cloud server or any third-party server.

[0103] S100: Upload the de-identified sentiment type tags obtained through cross-validation to the cloud server.

[0104] The de-identified sentiment type labels obtained through cross-validation are uploaded to the cloud server. The vehicle-side controller extracts sentiment type labels (e.g., anxiety) and behavioral trend statistics (e.g., fatigue when returning home on Friday nights) from the sentiment data. After removing the vehicle identification code and user identity information, the data is transmitted to the cloud server via HTTPS encryption for training of a large cloud model.

[0105] S110: Receives update data packets transmitted from the cloud server according to a preset cycle to update the vehicle's emotion recognition algorithm parameters, preset emotion data and route adaptation rules, preset emotion data and cabin service adaptation rules, and user's historical preference data for cabin services and routes.

[0106] The vehicle receives update data packets from the cloud server at preset intervals to update the vehicle's emotion recognition algorithm parameters, preset emotion data and route adaptation rules, preset emotion data and cabin service adaptation rules, and user historical preference data for cabin services and routes. The vehicle controller receives update data packets distributed remotely via online upgrades through the communication module, refreshing the vehicle's algorithm parameters and rules. The vehicle can compress its capacity based on the MobileNet architecture, support offline inference, reduce the latency of single emotion recognition, and meet automotive-grade real-time requirements. Through periodic updates, the accuracy of emotion recognition is improved.

[0107] Exemplary device In some embodiments of this application, a vehicle cockpit control device is provided. It includes: a data acquisition module, a score calculation module, and an adjustment module, wherein... The data acquisition module is used to acquire users' emotional data and historical preference data for cabin services and routes. The emotional data includes emotion type and emotional intensity.

[0108] The rating calculation module is used to calculate the service rating of each cabin service and the route rating of each alternative route based on the emotion data and the historical preference data. The service rating is used to characterize the degree of matching between the cabin service and the emotion data and historical preference data, and the route rating is used to characterize the degree of matching between the alternative routes and the emotion data and historical preference data.

[0109] The adjustment module is used to adjust cabin services and determine recommended routes based on the service rating of each cabin service and the route rating of each alternative route.

[0110] The vehicle cockpit control device provided in this embodiment belongs to the same concept as the vehicle cockpit control method provided in the above embodiments of this application. It can execute the vehicle cockpit control method provided in any of the above embodiments of this application and has the corresponding functional units and beneficial effects of the vehicle cockpit control method. Technical details not described in detail in this embodiment can be found in the specific processing content of the vehicle cockpit control method provided in the above embodiments of this application, and will not be repeated here.

[0111] Exemplary System In some embodiments of this application, such as Figure 7 As shown, a vehicle cockpit control system is provided. The vehicle cockpit control system includes a multimodal sensor 701 and a vehicle-side controller 702.

[0112] The vehicle-side controller 702 is communicatively connected to the multimodal sensor 701 and the cloud server 703. The multimodal sensor 701 is used to collect multimodal data, including user biometrics, user behavior information, vehicle environmental data, and road data.

[0113] The vehicle-side controller 702 is used to execute the vehicle cockpit control method described in any of the foregoing embodiments.

[0114] The cloud server 703 is used to transmit update data packets to the vehicle controller according to a preset period to update the vehicle's emotion recognition algorithm parameters, preset emotion data and route adaptation rules, preset emotion data and cabin service adaptation rules, and users' historical preference data for cabin services and routes.

[0115] Exemplary vehicle In some embodiments of this application, a vehicle is provided, comprising: The vehicle cockpit control system described above.

[0116] The vehicle can be a gasoline-powered vehicle, an electric vehicle, or a hybrid vehicle. The vehicle can be an autonomous vehicle or a manned vehicle. The vehicle can be an electric passenger vehicle or a commercial vehicle, such as a light truck, bus, or sanitation vehicle.

[0117] Exemplary electronic devices In some embodiments of this application, an electronic device is provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the vehicle cockpit control method described above.

[0118] Specifically, the aforementioned electronic device may further include: a bus, a communication interface, input devices, and output devices. The processor, memory, communication interface, input devices, and output devices are interconnected via the bus. The bus may include a pathway for transmitting information between various components of the computer system.

[0119] The processor can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0120] The processor may include the main processor, as well as baseband chips, modems, etc.

[0121] The memory stores a computer program that executes the technical solution of this application, and may also store an operating system and other critical business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0122] The computer program can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this specification. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0123] Input devices may include devices that receive data and information input by the user, such as keyboards, mice, cameras, scanners, light pens, voice input devices, touch screens, pedometers, or gravity sensors.

[0124] Output devices may include devices that allow information to be output to the user, such as displays, printers, speakers, etc.

[0125] The communication interface may include any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0126] The processor executes the program stored in the memory and calls other devices, which can be used to implement the various steps of the vehicle cockpit control method provided in the above embodiments of this application.

[0127] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This disclosure is not limited to any particular combination of hardware and software.

[0129] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It should also be understood that terms such as those defined in a common dictionary should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0130] The foregoing description is intended to illustrate the present disclosure and should not be construed as limiting it. While several exemplary embodiments of the present disclosure have been described, those skilled in the art will readily understand that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined by the claims. It should be understood that the foregoing description is intended to illustrate the present disclosure and should not be construed as limiting it to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

[0131] Exemplary computer-readable storage media In some embodiments of this application, a computer-readable storage medium is provided, which stores a computer program for performing the steps in the vehicle cockpit control methods of the various embodiments described above.

[0132] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0133] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program information that, when run by a processor, causes the processor to execute the steps in the vehicle cockpit control methods of various embodiments of this application.

[0134] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0135] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0136] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0137] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

Claims

1. A vehicle cockpit control method, characterized in that, include: Acquire user emotion data and user historical preference data for cabin services and routes, wherein the emotion data includes emotion type and emotion intensity; Based on the sentiment data and the historical preference data, a service score for each cabin service and a route score for each alternative route are calculated, wherein the service score is used to characterize the degree of matching between the cabin service and the sentiment data and the historical preference data, and the route score is used to characterize the degree of matching between the alternative routes and the sentiment data and the historical preference data. Based on the service rating of each cabin service and the route rating of each alternative route, the cabin services are adjusted and a recommended route is determined.

2. The vehicle cockpit control method according to claim 1, characterized in that, The acquisition of user sentiment data includes: Acquire multimodal data collected by multimodal sensors, including user biometrics, user behavioral information, vehicle environmental data, and road data; Based on the multimodal data, the user's emotional data is determined.

3. The vehicle cockpit control method according to claim 2, characterized in that, The process of determining the user's emotional data based on the multimodal data includes: The multimodal data is fused and cross-validated based on an emotion recognition algorithm to generate initial emotion data, which includes initial emotion type and initial emotion intensity. When the initial emotional intensity is greater than or equal to a preset emotional intensity threshold, a preliminary emotional data verification is initiated to the user so that the user can provide the first feedback information. Based on the first feedback information, the user's preliminary emotional data is corrected to determine the user's emotional data.

4. The vehicle cockpit control method according to claim 1, characterized in that, The calculation of a service rating for each cabin service and a route rating for each alternative route based on the sentiment data and historical preference data includes: Based on the user's emotional data, preset emotional data, and route adaptation rules, the emotional adaptation score of each alternative route is determined. Based on the user's emotional data, preset emotional data and cabin service adaptation rules, determine the emotional adaptation score for each cabin service; Based on users' historical preference data for cabin services and routes, determine the user's preference score for each cabin service and the user's preference score for each alternative route; Based on the intensity of the emotion, determine the emotion weight and the user preference weight; The service score for each cabin service is calculated based on the emotional fit score, emotional weight, user preference score for each cabin service, and user preference weight. The route score for each alternative route is calculated based on the emotional fit score, emotional weight, user preference score for each alternative route, and user preference weight.

5. The vehicle cockpit control method according to claim 1, characterized in that, When there are at least two users, the multimodal data also includes user identity information, voice command information, and touch command information; The method further includes: Based on the user's instruction information, the user's identity information, and the user's emotional data, determine the level of conflict of needs among users in the cockpit; Based on the user's conflict level and the preset conflict level handling rules, a first candidate set is generated and prompted to the user. The first candidate set includes instruction execution methods and cabin services.

6. The vehicle cockpit control method according to claim 5, characterized in that, The user's conflict of needs includes driving style conflict, emotional conflict, and cabin service conflict. Determining the level of user conflict of needs within the cabin based on the user's instruction information, the user's identity information, and the user's emotional data includes: Based on the user's instruction information, the user's identity information, and the user's emotional data, determine whether the user has a conflict of driving style, emotional conflict, or cabin service conflict. When the user has a conflicting driving style, it is determined that the user in the cockpit has a Level 1 conflict of needs. When the user is experiencing emotional conflict, it is determined that the user in the cabin has a secondary need conflict. When a user has a cabin service conflict, it is determined that the user has a level three demand conflict within the cabin.

7. The vehicle cockpit control method according to claim 6, characterized in that, The step of generating a first candidate set based on the user's conflict level and preset conflict level processing rules includes: When there is a primary demand conflict among the users in the cockpit, a first candidate set is generated, which includes the instruction execution method corresponding to the driver. When there is a secondary demand conflict among the users in the cabin, the priority of the users is determined based on the emotional intensity and identity information of the users, and the users with the highest priority are selected to generate a first candidate set including the cabin services corresponding to the users with the highest priority. When there is a conflict of three levels of needs among the users in the cabin, a first candidate set is generated, which includes cabin services that correspond to the compromise of the users' needs.

8. The vehicle cockpit control method according to claim 1, characterized in that, The method further includes: Obtain user feedback on the cabin service and the recommended route; Determine the target route based on user feedback on the recommended routes; The user's emotional data, feedback information, cabin services, and target route are uploaded to the cloud server.

9. A vehicle cockpit control system, characterized in that, include: Multimodal sensors and vehicle-side controllers, among which, The vehicle-mounted controller is communicatively connected to the multimodal sensors and the cloud server, respectively. The multimodal sensor is used to collect multimodal data, which includes the user's biological information, the user's behavioral information, the vehicle's environmental data, and road data. The vehicle-mounted controller is used to execute the vehicle cockpit control method as described in any one of claims 1 to 8.

10. A vehicle, characterized in that, include: The vehicle cockpit control system as described in claim 9.

Images