Intelligent cockpit personalized scene recommendation system based on AI large model

Abstract

The invention relates to the technical field of intelligent cabins, in particular to an intelligent cabin personalized scene recommendation system based on an AI large model, and the system comprises a multi-modal data sensing layer, a domain controller, an AI model engine, a scene execution and interaction module and a feedback optimization layer which achieve step-by-step connection based on the Internet of Vehicles technology. According to the intelligent cockpit personalized scene recommendation system based on the AI large model, a five-layer system architecture is constructed, a full-link closed loop from data acquisition to scene landing to model iteration is realized, and data privacy security is ensured by adopting desensitization processing; according to a safety and logic double-rule base, forcibly checking a recommended scene, and avoiding driving and hardware execution risks; recommendation presentation and feedback collection are achieved in a multi-mode mode, scene execution states are monitored in real time, explicit and implicit feedback is collected at the same time, a model is optimized through an offline mechanism, an online mechanism and a reinforcement learning mechanism, and continuous evolution of the model is ensured through OTA deployment after shadow mode and AB test evaluation.

Description

Technical Field

[0001] This invention relates to the field of intelligent cockpit technology, specifically to an intelligent cockpit personalized scene recommendation system based on an AI large model. Background Technology

[0002] Cockpit scenario recommendation refers to the system's proactive combination of discrete hardware functions such as seats, air conditioning, audio, lighting, and screens into a holistic service solution for users based on real-time perception and deep understanding of people, vehicles, and the environment, through reasoning using large AI models. It is an inevitable trend in the development of intelligent vehicles, capable of reshaping the relationship between people and vehicles, and possesses strong application imagination and commercial potential.

[0003] A search revealed that Chinese Patent Publication No. CN119443234A discloses a cockpit scene recommendation method, device, storage medium, program product, and vehicle. The cockpit scene recommendation method in this patent involves preprocessing the human-machine dialogue text data from the vehicle's voice tracking logs to obtain the user's natural language description; then, the user's natural language description is filtered for similarity in a pre-set scene knowledge base to obtain the scene knowledge corresponding to the user's natural language description; next, the user's natural language description and the scene knowledge are concatenated to obtain scene recommendation prompts for the vehicle's cockpit; finally, a pre-set large-scale intelligent model performs scene action reasoning on the scene recommendation prompts and outputs a list of scene action executions corresponding to the scene knowledge. Because this application converts human-machine dialogue text data into user natural language descriptions and filters corresponding scene knowledge, relying on the powerful text understanding and generation capabilities of the large-scale intelligent model for scene action reasoning, it can automatically provide intelligent cockpit scene recommendation services based on the driver's voice descriptions. However, this recommendation process relies solely on a linear logic of text preprocessing, similarity filtering of scene knowledge, prompt word concatenation, and large-scale model inference. Its core function is similarity filtering based on a scene knowledge base, and the recommendation results are simply filtered using a relevance framework, failing to consider dynamic factors such as real-time user status changes, scene execution security, and user preference intensity. This results in a simplistic recommendation generation logic, lacking dynamic decision-making capabilities and security verification mechanisms. Furthermore, it lacks user interaction feedback channels, execution status monitoring, and model optimization mechanisms, leading to insufficient interaction and execution aspects. Therefore, to address these technical issues, a personalized scene recommendation system for intelligent cockpits based on an AI large-scale model is proposed. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides an intelligent cockpit personalized scene recommendation system based on an AI large model. It has the advantages of achieving a closed-loop feedback optimization across the entire chain based on a multi-level decision-making system and full lifecycle management, thus solving the problems in the existing cockpit scene recommendation methods mentioned above that require further improvement in dynamic decision-making, safety verification, and interactive feedback.

[0005] (II) Technical Solution To achieve the aforementioned goal of realizing a closed-loop feedback optimization across the entire chain based on a multi-level decision-making system and full lifecycle management, this invention provides the following technical solution: a personalized scene recommendation system for intelligent cockpits based on an AI large model, comprising a multi-modal data perception layer, a domain controller, an AI model engine, a scene execution and interaction module, and a feedback optimization layer, all connected hierarchically based on vehicle networking technology.

[0006] Preferably, the multimodal data perception layer includes a user perception module, a vehicle perception module, an environment perception module, and a data transmission module, wherein the data acquisition and transmission process includes: 1) The user perception module is equipped with a driver monitoring system (DMS) camera, an in-vehicle occupant monitoring system (OMS) camera, and biosensors. The DMS camera captures facial images of the driver to identify the driver's fatigue state (including yawning and blinking frequency), distraction state (direction and angle of gaze away from the road), and facial emotions (including happiness, calmness, and frustration). The OMS camera is used to identify the number, location, age (including adults or children, men or women) of passengers in the vehicle, as well as their facial emotional state; the biosensor is integrated into the steering wheel to collect the driver's heart rate and skin conductance; and it is integrated into the seat to identify the passenger's weight distribution and determine whether there are passengers, in order to collect human physiological signals.

[0007] 2) The vehicle perception module collects body domain data and cabin domain data by connecting to CAN bus, LIN bus, Ethernet and vehicle sensors. The body domain data includes vehicle speed, acceleration, turn signal status, gear position, driving range, fuel level or battery level, tire pressure, door and trunk opening and closing status. The cabin domain data includes air conditioning set temperature, air volume and mode, current media source and volume, seat position, heating, ventilation and massage status, steering wheel position, ambient lighting color and brightness, and window and sunroof status.

[0008] 3) The environmental perception module collects external and internal environmental data by connecting to the BeiDou positioning system, the vehicle network cloud, and environmental sensors. The external environmental data includes geographical location, speed, altitude, and time information provided by the BeiDou positioning system. Real-time weather information (including temperature, humidity, rainfall and snowfall), traffic conditions (congestion, accident locations) and current air quality index (AQI) are obtained through the vehicle-to-everything (V2X) cloud. An ambient light sensor detects the intensity of external light to automatically adjust and control the screen brightness, and a rain or snow sensor determines whether it is raining or snowing and the amount of rain or snow. The internal environmental data includes actual temperature and humidity data detected by cabin temperature and humidity sensors, and concentration data of PM2.5, CO2, etc. detected by air quality sensors.

[0009] 4) Implement edge computing, preprocessing, and standardization in the data transmission module.

[0010] Preferably, the domain controller access data transmission module performs data anonymization and security processing, wherein the data anonymization processing is completed locally on the cockpit domain controller. While preserving data availability, it removes or obscures personally identifiable information, mainly covering three types of data: 1) User biometric data: facial images, heart rate, and voice waveform data; 2) Account-linked data: Personal preference ID and historical travel trajectory data synchronized to the cloud; 3) Vehicle identification data: VIN (Vehicle Identification Number) and domain controller unique identification data; The specific steps for data anonymization include: 1) The preprocessed standardized data are classified into levels according to their sensitivity, and the processing priority is determined using the sensitivity coefficient calculation method. The formula is as follows: , where S is the data sensitivity coefficient, ranging from 0 to 10; Assuming a weighted average based on identity relevance, Weighting of privacy breach risk percentages Weighting based on the irreplaceability of data; Score the relevance of identity. Assess the risk of privacy breaches. Scoring based on the irreplaceability of the data; 2) The corresponding desensitization algorithm is adopted according to the sensitivity level. All processing is completed locally on the cabin domain controller without uploading the original data. 3) After data anonymization, an availability-privacy balance check is used to ensure the data meets requirements. The check formula is as follows: Where B is the balance score, Let A be the weight, and P be the data availability score and P be the privacy protection score.

[0011] Preferably, the AI ​​model engine deploys a data fusion module, a user profiling model, a scene understanding and prediction model, and a recommendation ranking model. The specific process includes: 1) After the data fusion module receives the standardized data snapshot from the perception layer, the Transformer-based encoder aligns and fuses the standardized data in the time dimension, outputs a context vector, and fuses the data into a context description; 2) The user profiling model models the sequential dependencies of user behavior based on Transformer to build user profiles. It receives contextual descriptions through the user profiling model, thereby inferring the instantaneous state of users in real time and learning stable user preferences based on historical data.

[0012] 3) The scene understanding and prediction model is deployed based on the large language model LLM. The system constructs prompt words and inputs the prompt words into the scene understanding and intent prediction model. The model infers the user's possible needs based on the fused context snapshot, user profile and scene list, selects the most suitable scene from the scene library, outputs scene recommendation and inference logic, and deeply understands the current scene and behavioral intent. 4) Introduce a scene knowledge base into the scene understanding and prediction model. Set up a predefined scene library and a custom scene library in the scene knowledge base. Use the rule template from the scene knowledge base as the recommendation generation model. Receive parameter output from the large language model LLM through the recommendation generation model to generate cockpit function parameter combinations and recommended scenes. The recommendation ranking model is deployed based on the Learning to Rank model. By combining success probability, user preference strength and scenario novelty, the Learning to Rank model ranks the generated recommendation scenarios and selects the optimal solution. 5) Introduce a rule engine to the recommendation ranking model. The rule engine is divided into a security rule base and a logic rule base. The optimal scenario recommendation output is combined with the rule engine for verification. If the engine verification passes, the recommended scenario is sent to the cockpit domain controller. Otherwise, the recommended scenario is regenerated or the scenario is downgraded.

[0013] Preferably, the scene execution and interaction module is equipped with an execution control module and an interactive display module as main modules. The execution control module is further divided into an instruction translator, an actuator driver interface, and an electronic control unit (ECU). The interactive display module is further divided into a multimodal output manager, a multimodal input manager, and a scene state manager. The steps for implementing instruction execution control include: 1) The multimodal output manager manages all scene recommendation output channels, which are used to receive recommended scene instructions output by the AI ​​model engine after making a decision. The recommended scene instructions are first sent to the interactive display module, and the interaction layer selects the best way to prompt the user according to the urgency and type of the scene. 2) The multimodal input manager is used to receive feedback data on scene recommendations. The multimodal input manager is connected to the multimodal output manager, and all sensors are activated at the same time to wait for user feedback and capture the user feedback. 3) The execution control module starts working when the user agrees or the policy allows it to be executed by default.

[0014] Preferably, the feedback optimization layer includes a data acquisition module, a data storage and processing module, a model training optimization module, and a model evaluation module, which implements the system feedback optimization process as follows: 1) A data interface is set up in the data acquisition module to collect explicit and implicit feedback data on user acceptance of the recommended scenarios from the interactive display module. This data is mainly used for subsequent AI model iteration. Explicit feedback refers to clear interactive behaviors initiated by users, which directly express their acceptance or rejection of the scenario, such as voice commands and touch operations. Implicit feedback refers to user behavior without active interaction, which can indirectly reflect the scenario's suitability through behavioral data. The data collected from explicit and implicit feedback is standardized using a feedback intensity quantification algorithm. 2) The raw feedback data collected is cleaned, labeled, and formatted in the data storage and processing module; 3) The processed feedback data is used to form training samples and input into the model training optimization module to optimize and iterate the AI ​​model; 4) Based on the updated and optimized AI model, the new and optimized model is deployed to the vehicle system via over-the-air (OTA) download technology in the model evaluation module to evaluate and verify the model iteration effect.

[0015] (III) Beneficial Effects Compared with existing technologies, this invention provides a personalized scene recommendation system for intelligent cockpits based on AI large models, which has the following beneficial effects: 1. This AI-based intelligent cockpit personalized scene recommendation system constructs a five-layer system architecture to achieve a closed-loop process from data collection to scene implementation and model iteration. The collected data is anonymized to ensure data privacy and security.

[0016] 2. This AI-based intelligent cockpit personalized scene recommendation system uses the Transformer model to achieve multimodal data fusion and user profile construction, combines the LLM model to infer scene requirements, and then uses the Learning to Rank algorithm to sort candidate scenes. Based on a dual rule library of security and logic, the recommended scenes are forcibly verified to avoid driving and hardware execution risks.

[0017] 3. This AI-based intelligent cockpit personalized scene recommendation system uses a multimodal approach to present recommendations and collect feedback, monitors the scene execution status in real time, and collects explicit and implicit feedback. It optimizes the model using three mechanisms: offline, online, and reinforcement learning. After evaluation through shadow mode and A / B testing, it deploys new models via OTA to ensure continuous model evolution. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the architecture of the intelligent cockpit personalized scenario recommendation system of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1 In this embodiment, the data transmission module implements edge computing and preprocessing, as well as standardization processing. Specific processing steps include: 1) Signal input: Based on the continuous acquisition of raw signals by various sensors deployed inside and outside the vehicle, RAW image data is output through the camera, analog audio waveforms are output through the microphone, and binary data frames are output through the CAN bus.

[0021] 2) Edge computing and preprocessing: Raw data is transmitted to the Electronic Control Unit (ECU) for initial processing. Most of the processing is done locally. The processed data is divided into: Analog-to-digital converter (ADC) – converts analog signals into digital signals; Local AI inference—Deploying lightweight AI models on chips with integrated neural network processing units (NPUs); DMS chip — analyzes images directly locally and outputs structured results instead of uploading the raw video stream; Voice recognition chip — performs automatic voice recognition, converts voice waveforms into text commands, and then uploads the text; Signal processing—filtering data noise, enhancing and correcting images.

[0022] 3) Data standardization: Convert data from different suppliers and agreements into a unified data format defined within the system.

[0023] Example 2 In this embodiment, a corresponding desensitization algorithm is adopted according to the sensitivity level, specifically including: 1) Highly sensitive desensitization of user biometric data, applicable scenarios include facial images and heart rate data. For facial image data desensitization, a combination of Gaussian blur and feature desensitization algorithms is used. The formula for Gaussian blur processing of the image is as follows: Where G(x,y) is the pixel value of the blurred image at the (x,y) coordinates, and σ is the blur coefficient. , () represents the coordinates of the center point of the fuzzy kernel. These are the original image pixel values. This represents a convolution operation; after blurring, facial feature points are removed, retaining only the structured results of "whether there is a face" and "emotion category," achieving highly sensitive desensitization processing. The differential privacy perturbation algorithm is used to add Laplace noise to the raw heart rate values ​​of the heart rate data. The formula is as follows: ,in This is the heart rate value after desensitization. This is the original heart rate value. For Laplace distributed noise (location parameter is 0, scale parameter is...) ); Δf represents the sensitivity of heart rate data, and ϵ represents the privacy budget; 2) Sensitive data masking for account-related data, applicable scenarios include personal preference IDs and historical travel records. A hash masking algorithm is used to process the user account-related personal preference IDs using SHA-256 hashing. The formula is as follows: ,in This is the hash value after de-identification. For the original preference ID, A random salt value generated locally by the cockpit domain controller; A coordinate offset algorithm is used to randomly offset the latitude and longitude coordinates in the historical travel trajectory. The formula is as follows: ,in and These represent the desensitized longitude and latitude, respectively. and These represent the original longitude and original latitude, respectively. This is the offset distance. This is the offset angle; 3) Low-sensitivity desensitization of vehicle and environmental data, applicable scenarios include VIN codes and cabin temperature and humidity. A partial character replacement method is used to retain the first 8 characters of the VIN code, while the last 8 characters are replaced with "". ”; Numerical normalization was used to normalize the continuous temperature and humidity data inside the cabin, retaining only the relative trend. The formula is as follows: ,in These are normalized values, ranging from 0 to 1. The original data values, and These represent the minimum and maximum reasonable values ​​for this data type, respectively.

[0024] Example 3 In this embodiment, the inference process in the scene understanding and prediction model is exemplified as follows: Current scenario: {Time: Wednesday evening, 8 PM; Vehicle status: Driving; Location: On the way home from the gym; User status: High heart rate; Expression: Tired; Cabin temperature: 28℃}; User profile: {Long-term preference: Enjoys listening to relaxing electronic music after exercising and sets the air conditioner to 20℃}; Available scenarios: [Relaxation Mode; Refreshing Mode; Homecoming Mode; Family Mode, etc.]; Output requirements: Please output in the following format: Reasoning: <Your Reasoning>; Recommended Scenario: <Scenario Name>; Parameters: <Specific Parameters>.

[0025] The large language model LLM outputs the following based on the above example inference process: Reasoning: The user has just finished exercising and is physically and mentally tired with a high body temperature. His historical preference is to listen to electronic music and use low-temperature air conditioning to relax and cool down. Therefore, the soothing mode is recommended, but the music type should be his preferred electronic music, and the air conditioning should be turned down. Recommended scenario: Soothing mode; Parameters: {Air conditioning: 20℃; Music: Relaxing electronic playlist; Seat ventilation: On} The verifier will check the safety and compliance of this recommendation, including "turn on seat ventilation while driving" - pass safely; "play music" - pass safely.

[0026] Example 4 In this embodiment, if the engine verification fails, the recommended scene is regenerated or the scene is downgraded. The specific steps include: 1. Based on a dual rule base of security and logic, mandatory verification is performed on the recommendation scenario. This mandatory verification is quantitatively verified using a rule matching degree algorithm and a logical conflict coefficient algorithm. The steps include: 1) From the rule engine of the scene knowledge base, retrieve a subset of rules relevant to the current recommendation scene. The rule base is divided into a security rule base and a logic rule base. A rule relevance ranking algorithm is used to filter core rules, and the formula is as follows: ,in Let R be the relevance score between rule R and scenario S, C(S,R) be the overlap between scenario parameters and rule constraints, and M(S,R) be the matching degree between the scenario type and the rule applicable type. 2) For core related safety rules, the scenario parameters are verified to ensure they meet driving safety and hardware safety thresholds. The verification results are quantified using a safety compliance coefficient algorithm, the formula of which is: Where Scomp is the safety compliance coefficient, k is the number of parameters in the scenario that must satisfy the constraint of not exceeding the maximum value, and l is the number of parameters in the scenario that must satisfy the constraint of not being less than the minimum value. / The safety weight for the i / j-th parameter, Let i be the value of the i-th parameter in the scene. The maximum safe value for the i-th parameter. Let j be the value of the j-th parameter in the scene. The minimum safe value for the j-th parameter; 3) For core related logical rules, verify whether there are logical conflicts in the parameter combinations within the scenario, and use a logical conflict coefficient algorithm to quantify the degree of conflict. The formula is as follows: ,in The logical conflict coefficient, This represents the total number of functional parameters included in the scene. Let be the logical correlation coefficient between parameters p and q. The degree of conflict between the values ​​of parameter p and parameter q; 4) Based on the combined security compliance coefficient and logical conflict coefficient, output the final verification result and execute the corresponding operation. The arbitration rules are as follows: like and Once the time verification passes, the scenario is sent to the cockpit domain controller for execution. like <0.9 or If the value is greater than 0.1, the verification fails, triggering the regeneration of recommended scenarios or scenario downgrade processing; If 0.8 < <0.9 and When the process enters the manual confirmation stage, the user is prompted by voice or screen to confirm whether to proceed.

[0027] 2. When the optimal scenario recommendation fails the rule engine verification, the system triggers a scenario regeneration mechanism. The specific steps include: 1) Retrieve alternative templates from the scene knowledge base that are of the same type as the original recommended scene but with a parameter difference of ≥30%, and construct a temporary candidate pool to expand the candidate scene pool. The difference is calculated using the Euclidean distance formula, which is as follows: ,in This is the original recommended scenario. This is a list of candidate scene templates, where n represents the total number of functional parameters included in the scene. The weight of the i-th functional parameter is preset based on the intensity of user preference and ranges from 0 to 1. This represents the standardized value of the i-th parameter in the original recommendation scenario. The standardized value of the i-th parameter in the alternative scenarios; 2) Extract the rejection reasons from the rule engine and convert them into mathematical constraints. Then, filter the scene templates in the temporary candidate pool using parameters and remove scenes that still violate the constraints. 3) A weighted scoring algorithm is used to re-rank the filtered candidate scenes, and the scene with the highest score is selected as the new recommendation. The scoring formula is as follows: in The overall score for scenario S (range 0-100). , , All represent weight coefficients and α+β+γ=1. P(S) is the predicted probability that scenario S passes the rule verification (calculated based on historical verification data using a logistic regression model, ranging from 0 to 1). The matching degree between scenario S and user's long-term preferences (based on the user profile model output, ranging from 0 to 1). The novelty of scene S; 3. When regenerating the scene still fails to meet the requirements or requires rapid adaptation, the scene degradation mechanism is triggered. Specific steps include: 1) Parameter Classification and Coreness Division: Scene parameters are divided into core parameters (affecting core user needs) and non-core parameters (auxiliary needs). Their coreness is calculated using the entropy weight method, with the following formula: in Let m be the probability of the j-th value of the i-th parameter, m be the number of parameter values, and core degree. Parameters with a value ≥0.3 are defined as core parameters; 2) For core parameters that violate constraints, adjust them to within a safe threshold according to the gradient decay formula, which is as follows: ,in These are the parameter values ​​after the downgrade. This is the maximum safe value allowed by the rules engine. The original parameter value. This is the attenuation coefficient (dynamically adjusted based on the intensity of user preference). 3) Re-enter the adjusted parameter combination into the rule engine for verification. If it passes, a downgraded scenario is generated; otherwise, if it fails, the gradient adjustment of the k value is repeated until it passes or the minimum adjustment threshold is reached.

[0028] Example 5 In this embodiment, the multimodal input manager connects to the multimodal output manager to capture user feedback, and the types of feedback received include: 1) Explicit positive feedback: The user replies with a voice message "OK" or clicks "Confirm"; 2) Explicit negative feedback: The user replies with a voice message "No need" or clicks "Cancel"; 3) No feedback, default execution: If the user does not take any action, it may be assumed to be agreed by default, and the system will act according to the preset strategy after the timeout; 4) Implicit interrupt: The user can directly perform the opposite operation manually.

[0029] Data collected from explicit and implicit feedback is standardized using a feedback intensity quantization algorithm, specifically: 1) The system monitors the entire lifecycle of scene execution in real time and starts feedback collection when the following trigger conditions are met: when scene recommendation is presented (such as within 0-5 seconds after a screen pop-up or voice prompt). During scenario execution (such as when the user manually adjusts the seat angle or air conditioning temperature); After the scene execution ends (e.g., within 3 seconds after the 15-minute rest mode cycle ends), a trigger sensitivity algorithm is used to determine whether to start data collection. The formula is as follows: ,in To trigger sensitivity, The number of user interactions after scene recommendation. This represents the deviation between user behavior and recommended parameters during scenario execution. Total execution time of the scene; 2) Differentiated data collection methods are used for explicit and implicit feedback, relying on the multimodal input manager of the scene execution and interaction layers. Specifically, this is divided into: a. For explicit feedback collection, the voice interaction module (microphone) and touch interaction module (central control screen, steering wheel buttons) are used as the collection channels. The collection content and judgment rules are shown in the table below: b. For implicit feedback acquisition, the cockpit domain sensors (seat adjustment motor, air conditioning controller, screen operation recorder) and scene status manager are used as acquisition channels. The acquisition content and judgment rules are shown in the table below: The data storage and processing module cleans, labels, and formats the collected raw feedback data. The steps include: 1) Convert explicit positive and explicit negative feedback data into structured tags and record the interaction timestamp; convert implicit acceptance, implicit rejection, and implicit abort feedback into structured data and associate them with the scene ID and the values ​​before and after parameter modification. 2) The collected unstructured feedback behaviors are converted into quantized scores for AI model training and optimization. A feedback intensity quantization algorithm is used, with the following formula: in To quantify the intensity of feedback, , , All are weighting coefficients and + + =1, As the base score for feedback type, For feedback response speed correction coefficient, Correction coefficient for the degree of parameter modification; 3) Associate the quantized feedback data with the original context data that triggered the feedback to form a complete training sample, the structure of which includes: Input: {context: time, location, user status...}; Output: {action: system-recommended scenario and parameters}; Label: {reward:+1 (accept) or -1 (reject)} or {optimal action: manually set by the user to 24℃}; Then, the data is stored in the data storage and processing module of the feedback optimization layer, using an encrypted format (AES-256) to ensure the privacy and security of the feedback data.

[0030] The processed feedback data is used as training samples and input into the model training and optimization module to optimize and iterate the AI ​​model, specifically including: 1) Collect feedback data using an offline learning mechanism, perform batch processing in the cloud, and retrain or fine-tune the AI ​​model. For example, for batch data, the offline learning mechanism starts the training task in the cloud and uses massive samples to supervise and fine-tune the model. 2) Based on important feedback data, the system uses an online learning mechanism to fine-tune the model parameters in real time or near real time. For example, the preprocessed data is input into the model optimization module. For clear negative feedback data, the online learning mechanism slightly lowers the weight of the recommendation strategy. 3) Using the recommendation system as the intelligent agent, the cabin environment as the environment, positive feedback as the reward, and negative feedback as the penalty, the RL algorithm is combined to train the model for reinforcement learning. For example, for the reinforcement learning mechanism, user acceptance is the positive reward and rejection is the negative reward. The model's decision-making strategy is optimized and the long-term reward is maximized. Based on the updated and optimized AI model, the model evaluation module uses over-the-air (OTA) technology to deploy the validated and optimized new model into the vehicle system to evaluate and verify the model iteration effect. Specifically, this includes: 1) Test on historical data based on offline evaluation to verify whether its key indicators have improved; 2) Setting up shadow mode will run the updated model and the current model in parallel, comparing the differences in the decisions output by the two models to evaluate their effectiveness and safety; 3) Set up an AB test mode to group the vehicle cabins, with the old model test as group A and the new model test as group B. Evaluate the model's merits by comparing the feedback data from the two groups. 4) Generate new user experience data based on the updated model. This experience data is then integrated into the scene execution and interaction module as new interaction data for continuous model training and optimization.

[0031] In summary, this AI-based intelligent cockpit personalized scenario recommendation system achieves a closed-loop process from data collection to scenario implementation and model iteration by constructing a five-layer system architecture. The collected data is anonymized to ensure data privacy and security. Multimodal data fusion and user profile construction are achieved through the Transformer model, scenario requirements are inferred by the LLM model, and candidate scenarios are ranked by the Learning to Rank algorithm. Based on the dual rule base of security and logic, the recommended scenarios are forcibly verified to avoid driving and hardware execution risks. The recommendation presentation and feedback collection are achieved through a multimodal approach, the execution status of the scene is monitored in real time, and explicit and implicit feedback are collected at the same time. The model is optimized through three mechanisms: offline, online and reinforcement learning. After evaluation by shadow mode and A / B testing, the new model is deployed through OTA to ensure the continuous evolution of the model.

[0032] The relevant modules involved in this system are all hardware system modules or functional modules that combine computer software programs or protocols with hardware in the prior art. The computer software programs or protocols involved in these functional modules are technologies known to those skilled in the art and are not improvements to this system. The improvement of this system lies in the interaction or connection between the modules, that is, in improving the overall structure of the system to solve the corresponding technical problems that this system aims to address.

[0033] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A personalized scene recommendation system for intelligent cockpits based on AI large-scale models, characterized in that, It includes a multimodal data perception layer that achieves hierarchical connectivity based on vehicle networking technology, a domain controller, an AI model engine, a scene execution and interaction module, and a feedback optimization layer.

2. The AI-based large-scale intelligent cockpit personalized scene recommendation system according to claim 1, characterized in that, The multimodal data perception layer includes a user perception module, a vehicle perception module, an environment perception module, and a data transmission module, wherein the data acquisition and transmission process includes: 1) The user perception module is equipped with a driver monitoring system (DMS) camera, an in-vehicle occupant monitoring system (OMS) camera, and biosensors. The DMS camera captures facial images of the driver to identify the driver's fatigue state (including yawning and blinking frequency), distraction state (direction and angle of gaze away from the road), and facial emotions (including happiness, calmness, and frustration). The OMS camera is used to identify the number, location, age (including adults or children, men or women) of passengers in the vehicle, as well as their facial emotional state; the biosensor is integrated into the steering wheel to collect the driver's heart rate and skin conductance; and it is integrated into the seat to identify the passenger's weight distribution and determine whether there are passengers, in order to collect human physiological signals. 2) The vehicle perception module collects body domain data and cabin domain data by connecting to CAN bus, LIN bus, Ethernet and vehicle sensors. The body domain data includes vehicle speed, acceleration, turn signal status, gear position, driving range, fuel level or battery level, tire pressure, door and trunk opening and closing status. The cabin domain data includes air conditioning set temperature, air volume and mode, current media source and volume, seat position, heating, ventilation and massage status, steering wheel position, ambient lighting color and brightness, and window and sunroof status. 3) The environmental perception module collects external and internal environmental data by connecting to the BeiDou positioning system, the vehicle network cloud, and environmental sensors. The external environmental data includes geographical location, speed, altitude, and time information provided by the BeiDou positioning system. Real-time weather information (including temperature, humidity, rainfall and snowfall), traffic conditions (congestion, accident locations) and current air quality index (AQI) are obtained through the vehicle-to-everything (V2X) cloud. An ambient light sensor detects the intensity of external light to automatically adjust and control the screen brightness, and a rain or snow sensor determines whether it is raining or snowing and the amount of rain or snow. The internal environmental data includes actual temperature and humidity data detected by cabin temperature and humidity sensors, and concentration data of PM2.5, CO2, etc. detected by air quality sensors. 4) Implement edge computing, preprocessing, and standardization in the data transmission module.

3. The AI-based large-scale intelligent cockpit personalized scene recommendation system according to claim 2, characterized in that, The domain controller access data transmission module performs data anonymization and security processing. This data anonymization is completed locally on the cockpit domain controller, removing or obfuscating personally identifiable information while preserving data usability. This primarily covers three types of data: 1) User biometric data: facial images, heart rate, and voice waveform data; 2) Account-linked data: Personal preference ID and historical travel trajectory data synchronized to the cloud; 3) Vehicle identification data: VIN (Vehicle Identification Number) and domain controller unique identification data; The specific steps for data anonymization include: 1) The preprocessed standardized data are classified into levels according to their sensitivity, and the processing priority is determined using the sensitivity coefficient calculation method. The formula is as follows: , where S is the data sensitivity coefficient, ranging from 0 to 10; Assuming a weighted average based on identity relevance, Weighting of privacy breach risk percentages Weighting based on the irreplaceability of data; Score the identity relevance. Assess the risk of privacy breaches. Scoring based on the irreplaceability of the data; 2) The corresponding desensitization algorithm is adopted according to the sensitivity level. All processing is completed locally on the cabin domain controller without uploading the original data. 3) After data anonymization, an availability-privacy balance check is used to ensure the data meets requirements. The check formula is as follows: Where B is the balance score, Let A be the weight, and P be the data availability score and P be the privacy protection score.

4. The AI-based large-scale intelligent cockpit personalized scene recommendation system according to claim 1, characterized in that, The AI ​​model engine deploys a data fusion module, a user profiling model, a scene understanding and prediction model, and a recommendation ranking model. The specific process includes: 1) After the data fusion module receives the standardized data snapshot from the perception layer, the Transformer-based encoder aligns and fuses the standardized data in the time dimension, outputs a context vector, and fuses the data into a context description; 2) The user profiling model models the sequential dependencies of user behavior based on Transformer to build user profiles. It receives contextual descriptions through the user profiling model, thereby inferring the instantaneous state of users in real time and learning stable user preferences based on historical data. 3) The scene understanding and prediction model is deployed based on the large language model LLM. The system constructs prompt words and inputs the prompt words into the scene understanding and intent prediction model. The model infers the user's possible needs based on the fused context snapshot, user profile and scene list, selects the most suitable scene from the scene library, outputs scene recommendation and inference logic, and deeply understands the current scene and behavioral intent. 4) Introduce a scene knowledge base into the scene understanding and prediction model. Set up a predefined scene library and a custom scene library in the scene knowledge base. Use the rule template from the scene knowledge base as the recommendation generation model. Receive parameter output from the large language model LLM through the recommendation generation model to generate cockpit function parameter combinations and recommended scenes. The recommendation ranking model is deployed based on the Learning to Rank model. By combining success probability, user preference strength and scenario novelty, the Learning to Rank model ranks the generated recommendation scenarios and selects the optimal solution. 5) Introduce a rule engine to the recommendation ranking model. The rule engine is divided into a security rule base and a logic rule base. The optimal scenario recommendation output is combined with the rule engine for verification. If the engine verification passes, the recommended scenario is sent to the cockpit domain controller. Otherwise, the recommended scenario is regenerated or the scenario is downgraded.

5. The AI-based large-scale intelligent cockpit personalized scene recommendation system according to claim 4, characterized in that, The scene execution and interaction module is deployed with an execution control module and an interactive display module as the main modules. The execution control module is further divided into an instruction translator, an actuator drive interface and an electronic control unit (ECU). The interactive display module is further divided into a multimodal output manager, a multimodal input manager, and a scene state manager. The steps for implementing command execution control include: 1) The multimodal output manager manages all scene recommendation output channels, which are used to receive recommended scene instructions output by the AI ​​model engine after making a decision. The recommended scene instructions are first sent to the interactive display module, and the interaction layer selects the best way to prompt the user according to the urgency and type of the scene. 2) The multimodal input manager is used to receive feedback data on scene recommendations. The multimodal input manager is connected to the multimodal output manager, and all sensors are activated at the same time to wait for user feedback and capture the user feedback. 3) The execution control module starts working when the user agrees or the policy allows it to be executed by default.

6. The AI-based large-scale intelligent cockpit personalized scene recommendation system according to claim 1, characterized in that, The feedback optimization layer includes a data acquisition module, a data storage and processing module, a model training optimization module, and a model evaluation module. The system feedback optimization process includes: 1) A data interface is set up in the data acquisition module to collect explicit and implicit feedback data on user acceptance of the recommended scenarios from the interactive display module. This data is mainly used for subsequent AI model iteration. Explicit feedback refers to clear interactive behaviors initiated by users, which directly express their acceptance or rejection of the scenario, such as voice commands and touch operations. Implicit feedback refers to user behavior without active interaction, which can indirectly reflect the scenario's suitability through behavioral data. The data collected from explicit and implicit feedback is standardized using a feedback intensity quantification algorithm. 2) The raw feedback data collected is cleaned, labeled, and formatted in the data storage and processing module; 3) The processed feedback data is used to form training samples and input into the model training optimization module to optimize and iterate the AI ​​model; 4) Based on the updated and optimized AI model, the new and optimized model is deployed to the vehicle system via over-the-air (OTA) download technology in the model evaluation module to evaluate and verify the model iteration effect.

Images