Interaction method, device, system and equipment based on intelligent cabin

Abstract

The invention provides an interaction method, device, system and equipment based on an intelligent cockpit, and the method comprises the steps: obtaining the state information of a person in the intelligent cockpit, and obtaining a user portrait corresponding to the person from a local database according to the state information of the person; performing feature engineering and token coding on the state information of the personnel and the user portrait corresponding to the personnel to obtain a token sequence; and determining one or more services recommended to the personnel according to the token sequence. The corresponding function can be actively recommended according to the acquired state information of the personnel in the intelligent cabin and the corresponding user portrait, and the user experience can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent cockpits, and particularly to a technology for interaction based on an intelligent cockpit. Background Art

[0002] An intelligent cabin is an integrated digital platform built for intelligent transportation devices (such as intelligent vehicles, intelligent aircraft, etc.), aiming to integrate a variety of IT and artificial intelligence technologies within the intelligent transportation device, providing an intelligent experience for the personnel inside the intelligent cabin and promoting traffic safety.

[0003] With the continuous expansion of the application scenarios of large models, large model applications have also been introduced in the field of intelligent cockpits. Currently, there are various sensor-based systems related to intelligent transportation devices as basic components. For example, an Occupant Monitoring System (OMS), a Driver Monitoring System (DMS), a positioning system (Beidou, GPS, etc.), an Advanced Driver Assistant System (ADAS), an Audio Recognition System (ARS), a Human Machine Interface (HMI), etc. A large model can be deployed as the center in the intelligent cockpit. Personnel inside the intelligent cockpit (such as drivers and / or occupants) can use various operation methods (such as capacitive touch, biosensing, etc.) to interact with the large model through the HMI, enhancing the user experience of the personnel in the intelligent cockpit.

[0004] Existing intelligent cockpits mainly make recommendations for corresponding functions based on the collected data. The recommendation mode is limited by a manually predefined rule library, and can only implement limited recommended services defined by humans, with limited user experience and weak generalization ability. Summary of the Invention

[0005] The purpose of the present invention is to provide an interaction method, device, and equipment based on an intelligent cockpit to at least partially solve the technical problems of low efficiency, lack of dynamic adjustment ability, and insufficient accuracy existing in the prior art.

[0006] According to one aspect of the present invention, an interaction method based on an intelligent cockpit is provided, wherein the method includes:

[0007] Obtain the status information of the personnel inside the intelligent cockpit, and based on the status information of the personnel, obtain the user profile corresponding to the personnel from the local database;

[0008] Perform feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence;

[0009] Determine one or more services to be recommended to the person according to the token sequence.

[0010] Optionally, wherein the obtaining of the status information of the person in the intelligent cockpit includes:

[0011] Obtain the status information of the person in the intelligent cockpit through OMS and / or DMS.

[0012] Optionally, wherein the obtaining of the status information of the person in the intelligent cockpit and obtaining the user profile corresponding to the person from the local database according to the status information of the person further includes:

[0013] Obtain the environmental information of the intelligent cockpit;

[0014] Wherein the performing of feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person includes: performing feature engineering and tokenization encoding on the environmental information of the intelligent cockpit, the status information of the person, and the user profile.

[0015] Optionally, wherein the determining of one or more services to be recommended to the person according to the token sequence includes:

[0016] Input the token sequence into a function recommendation model to determine one or more services to be recommended to the person.

[0017] Optionally, the interactive method based on an intelligent cockpit further includes:

[0018] Display the one or more services to the person through the human-machine interface in the intelligent cockpit;

[0019] Call the corresponding function interface according to the interaction operation information of the person to provide the service confirmed by the person.

[0020] Optionally, the interactive method based on an intelligent cockpit further includes:

[0021] Update the user profile of the person in the local database according to the service confirmed by the person.

[0022] According to another aspect of the present invention, there is provided an interactive device based on an intelligent cockpit, wherein the device includes:

[0023] The first module is used to obtain the status information of the person in the intelligent cockpit and, according to the status information of the person, obtain the user profile corresponding to the person from the local database;

[0024] The second module is used to perform feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence;

[0025] The third module is used to determine one or more services recommended for the person according to the token sequence.

[0026] Optionally, the first module is further used to:

[0027] Obtain the environmental information of the intelligent cockpit;

[0028] Wherein, the second module is used to: perform feature engineering and tokenization encoding on the environmental information of the intelligent cockpit, the status information of the person, and the user profile to obtain a token sequence.

[0029] Optionally, the interactive device based on the intelligent cockpit further includes:

[0030] The fourth module is used to display the one or more services to the person through the man-machine interface in the intelligent cockpit;

[0031] The fifth module is used to call the corresponding function interface according to the interactive operation information of the person to provide the service confirmed by the person.

[0032] According to another aspect of the present invention, there is provided an interactive system based on an intelligent cockpit, wherein the system includes:

[0033] The input layer is used to collect video images in the intelligent cockpit and, according to the video images, determine the status information of the person in the intelligent cockpit;

[0034] The processing layer is used to obtain the status information of the person in the intelligent cockpit, and according to the status information of the person, obtain the user profile corresponding to the person from the local database, perform feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence, determine one or more services recommended for the person according to the token sequence, and call the corresponding function interface of the output layer according to the interactive operation information of the person to provide the service confirmed by the person;

[0035] An output layer for presenting the one or more services through a human-machine interaction interface of the intelligent cockpit, and receiving instructions from an execution layer through corresponding function interfaces to provide services confirmed by the person.

[0036] Compared with the prior art, the present invention provides an interaction method, device, system and equipment based on an intelligent cockpit. The method includes: obtaining status information of a person in the intelligent cockpit, and obtaining a user profile corresponding to the person from a local database according to the status information of the person; performing feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence; determining one or more services recommended to the person according to the token sequence. Optionally, the method further includes: obtaining environmental information of the intelligent cockpit, wherein feature engineering and tokenization encoding are performed on the environmental information of the intelligent cockpit, the status information of the person and the user profile to obtain a token sequence. The present invention can actively recommend corresponding functions according to the obtained status information of the person in the intelligent cockpit and the corresponding user profile, which can improve the user experience. Optionally, the environmental information of the intelligent cockpit can also be incorporated, which can further improve the recommendation accuracy to further improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0038] Figure 1 A schematic flowchart of an interaction method based on an intelligent cockpit according to one aspect of the present invention is shown;

[0039] Figure 2 A schematic diagram of an interaction device based on an intelligent cockpit according to another aspect of the present invention is shown;

[0040] Figure 3 A schematic diagram of an interaction system based on an intelligent cockpit according to still another aspect of the present invention is shown;

[0041] Identical or similar reference numerals in the drawings represent identical or similar components. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The present invention will be further described in detail below with reference to the drawings.

[0043] In a typical configuration of each embodiment of the present invention, the execution subject of the method, each trusted party of the system and / or each module of the device all include one or more processors (CPUs), input / output interfaces, network interfaces and memories.

[0044] Memory may include non - permanent memory in the form of computer - readable media, random access memory (RAM) and / or non - volatile memory such as read - only memory (ROM) or flash RAM. Memory is an example of computer - readable media.

[0045] Computer - readable media includes both permanent and non - permanent, removable and non - removable media that can store information by any method or technology. The information can be computer - readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase - change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read - only memory (ROM), electrically erasable programmable read - only memory (EEPROM), flash memory or other memory technologies, compact disc read - only memory (CD - ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non - transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, computer - readable media does not include transitory media such as modulated data signals and carrier waves.

[0046] Existing intelligent cockpits are mainly applied to intelligent vehicles. According to the collected environmental information, intelligent vehicle status information, and / or relevant information of the people in the cockpit, they provide service recommendations to the people in the cockpit, such as getting - in greetings, vehicle control services, charging services, navigation services, intelligent reporting services, OTA (Over - the - Air) services, etc. However, the existing service recommendation mode based on the intelligent cockpit in intelligent vehicles is limited by a manually predefined rule library, only supports fixed - scenario triggering, can only implement limited service recommendations defined by humans, cannot dynamically adjust according to the real - time status of users, has limited user experience, and weak generalization ability.

[0047] An interaction method based on an intelligent cockpit provided by the present invention performs feature engineering and tokenization encoding on the obtained status information of the people in the intelligent cockpit and the corresponding user portraits, converts multi - modal data into a token sequence suitable for large - model processing, and determines the recommended services according to the obtained token sequence. It can break through the limitation of the rule library, actively recommend services that the people in the cockpit may want to the people in the cockpit for selection. The people in the cockpit can also determine the desired services through interaction operations.

[0048] To further elaborate on the technical means and achieved effects of the present invention, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and various embodiments.

[0049] Figure 1 A schematic diagram showing an interaction method based on an intelligent cockpit according to an aspect of the present invention. In one embodiment, the method includes:

[0050] S101 Obtain the status information of the person in the intelligent cockpit, and based on the status information of the person, obtain the user profile corresponding to the person from the local database;

[0051] S102 Perform feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence;

[0052] S103 Determine one or more services to be recommended to the person according to the token sequence.

[0053] The methods of the various embodiments and / or alternative embodiments of the present invention are implemented through the intelligent cockpit system 100 integrated in the intelligent transportation device. The intelligent cockpit system 100 can be oriented to in-vehicle computer devices deployed in intelligent transportation devices. Among them, the in-vehicle computer devices include but are not limited to: in-vehicle embedded controllers / domain controllers, autonomous driving computing platforms, system-on-chip (SoC) integrated in the intelligent cockpit, in-vehicle infotainment systems with real-time processing capabilities, etc. Among them, the in-vehicle computer devices are integrated with relevant software and hardware environments, including but not limited to CAN (Controller Area Network) / LIN (Local Interconnect Network) bus interfaces, graphics processing units (GPUs), and in-vehicle operating systems, etc., and can support real-time data interaction with various vehicle-mounted sensors such as OMS / DMS. Here, the in-vehicle computer device is only an example, and other existing or future possible devices and / or resource platforms applicable to the present invention should also be included in the protection scope of the present invention, and are hereby included by reference.

[0054] In intelligent transportation devices, systems based on various sensors are usually deployed, such as OMS / DMS, Beidou / GPS, ADAS, ARS, HMI, etc. Through these systems, the internal and external environment information, status information and / or relevant information of the person in the cockpit of the intelligent transportation device can be collected.

[0055] In this embodiment, in step S101, the status information of the person in the intelligent cockpit can be obtained, and based on the status information of the person, the user profile corresponding to the person can be obtained from the local database.

[0056] Among them, the intelligent cockpit system 100 may include a local database. According to the historical data of the driver and / or passengers, relevant user portraits of the relevant personnel can be pre-entered into the local database (or called the edge-side database). Among them, the user portrait may include the basic information of the user, behavioral characteristic data, preference characteristic data, etc. Among them, the relevant data of the user portrait can be stored in a semi-structured manner with a JSON document as the carrier. The content of the JSON document may include: user behavior tags, user role tags, user setting preference tags, etc. The user portrait may include information directly entered manually. For example, basic information (name, common residence address, relationship with the vehicle owner, preference types of music, video, etc., commuting time, location, etc.) can be actively entered through the in-vehicle interface of the intelligent cockpit system 100. It may also include indirectly obtained information. For example, according to the recorded vehicle control setting preferences of the personnel in the cockpit (such as the temperature and humidity settings in the cockpit, common navigation locations, audio and video playback history, etc.) or information obtained from third-party applications frequently used integrated in the intelligent cockpit system 100 (such as the frequently visited locations in applications such as Meituan and Dianping, the playback history or preference settings in music playback software, the playback history or preference settings in video playback software), etc.

[0057] After the intelligent transportation device is started, the intelligent cockpit system 100 can obtain the status information of the personnel in the intelligent cockpit, and according to the obtained status information of the relevant personnel, obtain the corresponding user portrait from the local database. Among them, according to the identity identifier in the obtained status information of the relevant personnel, the user portrait corresponding to the identity identifier can be obtained from the local database. Among them, the identity identifier can be determined based on feature recognition such as face, iris, fingerprint or voiceprint that can uniquely identify the identity of the personnel in the intelligent cockpit. The personnel in the intelligent cockpit may include the driver (if it is a driverless intelligent transportation device, there is no driver) and / or passengers. In addition to the basic information such as the identity of the driver, the status information of the driver usually may include abnormal status and normal status. Among them, when the intelligent transportation device is moving, if the driver has situations such as fatigue, answering or making calls, smoking, drinking water, playing with mobile phones, chatting, etc., or the intelligent transportation device is in a traffic accident, it can be considered an abnormal status. In the normal state, the expression information, clothing information, etc. of the driver can also be further obtained as the status information of the driver. In addition to the basic information such as the identity of the passengers, the status information of the passengers may include the expression information of the passengers (such as happy, angry, melancholy, sad, calm, anxious, etc.), clothing information (such as suits, skirts, sportswear, cotton-padded clothes / down jackets / coats, short sleeves, etc.), etc. Further, if there is more than one person in the intelligent cockpit, the relationship between the personnel in the intelligent cockpit can also be determined according to the obtained status information of the personnel in the intelligent cockpit and / or the local database (such as family members, friends, lovers, etc.).

[0058] Optionally, the obtaining of the status information of the person in the intelligent cockpit includes:

[0059] Obtain the status information of the person in the intelligent cockpit through OMS and / or DMS.

[0060] Among them, through OMS and / or DMS, the real-time video and audio of the person in the intelligent cockpit can be collected. Based on computer vision algorithms (such as YOLO, U-Net series, etc.), the identity, expression, clothing, posture, etc. of the person in the intelligent cockpit can be recognized to confirm the status information of the person in the intelligent cockpit. For example, existing OMS products such as Bosch's occupant monitoring system and Continental's interior camera system, and existing DMS products such as Mobileye EyeQ series and NVIDIA Drive IX are applicable.

[0061] Continuing in this embodiment, in step S102, feature engineering and tokenization encoding can be performed on the status information of the person and the user portrait corresponding to the person to obtain a token sequence.

[0062] Among them, after obtaining the status information of the person in the intelligent cockpit and its corresponding user portrait, for each person in the intelligent cockpit, feature engineering and tokenization encoding can be performed on its status information and user portrait according to preset rules. Among them, performing feature engineering on the status information and user portrait of each person in the intelligent cockpit according to preset rules can convert multi-modal data such as status information and user portrait into structured feature data that can better express the essence of the problem. Then, the obtained feature data is input into a pre-trained tokenizer for tokenization encoding to obtain a token sequence corresponding to each person in the intelligent cockpit that is more suitable for the large model. This can not only reduce the data volume, improve the calculation efficiency, but also facilitate capturing the hidden semantic information, improve the recommendation accuracy, and enhance the user experience.

[0063] Optionally, step S101 further includes:

[0064] Obtain the environmental information of the intelligent cockpit;

[0065] Among them, step S102 includes:

[0066] Perform feature engineering and tokenization encoding on the environmental information of the intelligent cockpit, the status information of the person, and the user portrait to obtain a token sequence.

[0067] Among them, in order to improve the recommendation accuracy and further enhance the user experience, the environmental information inside and outside the intelligent cockpit can also be taken into account. In step S101, the environmental information inside and outside the intelligent cockpit is also obtained. Among them, the environmental information may include: the position information of the intelligent transportation device obtained through the deployed positioning system, the weather information determined by the real-time video and audio collected by the camera deployed outside the cockpit of the intelligent transportation device (the weather information can also be jointly determined by integrating the cloud meteorological data obtained by the vehicle-mounted system connected to the network), the date and time information obtained by the vehicle-mounted system connected to the network, and the temperature, humidity, and air quality inside the cockpit (for example, it can be determined by the data collected in real time by the deployed PM2.5 / VOC (Volatile Organic Compounds) sensor), etc. The above are only examples, and other environmental information collected by the relevant sensors and / or systems deployed in the intelligent transportation device that is applicable to the present invention should also be included within the protection scope of the present invention, which is not limited herein.

[0068] When the environmental information of the intelligent cockpit is also obtained in step S101, in step S102, for each person inside the intelligent cockpit, feature engineering and tokenization encoding can be performed on the obtained environmental information of the intelligent cockpit, the status information of the person inside the intelligent cockpit, and the user portrait corresponding to the person inside the intelligent cockpit to obtain a token sequence corresponding to the person inside the intelligent cockpit.

[0069] Continuing in this embodiment, in step S103, one or more services recommended to the person can be determined according to the token sequence.

[0070] Among them, the intelligent cockpit system 100 can determine one or more services recommended to the person inside the intelligent cockpit according to the processed token sequence corresponding to the person inside the intelligent cockpit.

[0071] Optionally, among them, step S103 includes:

[0072] Input the token sequence into the function recommendation model to determine one or more services recommended to the person.

[0073] Among them, the obtained token sequence corresponding to each person inside the intelligent cockpit can be input into the corresponding function recommendation model to determine one or more services recommended to the person inside the intelligent cockpit. Among them, the function recommendation model should correspond to the rules adopted in the foregoing feature engineering and tokenization encoding.

[0074] Among them, the function recommendation model can be obtained by training a Natural Language Processing (NLP) model or a Large Language Model (LLM) with supervised fine-tuning (SFT) or parameter-efficient fine-tuning (PEFT) using a dataset constructed with historical data adapted to the intelligent cockpit system 100. Exemplarily, in step S102, an encoder can be used, such as an encoder using encoding methods like Byte Pair Encoding (BPE), word segmentation encoding (such as WorldPiece), character-based encoding (such as Unigram), sentence segmentation encoding (such as SentencePiece), etc., to perform feature engineering and tokenization encoding on the obtained status information of the personnel in the intelligent cockpit and their corresponding user portraits, and the obtained environmental information of the intelligent transportation device can also be included to obtain a token sequence. In step S103, the obtained token sequence is input into a function recommendation model obtained by training an NLP model (such as Wav2Vec, Bert, etc.) or an LLM model (such as Llama3.1, Qwen2, DeepSeek-V3, etc.) with SFT or PEFT. It is also possible to first integrate the encoder with the NLP or LLM model, perform SFT or PEFT training on the integrated NLP or LLM model to obtain a function recommendation model integrated with the encoder, and directly input data such as the status information of the personnel in the intelligent cockpit and the user portrait into the function recommendation model integrated with the encoder to determine one or more services recommended to the personnel in the intelligent cockpit.

[0075] Through the above embodiments and / or alternative embodiments, the status information of the personnel in the intelligent cockpit can be obtained, combined with the corresponding user portrait, perform feature engineering and tokenization encoding to obtain a token sequence, and based on the token sequence, determine one or more services recommended to the personnel in the intelligent cockpit. Generalization processing can be performed on the obtained data, and the determined recommended services are not limited to one, which can improve the user experience. Further, in order to improve the accuracy of the recommended services, the environmental information of the intelligent cockpit can also be obtained, and a function recommendation model obtained by training with SFT or PEFT based on an NLP or LLM model is used to determine the recommended services.

[0076] Optionally, the above-mentioned interaction method based on an intelligent cockpit further includes:

[0077] S104 Display the one or more services to the personnel through the human-machine interaction interface in the intelligent cockpit;

[0078] S105 Invoke the corresponding function interface according to the interaction operation information of the person to provide the service confirmed by the person.

[0079] After determining one or more services recommended to the person in the intelligent cockpit through step S103, in this alternative embodiment, in step S104, the one or more services can be presented to the person in the intelligent cockpit through the human-machine interaction interface in the intelligent cockpit.

[0080] Continuing in this embodiment, the person in the intelligent cockpit performs interaction operations according to the one or more services presented through the human-machine interaction interface, selects and confirms the service therefrom. In step S105, the intelligent cockpit system 100 (which can be through the above-mentioned function recommendation model) invokes the function interface corresponding to the service confirmed by the person in the intelligent cockpit to provide the service confirmed by the person in the intelligent cockpit. Exemplarily, the above-mentioned function recommendation model integrates several function interfaces (Function Call Interface, FCF, which can be a small model after fine-tuning training, such as a Decision Tree / Random Forest, or an edge-side optimized neural network such as TensorFlow Lite Micro, ONNX Runtime, or a language model after edge-side distillation quantization such as Qwen2.5 1.5B, Octo-planner), and can interact in the form of a command line. For example, if the service confirmed by the person in the intelligent cockpit is "play rock music", the following command line interaction can be used:

[0081] play_music(genre="rock");

[0082] If the service confirmed by the person in the intelligent cockpit is "adjust the air conditioner temperature", the following command line interaction can be used:

[0083] adjust_air_conditioning(temp=22).

[0084] It can also be interacted in JSON format. If the service confirmed by the person in the intelligent cockpit is "play rock music", the following JSON code can be used for interaction:

[0085] {

[0086] "command":"play_music",

[0087] "params":{

[0088] "genre":"rock"

[0089] }

[0090] }

[0091] By invoking the function interface corresponding to the service confirmed by the person in the intelligent cockpit, the service confirmed by the person in the intelligent cockpit is provided to the person in the intelligent cockpit.

[0092] Among them, the way for the person in the intelligent cockpit to perform interactive operations through the human-computer interaction interface can be through capacitive touch, a knob integrated with a biosensor, voice control, etc.

[0093] Based on the application scenario examples 1 - the daily commuting to work on a winter weekday of the above embodiments and / or alternative embodiments: When the car owner sits in the driver's seat and starts the car, and the DMS detects that the car owner is wearing a down jacket, the user portrait of the car owner can be obtained from the local database, the preferences of the car owner can be obtained, and based on the intelligent cockpit system, the services that can be recommended to the car owner are: 1. Adjust the temperature inside the cockpit; 2. Set the navigation route to the unit; 3. Steering wheel heating; 4. Play the songs of the singer the car owner likes. And according to the interactive operations of the car owner, the services confirmed by the car owner are provided to the car owner. Application scenario example 2 - weekend family outing: When the car owner sits in the driver's seat and starts the car, the DMS detects the car owner, the OMS detects the child, and a work meeting invitation email is received from a third-party application. The user portrait of the car owner and the user portrait of the child can be obtained from the local database, the preferences of the car owner can be obtained, and the preferences of the child can be obtained. Based on the intelligent cockpit system, the service that can be recommended to the child is: play related types of cartoons. The services that can be recommended to the car owner are: 1. Start the massage function of the driver's seat; 2. Create a meeting schedule. And according to the interactive operations of the child and / or the car owner, the services confirmed by them are provided to them.

[0094] Optionally, the interactive method based on the intelligent cockpit further includes:

[0095] Update the user portrait of the person in the local database according to the service confirmed by the person.

[0096] Among them, after the person in the intelligent cockpit confirms the service through interactive operations, the intelligent cockpit system 100 can also update the user portrait of the person in the local database according to the service confirmed by the person in the intelligent cockpit, so that the user portrait of the person stored in the local database is the latest and most consistent with the recent behaviors and preferences of the person in the intelligent cockpit. Based on data security, the user portrait can be stored in the local database after being encrypted.

[0097] Figure 2 An interactive device based on an intelligent cockpit according to another aspect of the present invention is shown, wherein the device of one embodiment includes:

[0098] The first module 210 is configured to obtain the status information of the person in the intelligent cockpit, and based on the status information of the person, obtain the user profile corresponding to the person from the local database;

[0099] The second module 220 is configured to perform feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence;

[0100] The third module 230 is configured to determine one or more services to be recommended to the person according to the token sequence.

[0101] The device of this embodiment is integrated in the aforementioned intelligent cockpit system 100.

[0102] After the intelligent transportation device is started, in this embodiment, through the first module 210 of the device, the status information of the person in the intelligent cockpit is obtained, and based on the obtained status information of the relevant person, the corresponding user profile is obtained from the local database.

[0103] Continuing in this embodiment, after the status information of the person in the intelligent cockpit and the corresponding user profile are obtained through the first module 210, through the second module 220 of the device, for each person in the intelligent cockpit, feature engineering and tokenization encoding are performed on their status information and user profile according to preset rules. Among them, performing feature engineering on the status information and user profile of each person in the intelligent cockpit according to preset rules can convert multi-modal data such as status information and user profile into structured feature data that can better express the essence of the problem. Then, the obtained feature data is input into a pre-trained tokenizer for tokenization encoding to obtain a token sequence corresponding to each person in the intelligent cockpit that is more suitable for the large model. This can not only reduce the data volume, improve the calculation efficiency, but also facilitate capturing the hidden semantic information, improve the recommendation accuracy, and enhance the user experience.

[0104] Continuing in this embodiment, through the third module 230 of the device, the token sequence corresponding to the person in the intelligent cockpit obtained through the second module 220 can be processed to determine one or more services to be recommended to the person in the intelligent cockpit.

[0105] Optionally, wherein the first module 210 is further configured to:

[0106] Obtain the environmental information of the intelligent cockpit;

[0107] Wherein, the second module 220 is configured to: perform feature engineering and tokenization encoding on the environmental information of the intelligent cockpit, the status information of the person, and the user profile to obtain a token sequence.

[0108] The status information of the person in the intelligent cockpit and their corresponding user portrait are obtained through the first module 210. In order to improve the recommendation accuracy and further enhance the user experience, the environmental information inside and outside the intelligent cockpit can also be taken into consideration. In this alternative embodiment, the environmental information inside and outside the intelligent cockpit can also be obtained through the first module 210. Through the second module 220, for each person in the intelligent cockpit, feature engineering and tokenization encoding are performed on the obtained environmental information of the intelligent cockpit, the status information of the person in the intelligent cockpit, and the user portrait corresponding to the person in the intelligent cockpit, to obtain a token sequence corresponding to the person in the intelligent cockpit.

[0109] Optionally, the interactive device based on the intelligent cockpit further includes:

[0110] A fourth module 240, configured to display the one or more services to the person through the human-machine interaction interface in the intelligent cockpit;

[0111] A fifth module 250, configured to call a corresponding function interface according to the interaction operation information of the person, so as to provide the service confirmed by the person.

[0112] In this alternative embodiment, through the fourth module 240 of the device, the one or these services can be displayed to the person in the intelligent cockpit through the human-machine interaction interface in the intelligent cockpit.

[0113] The person in the intelligent cockpit performs interaction operations according to the one or more services displayed through the human-machine interaction interface, and selects and confirms the service therefrom. Continuing in this alternative embodiment, through the fifth module 240 of the device, the function interface corresponding to the service confirmed by the person in the intelligent cockpit can be called according to the service confirmed by the person in the intelligent cockpit, so as to provide the service confirmed by the person in the intelligent cockpit to the person in the intelligent cockpit.

[0114] Optionally, the interactive device based on the intelligent cockpit further includes:

[0115] A sixth module, configured to update the user portrait of the person in the local database according to the service confirmed by the person.

[0116] After the person in the intelligent cockpit confirms the service through the interaction operation, in this alternative embodiment, through the sixth module of the device, the user portrait of the person in the intelligent cockpit in the local database can also be updated according to the service confirmed by the person in the intelligent cockpit, so that the user portrait of the person in the intelligent cockpit stored in the local database is the latest and most consistent with the recent behaviors and preferences of the person in the intelligent cockpit.

[0117] In each of the embodiments and / or alternative embodiments of the above device, the parts not mentioned in the method steps executed by each module are the same as those in the foregoing relevant method embodiments and / or alternative embodiments, and will not be elaborated herein.

[0118] Figure 3 There is shown an interaction system based on an intelligent cockpit according to another aspect of the present invention. In one embodiment, the system includes:

[0119] An input layer, configured to collect video images in the intelligent cockpit and determine status information of the people in the intelligent cockpit based on the video images;

[0120] A processing layer, configured to obtain the status information of the people in the intelligent cockpit, obtain a user profile corresponding to the people from a local database based on the status information of the people, perform feature engineering and tokenization encoding on the status information of the people and the user profile corresponding to the people to obtain a token sequence, determine one or more services recommended to the people based on the token sequence, and call a corresponding function interface of the output layer according to the interaction operation information of the people to provide the services confirmed by the people;

[0121] An output layer, configured to display the one or more services through a man-machine interaction interface of the intelligent cockpit and receive an instruction from an execution layer through a corresponding function interface to provide the services confirmed by the people.

[0122] In this embodiment, through the input layer of the system, video images in the intelligent cockpit can be collected, and based on the video images, status information of the people in the intelligent cockpit can be determined.

[0123] Continuing in this embodiment, through the processing layer of the system, the status information of the people in the intelligent cockpit can be obtained, a user profile corresponding to the people can be obtained from a local database based on the status information of the people, feature engineering and tokenization encoding can be performed on the status information of the people and the user profile corresponding to the people to obtain a token sequence, one or more services recommended to the people can be determined based on the token sequence, and a corresponding function interface of the output layer can be called according to the interaction operation information of the people to provide the services confirmed by the people.

[0124] Continuing in this embodiment, through the output layer of the system, the one or more services can be displayed through the man-machine interaction interface of the intelligent cockpit, and an instruction from the execution layer can be received through a corresponding function interface to provide the services confirmed by the people.

[0125] In an alternative embodiment, the input layer of the system is further configured to collect environmental data inside and outside the intelligent cockpit, and determine the environmental information of the intelligent cockpit based on the collected environmental data. Among them, the processing layer of the system performs feature engineering and tokenization encoding on the status information of the person, the user profile corresponding to the person, and the environmental information to obtain a token sequence.

[0126] In an alternative embodiment, the processing layer of the system also sends a service confirmed by the person in the intelligent cockpit to the local database to update the user profile of the person in the intelligent cockpit in the local database.

[0127] In the above system embodiments and / or alternative embodiments, the parts not mentioned in the method steps executed by each layer are the same as the foregoing related method embodiments and / or alternative embodiments, and will not be elaborated here.

[0128] According to another aspect of the present invention, there is also provided a computer-readable medium storing computer-readable instructions that can be executed by a processor to implement part or all of the foregoing methods.

[0129] It should be noted that each method embodiment and / or alternative embodiment of the present invention can be partially or fully implemented in software and / or a combination of software and hardware. The software programs involved in the present invention can be executed by a processor to implement part or all of the steps or functions of the foregoing embodiments and / or alternative embodiments. Similarly, the software programs of the present invention (including related data structures) can be stored in a computer-readable recording medium.

[0130] In addition, a part of the present invention can be applied as a computer program product. For example, computer program instructions, when executed by a computer, can call or provide part or all of the methods and / or technical solutions according to the present invention through the operation of the computer. The program instructions for calling the methods of the present invention may be stored in a fixed or removable recording medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium, and / or stored in the working memory of a computer device running according to the program instructions.

[0131] According to still another aspect of the present invention, there is also provided an interactive device based on an intelligent cockpit, where the device includes: a memory storing computer program instructions and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the device is triggered to execute part or all of the methods and / or technical solutions of the foregoing embodiments and / or alternative embodiments.

[0132] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-mentioned exemplary embodiments and / or alternative embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved. In addition, it is obvious that the term "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices stated in the apparatus claims can also be implemented by one unit or device through software and / or hardware. The terms such as first and second are used to denote names and do not denote any particular order.

Claims

1. An interaction method based on an intelligent cockpit, characterized in that The method includes: Obtaining the status information of the person in the intelligent cockpit, and obtaining the user profile corresponding to the person from the local database according to the status information of the person; Performing feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence; Determining one or more services to be recommended to the person according to the token sequence.

2. The method according to claim 1, wherein The obtaining of the status information of the person in the intelligent cockpit includes: Obtaining the status information of the person in the intelligent cockpit through OMS and / or DMS.

3. The method according to claim 1, wherein The obtaining of the status information of the person in the intelligent cockpit, and obtaining the user profile corresponding to the person from the local database according to the status information of the person, further includes: Obtaining the environmental information of the intelligent cockpit; Wherein, the performing of feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person includes: performing feature engineering and tokenization encoding on the environmental information of the intelligent cockpit, the status information of the person, and the user profile to obtain a token sequence.

4. The method according to claim 1, wherein The determining of one or more services to be recommended to the person according to the token sequence includes: Inputting the token sequence into a function recommendation model to determine one or more services to be recommended to the person.

5. The method according to claim 1, characterized in that, The method further includes: Displaying the one or more services to the person through the human-machine interaction interface in the intelligent cockpit; Invoking the corresponding function interface according to the interaction operation information of the person to provide the service confirmed by the person.

6. The method according to claim 5, wherein The method further includes: Updating the user profile of the person in the local database according to the service confirmed by the person.

7. An interaction device based on an intelligent cockpit, characterized in that, The device includes: A first module, configured to obtain the status information of the person in the intelligent cockpit, and obtain the user profile corresponding to the person from the local database according to the status information of the person; A second module, configured to perform feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence; A third module, configured to determine one or more services to be recommended to the person according to the token sequence.

8. The device according to claim 7, characterized in that The first module is further configured to: Obtain the environmental information of the intelligent cockpit; Wherein, the second module is configured to: perform feature engineering and tokenization encoding on the environmental information of the intelligent cockpit, the status information of the person, and the user profile to obtain a token sequence.

9. The device according to claim 7, characterized in that, The device further includes: A fourth module, configured to display the one or more services to the person through the human-machine interaction interface in the intelligent cockpit; A fifth module, configured to invoke the corresponding function interface according to the interaction operation information of the person to provide the service confirmed by the person.

10. An interaction system based on an intelligent cockpit, characterized in that, The system includes: An input layer, configured to collect video images in the intelligent cockpit, and determine the status information of the person in the intelligent cockpit according to the video images; A processing layer, configured to obtain the status information of the person in the intelligent cockpit, and according to the status information of the person, obtain the user profile corresponding to the person from the local database, and perform feature engineering and tokenization encoding on the status information of the person and the user profile corresponding to the person to obtain a token sequence, determine one or more services recommended for the person according to the token sequence, and call the corresponding function interface of the output layer according to the interaction operation information of the person to provide the service confirmed by the person; An output layer, configured to display the one or more services through the human-machine interaction interface of the intelligent cockpit, and receive instructions from the execution layer through the corresponding function interface to provide the service confirmed by the person.

11. A computer-readable medium, characterized in that, computer-readable instructions are stored thereon, and the computer-readable instructions are executed by a processor to implement part or all of the method according to any one of claims 1 to 6.

12. An interaction device based on an intelligent cockpit, characterized in that, The device includes: one or more processors; and a memory storing computer-readable instructions, the computer-readable instructions, when executed, cause the processor to perform part or all of the operations of the method according to any one of claims 1 to 6.

Images