Cabin interaction system, automobile and control method of cabin interaction system
By collecting multi-modal situational perception information in the interactive terminal device in the car cockpit and using the scene drama generation model to generate personalized scene drama content, the problem of single cabin entertainment system is solved, and the user experience is enhanced with the fun and sense of participation. , realizes immersive cockpit interactive service.
Patent Information
- Application Number
- CN202411167465.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-03
AI Technical Summary
The traditional car cockpit entertainment system cannot meet the drivers and passengers' needs for personalized and interactive entertainment, resulting in a single interaction between the cockpit scene and a lack of attractive and highly engaged entertainment solutions.
A cockpit interaction system is designed to collect the situational perception information of the target object through the interactive terminal device, and use the trained sitcom generation model to generate personalized sitcom content based on multimodal information, including voice, action, face and Physiological signal information is adaptively adjusted in combination with environmental perception information, providing immersive interactive services.
It realizes immersive interactive service in the smart cockpit, enhances the fun and sense of participation of the user's ride experience, solves the problem of single interaction between the cockpit scenes, and provides a personalized and customized entertainment experience.
Smart Images

Figure CN119068465A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automobile technology, and in particular to a cockpit interaction system, an automobile, and a control method for the cockpit interaction system. Background Art
[0002] Traditional car cockpit entertainment systems, such as audio players, car TVs, and game consoles, provide fixed and single entertainment content, which cannot meet the growing demand of drivers and passengers for personalized and interactive entertainment. Especially during long-distance travel or waiting, drivers and passengers often seek more attractive forms of entertainment to relieve travel fatigue and increase travel fun. Although there are entertainment methods such as in-car movies and audiobooks on the market, there is a lack of innovative entertainment solutions with high user participation and strong experience, resulting in a single scene interaction in the car cockpit.
[0003] Currently, no effective solution has been proposed for the problem of single scene interaction in car cockpits in related technologies. Summary of the invention
[0004] The embodiments of the present application provide a cockpit interaction system, a car, and a method for controlling the cockpit interaction system, so as to at least solve the problem of single scene interaction in the car cockpit in the related art.
[0005] In a first aspect, an embodiment of the present application provides a cockpit interaction system, the system comprising: an interaction terminal device and a processing device;
[0006] The interactive terminal device is used to collect contextual awareness information of the target object;
[0007] The processing device is connected to the interactive terminal device, and is used to input the situational awareness information into the trained target situational drama generation model for recognition processing, and output the situational drama content.
[0008] In some of the embodiments, the context awareness information includes voice information, motion information, face information and physiological signals.
[0009] In some embodiments, the system further comprises an environment sensing device; wherein the environment sensing device is connected to the processing device;
[0010] The environment sensing device is used to collect the environment information of the cockpit;
[0011] The processing device is also used to generate situational analysis information based on the situational perception information and the environmental information, and input the situational analysis information into the target situational drama generation model to output the situational drama content.
[0012] In some of the embodiments, the processing device is further used to obtain a user portrait of the target object and adjust the situational drama content according to the user portrait.
[0013] In some of the embodiments, the interactive terminal device is further used to collect interaction information between the target object and the constructed virtual character;
[0014] The processing device is further configured to adjust the situational drama content according to the interaction information.
[0015] In some of the embodiments, the processing device is further configured to determine a plot branch according to the interactive information, and adjust the situational drama content based on the plot branch.
[0016] In some of the embodiments, the interactive terminal device is further used to collect feedback information of the target object;
[0017] The processing device is also used to optimize the target scenario generation model according to the feedback information to obtain a new scenario generation model.
[0018] In some of the embodiments, the processing device is further configured to obtain user login information of the target object, and configure user permissions for the target object according to the user login information.
[0019] In a second aspect, an embodiment of the present application provides a car, including a cockpit and a cockpit interaction system as described in the first aspect above.
[0020] In a third aspect, an embodiment of the present application provides a method for controlling a cockpit interaction system, wherein the cockpit interaction system includes an interaction terminal device and a processing device; the method includes:
[0021] The interactive terminal device collects contextual awareness information of the target object;
[0022] The processing device inputs the situational awareness information into the trained target situational drama generation model for recognition processing, and outputs the situational drama content.
[0023] Compared with the related art, the embodiment of the present application provides a cockpit interaction system, a car and a control method of the cockpit interaction system, wherein the cockpit interaction system includes: an interaction terminal device and a processing device; the interaction terminal device is used to collect the context perception information of the target object; the processing device is connected to the interaction terminal device, and is used to input the context perception information into the trained target scenario generation model for recognition and processing, and output the scenario content. Through the above method, immersive interactive services in the smart cockpit are realized, and the services and content in the cockpit can be adaptively adjusted according to the multi-modal context perception information, which effectively solves the problem of single scenario interaction in the car cockpit and enhances the fun and participation of the user's riding experience.
[0024] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0026] Figure 1 is a hardware structure block diagram of a terminal of a cockpit interaction system control method according to an embodiment of the present application;
[0027] Figure 2 is a structural block diagram of a cockpit interaction system according to an embodiment of the present application;
[0028] Figure 3 It is a flowchart of a control method of a cockpit interaction system according to an embodiment of the present application. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.
[0030] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.
[0031] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.
[0032] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 1 is a hardware structure block diagram of a terminal of a control method of a cockpit interaction device according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0033] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the control method of the cockpit interaction system in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0034] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.
[0035] This embodiment provides a cockpit interaction system. Figure 2 is a structural block diagram of a cockpit interaction system according to an embodiment of the present application, such as Figure 2 As shown, the system includes an interactive terminal device 22 and a processing device 24.
[0036] The above-mentioned interactive terminal device is used to collect the contextual perception information of the target object. The interactive terminal device may be a hardware device such as an on-board terminal, a mobile terminal or a display device deployed in the car cabin, which is deployed with a software program for interacting with the target object. The interactive terminal device may collect the contextual perception information of the target object riding in the car through its integrated multimodal sensor. The contextual perception information may include but is not limited to various voice information, action information, face information and physiological signals.
[0037] For example, the current situational awareness information to be collected is the voice information of the target object; the interactive terminal device obtains the voice information collected by the microphone or other sound receiving devices set therein. For another example, the situational awareness information includes action information; the interactive terminal device can collect the actions of the target object through a camera device, such as further understanding the driver's state through changes in posture and action, or identifying various interactive actions such as touch and gesture between the target object and the display screen, and then obtain the action information. For another example, the situational awareness information includes facial information; the interactive terminal device collects the facial information of the target object through a camera device, and performs facial expression recognition on the facial information. Among them, the facial expression recognition method can be to use computer vision and machine learning technology to analyze facial expressions by identifying facial features (such as eyes, mouth, eyebrows, etc.) and movements (such as blinking, smiling, frowning, etc.) to determine the emotional state of the person. In addition, the interactive terminal device can also combine the language information and face information of the target object to comprehensively identify the target object's emotions, that is, through the neural network model, the facial image information and voice information are respectively recognized and classified. For example, if the interactive terminal detects that the facial expression in the current facial image information is frowning based on deep learning technology, and recognizes that the emotion corresponding to the voice information is anxious, then a comprehensive analysis can be performed to obtain the target emotion recognition result that the target object's emotional state is anxious. For example, the situational perception information includes physiological signals, and the interactive terminal device collects signals such as the target object's heart rate changes or pulse through a sports bracelet, etc., so as to monitor the target object's physical state in real time.
[0038] The above-mentioned processing device is connected to the interactive terminal device, and is used to input the situational awareness information into the trained target situational drama generation model for recognition processing, and output the situational drama content. Among them, the situational drama content refers to a drama form that simulates various scenes, situations or specific environments in real life, with character dialogue and performance as the main form, to show the plot, character personality, social phenomenon or a certain theme. It should be supplemented that the target situational drama generation model can be a large language model implemented based on the Transformer architecture. In this embodiment, situational data for training can be collected from a variety of sources, including user behavior, physiological signals or environmental conditions, etc., and they are cleaned and labeled to collect training data sets for training models. Then the model is trained using the preprocessed and labeled data set, and the loss function and performance on the validation set during the training process are monitored to adjust the hyperparameters and prevent overfitting. The performance of the model is evaluated on the test set and compared with the baseline model; the model is iteratively optimized according to the evaluation results. Finally, the trained model is deployed to the actual scene to generate new situational drama content.
[0039] Specifically, the above-mentioned processing device preprocesses the context-aware information. It should be supplemented that, in the present embodiment, the context-aware algorithm can also be used to process the above-mentioned context-aware information. For example, supervised learning methods such as Support Vector Machine (SVM) and random forest can be used to classify contexts; for another example, unsupervised learning methods can be used to find the similarities between different contexts to achieve cluster analysis; for another example, convolutional neural networks (CNN) can be used for image recognition, and deep learning models such as recurrent neural networks (RNN) can be used for time series analysis.
[0040] Next, the processed information is input into the target scenario generation model for recognition processing, so that personalized scenario content that conforms to the current situation can be automatically generated based on the situational awareness information, including but not limited to stories, games, educational courses, etc., to enhance the fun and participation of the riding experience. For example, the processing device inputs the integrated situational awareness information into the target scenario generation model, converts the voice information into text information using the speech-to-text (STT) interface set by the model, and extracts the text information features using the target scenario generation model to adaptively generate the corresponding scenario content. Alternatively, the processing device integrates the facial information to identify the emotions of the target object, and uses the model to adjust the required scenario content in combination with the identified target object emotions. Alternatively, the processing device can also control the smart cockpit to actively adapt to passenger needs and environmental changes based on the above-mentioned integrated situational understanding data, such as automatically adjusting the cabin atmosphere lighting effect, playing relaxing music, or starting decompression guidance when detecting passenger tension or fatigue, etc. In addition, in the above content generation process, predefined dialogue templates can also be used to fill dynamic content; or the most matching response can be retrieved from a pre-written dialogue database.
[0041] The above-mentioned cockpit interaction system collects the situational perception information of the target object through the interactive terminal device, and the processing device inputs the situational perception information into the target situational drama generation model to output the situational drama content, thereby realizing immersive interactive services in the smart cockpit. It can adaptively adjust the services and content in the cockpit according to the multimodal situational perception information, effectively solving the problem of single situational interaction in the car cockpit and enhancing the fun and sense of participation of the user's riding experience.
[0042] In some of the embodiments, the cockpit interaction system further includes an environment sensing device, wherein the environment sensing device is connected to the processing device.
[0043] The above-mentioned environmental sensing device is used to collect environmental information of the cabin. The environmental information refers to information that characterizes environmental changes such as weather, lighting, and traffic conditions inside and outside the cabin. Specifically, the environmental sensing device may include a temperature sensor such as a thermistor, and collect the real-time temperature of the cabin through the temperature sensor. Alternatively, the environmental sensing device may include a light sensor to collect lighting information in the cabin; the processing device may automatically adjust the lighting system according to the light intensity in the cabin to provide a comfortable visual environment and immersive services that match the situation. Alternatively, the environmental sensing device may include a camera device for collecting road conditions, etc.
[0044] The processing device is also used to generate situational analysis information based on situational perception information and environmental information, and input the situational analysis information into the target situational drama generation model to output situational drama content. After receiving the multimodal sensor data, the processing device performs complex algorithm processing, including image recognition, data analysis, machine learning, etc., to generate situational analysis information. Such information may include traffic congestion, weather conditions, driver behavior, passenger needs, etc. Next, the processing device inputs the situational analysis information into the target situational drama generation model, which can generate or recommend situational drama content that conforms to the current situation based on the input information. For example, if it is detected that the vehicle is driving on a highway and the traffic conditions are good, the system may recommend some relaxing and enjoyable entertainment content; and if it is detected that the vehicle is driving on a congested urban road, the system may provide situational drama content about improving urban traffic or relieving pressure. Alternatively, the processing device adjusts the scene setting and atmosphere creation of the situational drama in combination with the current driving state of the vehicle and external environmental information (such as time, weather, road conditions, etc.); for example, when driving at night, the system can recommend some situational dramas with night atmosphere or warm emotions. Ultimately, these situational drama contents can be presented to drivers and passengers through in-vehicle displays, audio systems and other devices, enhancing their driving and riding experience.
[0045] Through the above-mentioned embodiments, context analysis information that is more in line with the current context is generated in combination with environmental information, so that the content of the situational drama finally generated can be highly consistent with the current context, thereby effectively improving the immersion level of the generated situational drama.
[0046] In some of the embodiments, the processing device is also used to obtain a user portrait of the target object, and use the target scenario generation model to adjust the scenario content according to the user portrait. Among them, the user portrait is a comprehensive description of user characteristics, behaviors, preferences, needs and other information. The processing device can obtain the user portrait through a variety of channels: for example, when the vehicle is purchased or the system is used for the first time, basic information and preferences are collected through user registration information and questionnaires. For another example, the processing device analyzes the user's behavior data in the car, such as frequency of use, playback history, search history, interaction methods, etc., to infer the user's interests and habits. For another example, the user's facial expressions, sounds, gestures and other non-verbal behavior data can be collected through in-car sensors (such as cameras, microphones) to further understand the user's emotions and reactions. Alternatively, the processing device can also cooperate with third parties such as social media, music platforms, shopping websites, etc. to integrate the user's behavior data on other platforms to form a more comprehensive user portrait.
[0047] After obtaining the user portrait, the processing device can combine it with the situational analysis information and adjust the situational drama content through the target situational drama generation model to meet the user's personalized needs; that is, based on the user's interests, preferences and historical behaviors, recommend situational drama content that suits their tastes. For example, for users who like science fiction themes, the system can give priority to recommending science fiction-style situational dramas. Alternatively, the processing device can customize the interaction method of the situational drama according to the user's interaction habits and needs. For example, provide voice interaction options for users who like voice control, and optimize the touch screen interface for users who like touch screen operation.
[0048] Through the above-mentioned embodiments, the processing device deeply obtains the user portrait of the target object and adjusts the situational drama content accordingly, thereby providing users with a more intelligent, personalized and customized situational drama content experience, effectively enriching the situational interaction function of the car cabin.
[0049] In some embodiments, the interactive terminal device is also used to collect interactive information between the target object and the constructed virtual character; wherein the interactive information may include the user's voice commands, gesture operations, eye gaze, expression changes, etc. The virtual character can be generated through deep learning model training, so that it can interact with passengers in a highly anthropomorphic way in natural language, expression, action, etc., simulate real social experience, respond to passengers' emotional changes, and provide emotional support and psychological comfort.
[0050] The following is a brief description of the modeling process of the virtual character. First, use 3D modeling software to create the appearance of the virtual character. In the selected 3D modeling software, use polygon modeling, non-uniform rational B-splines (NURBS) modeling or other technologies to create the basic shape of the virtual character, including the head, body, limbs and other parts; and the character should have rich facial expressions and realistic movements. Then, use bone binding and skinning technology to create animations, and use real-time rendering engines (such as Unity engine or Unreal Engine engine) to render the character to ensure the performance on the in-vehicle display.
[0051] The processing device is also used to use the target scenario generation model to adjust the scenario content according to the interaction information. Specifically, after receiving the interaction information, the processing device will use the target scenario generation model to dynamically adjust the scenario content. This adjustment can be based on a variety of strategies and algorithms to achieve a more immersive and personalized entertainment experience. The following are some possible adjustment methods: The processing device can present different plot branches according to the user's interactive selection. For example, the user can influence the decision-making of the virtual character or the direction of the story through voice or selection of options on the interface. Alternatively, the processing device adjusts the depth of interaction and emotional connection between the characters according to the degree of interaction between the user and the virtual character (such as dialogue turns, intimate actions, etc.) to increase the user's sense of participation and immersion. Alternatively, the processing device can ensure that the virtual character provides instant feedback on the user's interaction, such as through voice, action, expression, etc., to enhance the interactive experience between the user and the virtual character. In addition, the processing device supports multiple interaction methods (such as voice, gesture, eye tracking, etc.), and automatically selects the most appropriate interaction mode according to the user's interaction habits, thereby improving the efficiency and naturalness of the interaction between the user and the virtual character.
[0052] More specifically, the processing device is also used to determine the plot branches according to the interactive information, and adjust the content of the situational drama based on the plot branches. Among them, according to the parsed interactive information, the processing device will determine the current plot branch by comparing the preset plot branch logic or decision tree. The plot branch logic may be defined based on the user's choice (such as by selecting a menu item), the user's behavior (such as the way of interacting with the virtual character), or the user's emotional state (such as by analyzing facial expressions and voice intonation). For each determined plot branch, the processing device will apply the corresponding content adjustment strategy. These strategies may include changing the storyline, adjusting character dialogues, modifying scene settings, introducing new characters or props, etc. The purpose of the adjustment is to keep the situational drama content consistent with the user's interaction and enhance the user's sense of participation and immersion.
[0053] It should be noted that in the above-mentioned interactive information management process, a series of rules can be defined to guide the dialogue process to ensure that the generated situational drama content meets the feature standards. Exemplarily, the series of rules may include situational adaptability rules, such as when traffic congestion is detected, the virtual character can choose to narrate the plot branch classified as an easy path to relieve stress; if the weather is fine, the virtual character can also propose to go for a walk in a nearby park. Alternatively, the above rules may include conversational rules, that is, in a conversation, maintain the consistency of the topic, avoid sudden changes in the topic, or use contextual words (such as "then", "therefore", etc.) to connect sentences. Alternatively, the above rules may also include emotional regulation rules; if the user shows negative emotions, adjust the content of the conversation to appease the emotions, and encourage the user to share feelings, provide positive feedback and support. Alternatively, the above rules may also include personalized rules: recommend relevant content based on the user's interests and hobbies; if the user often listens to a certain type of music, mention related artists or songs in the conversation. Alternatively, the above rules may also include safety rules to avoid complex interactions during driving or require the driver to do additional operations.
[0054] Through the above embodiments, the processing device can dynamically generate and adjust the situational drama content according to the real-time interactive information of the user, providing the user with a highly personalized and immersive entertainment experience. This experience not only meets the user's entertainment needs, but also enhances the emotional connection and interactive fun between the user and the vehicle.
[0055] In some embodiments, the interactive terminal device is also used to collect feedback information from the target object. The feedback information can be direct feedback information submitted by collected passengers, or user interaction behavior data obtained through log recording, behavior tracking and data analysis technology.
[0056] The processing device is also used to optimize the target scenario generation model according to the feedback information to obtain a new scenario generation model. The processing device automatically optimizes the target scenario generation model and the interaction strategy according to the feedback information through a machine learning algorithm. Specifically, the processing device cleans, organizes and classifies the collected feedback information to ensure its accuracy and usability; for example, the rating data of user feedback is standardized, the text feedback is subjected to sentiment analysis and topic extraction, and the behavior data is converted into quantifiable indicators. Next, the processing device can optimize the target scenario generation model using a machine learning algorithm. The following is a brief description of the optimization process: The processing device extracts key features from the processed feedback information, which can reflect the user's preferences, needs and expectations. For example, the user's rating, stay time, interaction frequency, etc. can all be used as features. In the model optimization process, the extracted features are used as input to iteratively optimize the machine learning model. Finally, the model parameters, structure or algorithm are adjusted according to the evaluation results, and a new round of training and evaluation is carried out. This process is repeated until satisfactory performance is achieved. Therefore, through machine learning optimization, the processing device can automatically adjust the various parameters and logic of the target situational drama generation model to better meet the needs and expectations of users; the optimized situational drama generation model can generate more personalized, interesting and fascinating situational drama content. In addition to optimizing the situational drama generation model, the processing device also optimizes the interaction strategy with the user, including improving the user interface, optimizing the interaction process, and enhancing the feedback mechanism. By optimizing the interaction strategy, the user's participation and satisfaction can be improved, further enhancing the attractiveness of the situational drama content.
[0057] Through the above embodiments, a feedback loop mechanism is introduced into the cockpit interaction system, so that as time goes by, the smart cockpit will become more and more in line with the personalized needs of each passenger, realizing the long-term evolution and continuous improvement of services.
[0058] In some embodiments, the processing device is also used to obtain the user login information of the target object, and configure user permissions for the target object according to the user login information. Among them, the processing device first verifies the user's login information, including the user name and password; the password is usually hashed when stored to protect its security. After the login verification is successful, the processing device can configure the corresponding user permissions according to the user's identity. User permission configuration refers to assigning corresponding data access and sharing permissions to the user according to the user's identity and needs. The processing device provides user-configurable data access and sharing options, and the user can set which data can be accessed and which data can be shared with third parties according to their needs. Permission configuration can be refined to specific data items or data categories to provide more sophisticated privacy protection. In another embodiment, the processing device can also encrypt all user data before storing it; it includes user login information (such as user name, password hash value), user portrait data, interactive behavior data, direct feedback information, etc. Encryption can use strong encryption algorithms such as Advanced Encryption Standard (AES) and asymmetric encryption algorithm RSA to ensure that even if the data is illegally obtained during storage, it cannot be easily decrypted.
[0059] Through the above-mentioned embodiments, the processing device provides users with comprehensive data protection and privacy protection through measures such as authority management, thereby effectively improving the security of the cockpit interaction system and helping to build user trust in the system.
[0060] The following is an explanation based on actual application scenarios. One of the application scenarios of the cockpit interaction system can be family entertainment during long-distance self-driving tours. The scenario setting in this scenario is: a family of four plans to take a long-distance self-driving tour. During the journey, the parents hope to provide their children with entertainment activities that are both interesting and educational, and also hope to have some opportunities to relax and rest while driving. In the implementation process, the system captures the activities and expressions of the children in the back seat through the on-board camera and finds that they seem a little bored and anxious; at the same time, the system also notices that the weather is sunny and suitable for the story theme of outdoor adventure. The system's processing device generates a story about a brave little explorer based on a deep learning model. The story contains educational elements, such as geographical knowledge and environmental awareness. The story is played through the car audio, and the corresponding animation screen is displayed on the screen, which enhances the sense of immersion. During the interaction, the system allows the men to communicate with the characters in the story through voice commands, and even influence the direction of the story, such as choosing different adventure routes or solving problems encountered on the way. Parents can also join the game at the appropriate time, play the roles in the story, and enhance the interaction between family members. Through the above plan, the whole journey becomes more enjoyable and meaningful. The children not only have fun but also learn new knowledge, while the parents get precious rest time and reduce the fatigue of long-distance driving.
[0061] Another application scenario for the cockpit interactive system is to customize the experience for individuals during business trips. The scenario in this scenario is: a business person who travels frequently hopes to get some relaxation and inspiration through the cockpit system on the way to an important meeting in order to better prepare for the upcoming speech. During the implementation process, the system monitors the user's conversation about the upcoming meeting through the microphone and understands the user's needs through voice recognition technology. At the same time, the system also takes into account the current traffic conditions and the estimated time of arrival. The system generates a series of short stories related to the theme of the meeting, which combine industry insights, success stories and motivational statements to inspire users' creativity and self-confidence. In addition, the system automatically adjusts the mood lighting and music in the cockpit to create an environment conducive to thinking and relaxation. During the interaction, users can request specific types of stories or adjust the volume and lighting through voice commands, and the system can respond to these requests quickly and provide personalized services. Through the above solution, the user is not only fully mentally prepared before arriving at the meeting place, but also gains new ideas and perspectives, which makes his performance in the meeting better and wins the praise of colleagues and customers.
[0062] Another application scenario of the cockpit interaction system can be: solving the problem of boredom and sleepiness during long-distance driving. Suppose a driver feels bored and sleepy during a long-distance drive, he can use the above-mentioned cockpit interaction system. The system first learns that he likes science fiction themes and prefers adventure stories through the previously analyzed social network data and historical behavior records. After the system is started, it will generate a situational drama about interstellar exploration, and the driver and passengers can choose to play the role of a brave captain or a witty scientist. Through voice commands, the driver and passengers can talk to the generated situational drama virtual characters and even decide the direction of the plot, such as exploring unknown planets or avoiding the pursuit of alien creatures. In particular, this system can activate holographic projection and surround sound in autonomous driving mode to provide an immersive experience, while in ordinary manual driving mode, it only provides audio and simple touch screen interaction to ensure driving safety.
[0063] Based on the above cases, the wide applicability and flexibility of the smart cockpit scenario generation and interaction system are demonstrated. Whether it is home entertainment or business needs, it can provide customized experience and improve user satisfaction and overall journey quality.
[0064] This embodiment also provides a car, including a cockpit and a cockpit interaction system as described in any of the above embodiments.
[0065] This embodiment also provides a control method for the cockpit interaction system. Figure 3 is a flow chart of a control method of a cockpit interaction system according to an embodiment of the present application. Figure 3 As shown, the process includes the following steps:
[0066] Step S310: the interactive terminal device collects context awareness information of the target object.
[0067] In step S320, the processing device inputs the context perception information into the trained target situational drama generation model for recognition processing, and outputs the situational drama content.
[0068] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0069] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0070] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0071] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0072] S1, the interactive terminal device collects contextual awareness information of the target object.
[0073] S2, the processing device inputs the situational awareness information into the trained target situational drama generation model for recognition processing, and outputs the situational drama content.
[0074] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.
[0075] In addition, in combination with the control method of the cockpit interaction system in the above embodiments, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any of the control methods of the cockpit interaction system in the above embodiments is implemented.
[0076] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0077] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0078] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0079] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A cockpit interaction system, characterized in that: The system comprises: an interactive terminal device and a processing device; The interactive terminal device is used to collect contextual awareness information of the target object; The processing device is connected to the interactive terminal device, and is used to input the situational awareness information into the trained target situational drama generation model for recognition processing, and output the situational drama content.
2. The cockpit interaction system according to claim 1, characterized in that: The context awareness information includes voice information, action information, face information and physiological signals.
3. The cockpit interaction system according to claim 1, characterized in that: The system further comprises an environment sensing device; wherein the environment sensing device is connected to the processing device; The environment sensing device is used to collect the environment information of the cockpit; The processing device is also used to generate situational analysis information based on the situational perception information and the environmental information, and input the situational analysis information into the target situational drama generation model to output the situational drama content.
4. The cockpit interaction system according to claim 1, characterized in that: The processing device is also used to obtain a user portrait of the target object and adjust the situational drama content according to the user portrait.
5. The cockpit interaction system according to claim 1, characterized in that: The interactive terminal device is also used to collect interactive information between the target object and the constructed virtual character; The processing device is also used to adjust the situational drama content according to the interactive information.
6. The cockpit interaction system according to claim 5, characterized in that: The processing device is further configured to determine a plot branch according to the interactive information, and adjust the content of the situational drama based on the plot branch.
7. The cockpit interaction system according to claim 1, characterized in that: The interactive terminal device is also used to collect feedback information from the target object; The processing device is also used to optimize the target scenario generation model according to the feedback information to obtain a new scenario generation model.
8. The cockpit interaction system according to any one of claims 1 to 7, characterized in that: The processing device is also used to obtain user login information of the target object and configure user permissions for the target object according to the user login information.
9. A car, characterized in that: It comprises a cockpit and a cockpit interaction system as claimed in any one of claims 1 to 8.
10. A control method for a cockpit interaction system, characterized in that: The cockpit interaction system includes an interaction terminal device and a processing device; the method includes: The interactive terminal device collects contextual awareness information of the target object; The processing device inputs the situational awareness information into the trained target situational drama generation model for recognition processing, and outputs the situational drama content.
Citation Information
Cited By
Subscription type service method, system and equipment for emotion cabin function and medium
CN120439963A
