Interactive interface generation method and device, electronic equipment and vehicle

Abstract

The invention relates to an interactive interface generation method and device, electronic equipment and a vehicle, and relates to the technical field of data processing.The method comprises the steps that firstly, multi-modal feature data included in interactive input information of a user serves as input data to be input into a large language model; identifying language description information corresponding to the interactive input information through a large language model; analyzing according to the language description information to obtain interface description information of the interactive interface; and generating an interactive interface based on the interface description information. According to the technical scheme, the interaction interface needed by the user is accurately recognized through the interaction input of the user, the corresponding interaction interface is generated according to the interface description information of the interaction interface, manual searching operation of the user is reduced, direct interaction of the user is facilitated, and the interaction efficiency is improved. For example, the user can directly adjust and set the sensing function according to the sensing function setting information in the interactive interface, and create the sensing function required by the user.

Description

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 16, 2023, with application number 2023107206080 and application name “Method, device, electronic device and vehicle for generating interactive interface”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of data processing technology, and specifically to a method, device, electronic device and vehicle for generating an interactive interface. Background Art

[0003] With the popularization of intelligent terminals, users can surf the Internet or customize various application software through the terminals. In addition, users can realize many interactive functions through the terminals.

[0004] Currently, users can experience interactive functions through the interactive interface of the terminal. However, the layout of the interactive interface, functional content, and switching logic of different pages are all pre-defined fixed modes. In actual use, there may be situations where the functions are too hidden and require users to operate multiple times, which in turn affects the interaction efficiency and thus affects the user's usage efficiency and experience. Summary of the invention

[0005] In view of this, the present application provides a method, device, electronic device and vehicle for generating an interactive interface, the main purpose of which is to improve the technical problems of inconvenient interaction and low interaction efficiency in the above-mentioned prior art.

[0006] In a first aspect, the present application provides a method for generating an interactive interface, comprising:

[0007] Acquiring interactive input information of a user, wherein the interactive input information includes multimodal feature data;

[0008] Inputting the multimodal feature data as input data into a large language model, so as to identify language description information corresponding to the interactive input information through the large language model;

[0009] Analyze the language description information to obtain interface description information of the interactive interface;

[0010] An interactive interface is generated based on the interface description information.

[0011] In a second aspect, the present application provides a device for generating an interactive interface, comprising:

[0012] An acquisition unit, configured to acquire interactive input information of a user, wherein the interactive input information includes multimodal feature data;

[0013] an identification unit configured to input the multimodal feature data as input data into a large language model, so as to identify language description information corresponding to the interactive input information through the large language model; and to analyze the language description information to obtain interface description information of the interactive interface;

[0014] The generating unit is configured to generate an interactive interface based on the interface description information.

[0015] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for generating an interactive interface described in the first aspect is implemented.

[0016] In a fourth aspect, the present application provides an electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the method for generating an interactive interface as described in the first aspect when executing the computer program.

[0017] In a fifth aspect, the present application provides a vehicle, comprising the device as described in the second aspect, or the electronic device as described in the fourth aspect.

[0018] By means of the above technical scheme, the present application provides a method, device, electronic device and vehicle for generating an interactive interface, which first obtains the user's interactive input information, including multimodal feature data; then inputs the multimodal feature data as input data into a large language model, so as to identify the language description information corresponding to the interactive input information through the large language model; then analyzes the language description information to obtain the interface description information of the interactive interface, and then generates the interactive interface based on the interface description information. Compared with the current related technologies, the present application accurately identifies the interactive interface required by the user through the user's interactive input, such as simple voice input, and generates the corresponding interactive interface according to the interface description information of the interactive interface, which reduces the user's manual search operations, facilitates the user to directly experience the interactive functions, improves the interaction efficiency, and thus improves the user's usage efficiency and experience.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 A schematic diagram showing a flow chart of a method for generating an interactive interface provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram showing a flow chart of another method for generating an interactive interface provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram showing a flow chart of another method for generating an interactive interface provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram showing an example provided by an embodiment of the present application;

[0026] Figure 5 A schematic diagram showing an example provided by an embodiment of the present application;

[0027] Figure 6 A schematic diagram showing an example provided by an embodiment of the present application;

[0028] Figure 7 A schematic diagram showing an example provided by an embodiment of the present application;

[0029] Figure 8 A schematic diagram showing an example provided by an embodiment of the present application;

[0030] Fig. 9 A schematic diagram showing an example provided by an embodiment of the present application;

[0031] Fig.10 A schematic diagram showing an example provided by an embodiment of the present application;

[0032] Fig.11 A schematic diagram of the structure of a device for generating an interactive interface provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0034] In order to improve the technical problems of inconvenient interaction and low interaction efficiency in the current prior art. This embodiment provides a method for generating an interactive interface, such as Figure 1 As shown, the method includes:

[0035] Step 101: Obtain user interactive input information.

[0036] The user's interactive input information may include multiple methods. Specifically, the user may use one or a combination of voice, panel touch, line of sight, body movements, and gestures, such as voice combined with panel touch, voice combined with gesture recognition, etc.

[0037] The interactive input information includes multimodal feature data, which can be language, image, audio, video, text, gesture, vision, touch, biometrics, voiceprint, face ID, user sight, environmental information and other fusion data.

[0038] The execution subject of this embodiment may be a main control device used for interactive interface generation processing.

[0039] Step 102: Input the multimodal feature data included in the interactive input information as input data into the large language model, so as to identify language description information corresponding to the interactive input information through the large language model.

[0040] The language description information may be a domain specific language (DSL) description information. The DSL description information may be description information of an interactive information set corresponding to the multimodal feature data, and the interactive information set may include instruction description information (used to describe operation commands, such as vehicle control commands, etc.), user intention description (used to describe the user's intention, such as the need to listen to music, watch videos, set functions, etc.), and description information about the interface (such as page layout elements used to describe the interactive interface, and interactive elements in the page, etc.).

[0041] A large language model (LLM), i.e., a large-scale language model, can be a deep learning model trained using a large amount of sample data. In this embodiment, the large language model can be arranged on a client or a cloud, etc. If arranged on a cloud, the multimodal feature data included in the interactive input information can be sent to a cloud server, so that the cloud server can identify and analyze the multimodal feature data through the large language model to obtain language description information corresponding to the interactive input information.

[0042] For the training process of the large language model, a training set can be created in advance, which can include a large number of different sample interactive input information and corresponding language description information, such as instruction description information, user intention description and interface description information, etc. The large language model is trained using the data in these training sets. The large language model can be used to identify the user's current interactive input information and obtain corresponding language description information, such as corresponding instruction description information, user intention description and interface description information, etc.

[0043] Step 103: Analyze the language description information to obtain interface description information of the interactive interface.

[0044] This embodiment can further analyze the language description information obtained by the large language model to obtain interface description information of the interactive interface.

[0045] Interface description information refers to the descriptive content used to generate the corresponding interactive interface based on the analysis of user interactive input information such as voice and touch input, such as describing the page layout elements of the interactive interface and the interactive elements in the page. There are many optional ways to describe the language of the interface, such as describing it in HTML (HyperText Markup Language) for describing web page content.

[0046] In some embodiments, step 103 may specifically include: parsing the language description information to obtain the user's intention information; analyzing the user's intention information to obtain the interface description information of the interactive function that matches the user's intention information. For example, the interface description information of the interactive function that matches the user's intention information is obtained through the intention matching unit in the main control device. The interactive function may be a perception function, a music listening function, a video watching function, a navigation function, a takeaway ordering function, an instant messaging function, etc., for setting the cockpit system.

[0047] Exemplarily, analyzing the user intent information to obtain the interface description information of the interactive function matching the user intent information may specifically include: analyzing the user intent information to obtain the interface content layout information and interface dynamic effect information of the interactive function matching the user intent information. The interface content layout information may include the layout of the interface content, such as the display position of the elements, the correlation between the elements, and the interactive form of the interactive elements, etc. The interface dynamic effect information may include the animation of the interface to enhance the user experience.

[0048] This embodiment can analyze and obtain interface description information corresponding to the interactive function that matches the user intention information, such as the interface description of the interactive interface that matches the perception function setting, or the interface description of the interactive interface that matches the music listening function, etc., which may specifically include relevant description content of the interface content layout information and the interface dynamic effect information.

[0049] Step 104: Generate an interactive interface based on the interface description information.

[0050] In this embodiment, the interactive interface can pay more attention to the interaction with the user and the accuracy of the intention recognition. For example, the user can choose to experience the corresponding functional services (such as listening to music, watching videos, navigation, playing games, ordering takeout, etc.) in the interactive interface, or can set some functional options in the interactive interface, or can also communicate with other clients in real time in the interactive interface, etc.

[0051] The interactive interface of this embodiment can be a channel for information exchange between people and computer devices. Users can operate computer input information through the interactive interface, and the computer device provides information to users through the interactive interface with people for reading, analysis and judgment. For example, it can be: user interface, direct operation interactive interface, multimedia interactive interface, voice interactive interface, gesture interactive interface, mixed reality (MR) interface, multi-channel interactive interface (comprehensively adopts new interactive channels, devices and interactive technologies such as sight, voice, gesture, etc., so that users can use multiple channels to conduct human-computer dialogue in a natural, parallel and collaborative manner, and capture the user's interactive intention by integrating accurate and imprecise inputs from multiple channels, so as to improve the naturalness and efficiency of human-computer interaction), etc.

[0052] Based on the optional manner in step 103, in some embodiments, step 104 may specifically include: generating an interactive interface of the interactive function based on the interface description information of the interactive function. Exemplarily, generating an interactive interface of the interactive function based on the interface description information of the interactive function may specifically include: generating an interactive interface of the interactive function according to the interface content layout information and the interface dynamic effect information of the interactive function that match the user intention information.

[0053] For example, based on the interface description of the interactive interface for the listening function, an interactive interface for the listening function is generated; or based on the interface description of the interactive interface for the perception function setting, an interactive interface for the perception function setting is generated, and the user can set the perception function in the interactive interface, such as executing action B when condition A is met, etc.

[0054] In some embodiments, after step 104, the following may also be included: outputting an interactive interface of an interactive function (an interactive function that matches the user's intention); obtaining the user's setting result information for the interactive function in the interactive interface; and then updating the trained large language model based on the setting result information.

[0055] For example, the interactive interface of the interactive function is displayed on a display device (such as a display screen), and the user can adjust the displayed interactive interface, such as confirming whether it is the currently required interactive interface, or recording the user's interactive behavior in the interactive interface, etc. These data are collected and recorded, and then used as a basis to update the training set of the large language model, and the updated training set is used to update and train the large language model. In this way, the large language model is continuously updated, so that the large language model can obtain more accurate results that better meet user needs.

[0056] In actual use, the target interactive function set by the user may be non-compliant and / or unsafe. In order to avoid these problems, in some embodiments, after obtaining the setting result information of the interactive function by the user in the interactive interface, it may also include: performing a function check on the legality and / or security of the target interactive function corresponding to the setting result information; if the function check passes, the user is allowed to use the target interactive function. In this way, it can be ensured that the target interactive function set by the user will not be non-compliant and / or unsafe, thereby improving the success rate and security of the user's interactive function.

[0057] Exemplarily, the above-mentioned functional verification of the legality and / or safety of the target interactive function corresponding to the setting result information includes: performing a functional verification of the legality and / or safety of the target interactive function in combination with the vehicle's driving environment information and the user's function usage permission information.

[0058] For example, according to the user's setting results, the corresponding target interactive function is to turn on the vehicle's interior lights to listen to music and sing. Combined with the current vehicle driving environment information, the vehicle is currently at night and / or in a tunnel. If the interior lights are turned on, it is likely to affect the driver's safe driving. Therefore, it can be determined that the target interactive function has not passed the safety function verification. The user is not allowed to use this target interactive function to ensure driving safety.

[0059] Compared with the current related technologies, this embodiment can accurately identify the interactive interface required by the user through the user's interactive input, such as simple voice input, and generate a corresponding interactive interface according to the interface description information of the interactive interface, thereby reducing the user's manual search operations, facilitating the user to directly experience the interactive functions, improving the interaction efficiency, and further improving the user's usage efficiency and experience.

[0060] The following takes the perception function setting of the cockpit system as an example to illustrate the implementation process of the method of this embodiment. Figure 2 As shown, it may specifically include:

[0061] Step 201: Obtain user interaction input information to the cockpit system.

[0062] The interactive form of the cockpit system may include multimodal interaction. The cockpit system can collect data in the cockpit and then determine the user's interactive input information to the cockpit system based on the cockpit data. The cockpit data refers to the multimodal data collected from the car cockpit through sensors such as cameras, infrared sensors, fingerprint sensors, microphones, etc., as well as the data generated after processing.

[0063] The interactive input information may include multiple methods. Specifically, the user may use one or a combination of methods such as voice, panel touch, and gesture recognition, such as voice combined with panel touch, voice combined with gesture recognition, etc.

[0064] Step 202: Input the multimodal feature data included in the interactive input information as input data into the large language model, so as to recognize language description information corresponding to the interactive input information through the large language model.

[0065] Step 203: Analyze the language description information to obtain interface description information of the perception function setting interface.

[0066] The perception function can be an in-cabin sensing function: by detecting and tracking human visual features including head orientation, facial expressions, gaze direction, gestures and limb key points, the identity information and behavior of the driver and passengers are analyzed, while paying attention to the activities of people in the cockpit and the detection of related objects, and striving to provide a safer and smarter in-car experience.

[0067] The setting information of the perception function may include the setting information related to the triggering conditions of the perception function, and the setting information related to the execution action corresponding to the triggering conditions, etc. The triggering conditions include the following parts according to classification:

[0068] (1) Triggered by time, such as scheduled tasks, sunrise and sunset, etc.

[0069] (2) Triggered by environment, such as temperature, PM2.5, etc.

[0070] (3) Triggered by location, for example: in a certain geofence, out of a certain geofence;

[0071] (4) Triggered by navigation, for example: navigation to a certain place, estimated arrival time, etc.;

[0072] (5) Triggered by seat position, for example: driver, co-driver;

[0073] (6) Triggered by driving status, such as gear position, speed, mileage, and battery level;

[0074] (7) Triggered by door and window actions, such as locking the car and raising the window;

[0075] (8) Triggered by seat belt status, for example: not wearing a seat belt;

[0076] (9) Triggered by accessory status, such as charging panel status, child seat status, etc.

[0077] The execution actions corresponding to the trigger conditions can be classified into the following categories:

[0078] (1) Delayed action control;

[0079] (2) Air-conditioned seat posture, driver's door and window and other vehicle controls;

[0080] (3) Control of system capabilities such as Bluetooth, volume, screen, and open applications;

[0081] (4) Travel navigation;

[0082] (5) Voice broadcast, hearing.

[0083] This embodiment can obtain trigger condition setting information of the perception function and execution action setting information corresponding to the trigger condition by parsing the language description information obtained from the large language model. For example, the user's interactive input information to the cockpit system can be voice input information. The user can issue voice input information that meets the preset sentence structure, such as saying "execute XX action under XX conditions", and then instruct to generate an interactive interface for setting the perception function. Alternatively, there may be no restrictions on preset sentence structure, such as pre-training to obtain a large language model that can accurately recognize semantics. The user can accurately perform semantic analysis through the large language model through voice input containing elements such as trigger conditions and execution actions, and then instruct to generate an interactive interface for setting the perception function.

[0084] Step 204: Generate an interactive interface for setting the perception function based on the interface description information.

[0085] The interactive interface can display the trigger condition setting information required by the user, as well as the execution action setting information corresponding to the trigger condition. For example, if the "vehicle is started", "the left front door is closed", "all the locks are locked", and "the vehicle speed is greater than 0km", the voice message "report safe journey" will be executed. The user can further edit this information on the interactive interface, such as adding or deleting trigger conditions, adding or deleting execution actions, etc. In addition, the user can also edit the name of this perception function in the interactive interface, and set whether to ask before running, etc., to meet the different needs of users.

[0086] As the most information-rich interactive form, the interactive interface has its layout, functions, and switching logic of different pages pre-defined by the software in the current related technologies. The interactive forms of the software are also pre-defined, and the user completes the interaction in a unified path during use. Therefore, in the related technologies, if the user needs to set the perception function of the cockpit system, he needs to actively go to the car system to manually find the setting page of the perception function. In actual use, there may be situations where the function is too deeply hidden and requires the user to operate multiple times, which affects the efficiency of the cockpit system's perception function setting, and thus affects the efficiency and experience of the user's use of the cockpit system.

[0087] Compared with the current related technologies, this embodiment accurately identifies the interaction interface required by the user through the user's interactive input, such as simple voice input, and generates a corresponding interaction interface according to the interface description information of the interaction interface, thereby reducing the user's manual search operations, facilitating the user's direct interaction, and improving the interaction efficiency. For example, when generating an interactive interface for the perception function setting, the user can directly adjust the perception function according to the perception function setting information in the interaction interface, and create the perception function they need. In the whole process, the user's operations can be effectively reduced, and there is no need to search for the relevant setting pages layer by layer in the vehicle system. The perception function of the cockpit system can be quickly set, which can improve the efficiency of the perception function setting of the cockpit system, and thus improve the efficiency and experience of the user in using the cockpit system.

[0088] Further, as a refinement and extension of the above embodiment, in order to fully illustrate the specific implementation process of the method of this embodiment, this embodiment provides the following Figure 3 The specific method shown includes:

[0089] Step 301: Obtain user interaction input information to the cockpit system.

[0090] Step 302: Identify the natural language text information corresponding to the interactive input information, and analyze the natural language text information through a large language model in combination with the user's historical behavior data and car usage habit data to obtain language description information corresponding to the interactive input information.

[0091] Historical behavior data may include historical driving behavior data of users, interactive behavior data of vehicle functions, setting behavior data of perception functions, historical experience data of perception functions, etc. Car use habit data may include data on users' habitual vehicle operation at different times, locations, and driving behaviors, such as frequently turning on the radio to listen to the broadcast at 3 p.m. on Mondays; or frequently stopping for a while when passing by location A, and turning on the lighting device in the car.

[0092] The user's historical behavior data and car-using habit data may be the current user's historical behavior data and car-using habit data, or the historical behavior data and car-using habit data of different users (such as a large number of users of all car models, or different users of the same car model, etc.). The large language model is pre-trained with such data, that is, this embodiment can identify user interactive input information through the large language model, establish personalized tasks and execute them through natural language programming. The interactive input information can provide the large language model with data that can be recognized by language intelligence, including: speech, image, audio, text, gesture, touch, voiceprint, FaceID, environment and other fusion data.

[0093] Specifically, data collection and preprocessing can be carried out first: collecting user historical behavior data, car usage habit data and other information, and denoising, classifying and storing the data; then model training: training the model based on the collected data and the vehicle-side business capability map to obtain a large language model.

[0094] As the perception capability becomes stronger, the algorithm is also constantly developing. The generative capability demonstrated by the large language model has the basis for connecting perception and execution, providing help for creating a more flexible and convenient interaction method, and can improve the interaction efficiency of the smart cockpit. This embodiment uses the large language model's ability to understand and generate natural language to realize a new cockpit user interface and system, and implements it into a software application, greatly improving the efficiency and experience of users using the cockpit system. The software application can be installed on the vehicle side, and can perceive and understand the user's interaction intention with the help of the large language model, and convert the interaction intention into the trigger conditions and execution actions of the perception function, and provide the user with the interaction process and interaction results through the generative user interface (UI) form, and can intelligently manage the creation, deletion, modification, query, execution and other life cycle states of tasks through manual editing, automatic recommendation, multi-mode input, car cloud APP remote control, etc., and support immediate execution and delayed execution. Two strategies, when executing, tasks and conditions distribute control instructions to basic driving control vehicles, travel and other businesses through the task processing unit (Task Process Unit, TPU), and complete specific vehicle control and travel strategies.

[0095] Step 303: parse the language description information to obtain the user's intention information, and analyze based on the user's intention information to obtain interface description information of the interactive interface of the perception function setting that matches the intention information.

[0096] The interactive interface for setting the perception function includes the trigger condition interaction element of the perception function and the corresponding execution action interaction element.

[0097] In some embodiments, the result of the large language model recognition may be domain-specific language (DSL) description information. Accordingly, step 303 may specifically include: parsing the DSL description information to obtain the trigger condition setting information and corresponding execution action setting information required by the user.

[0098] like Figure 4 As shown, in this embodiment, the setting information of the perception function corresponding to the intent information can be obtained through the TPU. The TPU includes three modules: DSL engine, task engine, and Agent management. As a central management role, the TPU coordinates the task execution and information display on the vehicle unit (HU). Among them, the DSL engine: as the interactive protocol carrier with the large language model, it is the core of the interaction between the large language model and the HU. Parse, distribute, manage and generate the DSL information output by the large language model, and at the same time be responsible for returning the execution result of the task to the large language model. Task engine: responsible for the management of the task list, mainly generation, execution, result feedback, etc. The task engine receives tasks from the large language model and the HU, parses the tasks into task lists, schedules the execution of related tasks, and feeds back the execution results to the corresponding scheduler. Agent: a unified agent for all businesses to the outside world, receives the parsing results of the task engine to perform full-context intent matching, intent distribution, business arbitration and other operations, and uniformly manages the Plugin Proxy, that is, uniformly manages the capabilities of the system and vertical domains, and provides relevant capabilities to the large language model and HU in a standardized and atomic manner. Each vertical domain releases its capabilities by publishing a self-describing plugin to the Agent system. At the same time, the Agent system calls the capabilities of each vertical domain through a Uniform Resource Identifier (URI). Plugin Proxy: Plugin is responsible for business distribution and logic execution in each vertical domain, and provides self-describing information of all atomic service capability sets.

[0099] The TPU receives and parses the DSL description information from the large language model, generates a task list through the task engine, and distributes task scheduling and UI rendering, thereby implementing the method shown in this embodiment.

[0100] Step 304: Generate an interactive interface for perception function settings based on the interface description information.

[0101] In some embodiments, step 304 may specifically include: generating a task list based on trigger condition setting information and corresponding execution action setting information; generating an interactive interface for setting the perception function by scheduling the execution of tasks in the task list, wherein the interactive interface includes trigger condition setting information and corresponding execution action setting information.

[0102] The task list may include various tasks required for scheduling to generate the interactive interface. By scheduling and executing the tasks in the task list, an interactive interface for setting the perception function is generated.

[0103] Exemplarily, generating a task list based on trigger condition setting information and corresponding execution action setting information may specifically include: obtaining corresponding interface content layout information and interface dynamic effect (or simply referred to as dynamic effect) information according to the trigger condition setting information and corresponding execution action setting information required by the user; then generating a task list based on the interface content layout information and the interface dynamic effect information; accordingly, generating an interactive interface by scheduling the execution of tasks in the task list, which may specifically include: generating an interactive interface for setting perception functions according to the interface content layout information and the interface dynamic effect information by scheduling the execution of tasks in the task list.

[0104] The interface content layout information may include the setting items of the trigger conditions contained in the interface, the setting items of the corresponding execution actions, and the layout and display style of these setting items in the page. In addition, it may also include the name setting item of the perception function and whether to ask the user before starting the operation. The interface dynamic effect information may include the dynamic display effects of some styles in the interface to enhance the user experience.

[0105] In this embodiment, personalized or unified interface content layout information and interface dynamic effect information can be obtained based on the trigger condition setting information that matches the user intention information and the corresponding execution action setting information, and then used as a basis to generate a corresponding interactive interface to enhance the user's visual perception and improve the user experience.

[0106] Step 305: output an interactive interface for setting the perception function.

[0107] Technically, this can be achieved through a generative interaction framework. Figure 4 As shown in the figure, after receiving the interface display request from the task engine, the generative interaction uses the user interface generated by the large language model to present the interactive content to the user in a direct way. The generative interactive interface emphasizes content that is strongly related to the user's intention. The user no longer needs to click layer by layer to find related functions, and does not need to see other irrelevant content. Generative interaction mainly includes three core parts: generative UI, generative animation, and generative action. After the user triggers the generative action, it will be directly consumed or forwarded to the task engine to form a new round of scheduling.

[0108] Among them, the presentation capabilities of the generative interaction framework include: receiving the DSL layout part distributed from the TPU, and generating core UI interaction elements such as trigger conditions and execution actions through the generative interaction framework in combination with the existing motion effect engine, such as showing the user: "When certain conditions are met, delay scheduling to execute certain actions." This stage completes the transformation of the language intelligent DSL layout part to the visible UI interaction part. The interactive capabilities of the generative interaction framework include: motion effects, events and other components, supporting manual touch and perceptible speaking. The results of the interactive behavior will be passed to the TPU in the form of tasks in the task engine, and will be further distributed and processed by the TPU. Users can edit trigger conditions and corresponding execution actions in the interactive interface.

[0109] TPU allows business-oriented agents to register self-describing plugins to provide atomic capabilities to the system. Agents receive TPU distribution requests in the form of tasks and hand over the requests to business plugins for processing. At this stage, each business Agent Plugin receives the URI distributed from the TPU, parses it according to the URI rules, and completes the specific API call execution.

[0110] For this embodiment, the corresponding tasks are generated based on the trained large language model and the user's multimodal input; after the tasks are sent to the vehicle side, the vehicle side parses them and manages the execution status of the tasks; the parsed tasks are distributed to various businesses for execution, and the execution results are fed back to the user.

[0111] This embodiment is equivalent to identifying user intentions through fusion event input, completing flexible task arrangement through a large language model, and building a programming platform for users to control personalized vehicle interactions. By creating a vehicle-side control center through a large language model, you can get what you want without having to worry about the preset interaction form, which greatly improves the interaction efficiency.

[0112] Step 306: Obtain information on the user's setting result for the perception function in the interactive interface.

[0113] The user can adjust the trigger condition options and the corresponding execution action options displayed in the interactive interface, or directly confirm the perception function composed of the trigger condition and the corresponding execution action, and decide to turn on the perception function, etc.

[0114] In actual use, it is possible that the perception function set by the user may be non-compliant and / or unsafe. In order to avoid these problems, in some embodiments, after step 306, this embodiment may also include: performing a functional check on the legitimacy and / or security of the target perception function corresponding to the setting result information; if the functional check passes, the user is allowed to use the target perception function. If the target perception function set by the user passes the functional check on the legitimacy and / or security, it means that there will be no non-compliant and / or unsafe situations after the target perception function is turned on, and the user can be allowed to use the target perception function.

[0115] Exemplarily, performing a functional check on the legality and / or safety of a target perception function corresponding to the setting result information may specifically include: performing a functional check on the legality and / or safety of the target perception function in combination with the vehicle's driving environment information and the user's function usage permission information.

[0116] The vehicle's driving environment information may include weather, geographic location, temperature inside and outside the vehicle, current driving speed, etc. The user's function usage permission information may include the user's permission to use vehicle functions, such as the user has the right to use vehicle function A, has no right to use vehicle function B, has the right to use vehicle function C after 1 p.m., etc.

[0117] This embodiment can combine the vehicle's driving environment information and the user's function usage permission information to perform a functional check on the legality and / or safety of the target perception function set by the user. It can accurately determine whether there will be non-compliant and / or unsafe situations after the target perception function is turned on. For example, during the driving of the vehicle, the perception function set by the user is to trigger the door to open through gestures. Considering safety, the user may not be allowed to use this perception function to ensure driving safety. For another example, the rear-seat user has no right to use the light perception activation function in the front row area. If the perception function set by the rear-seat user is to trigger the light in the front row area through gestures or voice, the user may not be allowed to use this perception function in consideration of compliance, which is also to avoid interfering with the driving of the front driver.

[0118] Step 307: update the training large language model according to the setting result information of the perception function by the user in the interactive interface.

[0119] The intention update information is used to update the training set, and then the updated training set is used to update and train the large language model, thereby continuously improving the large language model so that the user intention calculated by the large language model is more accurate. This can be achieved through the shadow system.

[0120] In terms of model training, learning corpus is provided for the big model in structured and semantic forms such as Agent, Plugin, and API. In addition to task execution results and historical data, it can also include: source code, system documentation, open interface descriptions, application descriptions and other data, such as: historical data, travel data, basic vehicle data such as vehicle control and vehicle settings, and the capabilities supported by the dynamic effect engine. TPU data is fed back to the big language model. After the API method call is executed, feedback DSL is generated through the DSL engine and fed back to the big language model through TPU to achieve model update training.

[0121] The technical implementation process involved in this embodiment is described in detail below:

[0122] In order to achieve reliable, efficient, and scalable AI interaction, a more systematic design is required in the four links of model training, task generation, task analysis, and task execution:

[0123] Model training: Build a unified model training platform, which receives feedback on execution results from the shadow system and simulation platform, and continuously optimizes the model; Shadow system: runs in the virtual system on the vehicle side. The shadow system controller continuously inputs environmental information, user behavior and other information, calls the vehicle side TPU (DSL engine, Agent system and task engine) for background training, generates user personalized results in real time, and feeds the results back to the cloud to provide input for personalized tuning of the model. Simulation platform: simulates the user's actual vehicle status. The simulation controller continuously inputs massive user historical data and other information, calls the same TPU service as the vehicle side for large-scale training, and feeds the execution result information back to the cloud large model to provide input for general effect tuning of the model.

[0124] Task generation: Tasks are automatically generated by a large language model based on DSL. The DSL must ensure that the task description is accurate, the vehicle side can fully parse it, and the business can be executed normally. Task parsing: The vehicle side needs to have a unified task processing unit (TPU) to receive, parse and manage tasks. It has four functions:

[0125] (1) Call the DSL parser to parse the DSL information and generate an intent list;

[0126] (2) Unified scheduling and management of one or more intents required for the task;

[0127] (3) Manage the results and context of task execution;

[0128] (4) Control center when related business jumps.

[0129] Task execution: A unified agent is required to manage the API capabilities of each vertical domain or system. The Agent acts as a unified agent for business capabilities and connects to the task engine module of the TPU service, making the boundaries clearer. Each business provides atomic service capabilities and is registered in the Agent in the form of a plugin (such as LBS for travel services and CAR_CONTROL for vehicle control services). The business plugin and the corresponding business execute operations through remote calls.

[0130] Task feedback and optimization: After the task is completed, the execution results can be collected and stored as new corpus for large language model training to improve quality.

[0131] The following will expand on three aspects: Agent capability building, model training, and DSL information analysis and execution.

[0132] For Agent capability building, Figure 5 As shown in the figure, it describes the calling relationship between Agent, Plugin, TPU and business. After receiving multi-modal input (voice, gesture, environment and other information) from the vehicle, the large language model outputs DSL instruction information and sends it to the TPU service for unified processing; the DSL parser parses, distributes and manages the tasks; the task engine is responsible for generating task lists and scheduling the tasks; after receiving the tasks sent by the engine, the Agent performs address matching, intent distribution and other operations, and uniformly manages the Plugin; the Plugin and the business party perform logical execution through remote calls; the task engine sends the results, context and other information after execution back to the DSL engine; the DSL engine generates DSL receipt information based on the execution results and reports it to the large language model for multi-round conversation management and model optimization.

[0133] For model training, the core capability of the big language model is to fully understand user intent and dynamically generate tasks in combination with vehicle-side business capabilities. A good model requires a reasonable training strategy, which should have the following five characteristics:

[0134] a. Continuous massive training: Uninterrupted large-scale training is carried out by inputting multi-modal data (historical data of massive users, data on users' car-using habits, etc.). The entire system runs in the simulation platform. This link provides general optimization input for the large language model. b. Personalized fine-tuning: Collect the real-time execution results of vehicle-side tasks for fine-tuning and optimization to achieve a model effect with thousands of faces for thousands of people. This mode is integrated in the vehicle-side virtual system in the form of a shadow system. This link provides personalized input for the large language model. c. Possessing a twin training environment: Whether it is a simulation platform or a shadow system, in addition to inputting user multi-modal data, it should also be able to call on the same business set capabilities as the vehicle side. d. Easy to expand: The atomic service capabilities of each business on the vehicle side should support semanticization according to certain rules, so that the large language model can realize automatic labeling. e. Maintainable: With the rapid iteration of business and the continuous updating of atomic service capabilities, the large language model should have the ability to maintain multiple versions and be fault-tolerant.

[0135] like Figure 6 As shown in the figure, it describes the overall structure that a large language model training platform should have. Among them, the shadow system: runs in the virtual system on the vehicle side. The shadow system controller continuously inputs environmental information, user behavior operations and other information, calls the vehicle-side TPU service for background training, generates user personalized results in real time, and feeds back the results to the cloud to provide input for personalized tuning of the model; simulation platform: simulates the user's actual vehicle status. The simulation controller continuously inputs massive user historical data and other information, calls the same TPU service as the vehicle side for large-scale training, and feeds back the execution result information to the cloud-based large model to provide input for general effect tuning of the model; the cloud-based large model continuously optimizes and updates the model, maintains multiple versions of management internally, and integrates the execution result feedback of the shadow system and the simulation platform to improve the task generation effect.

[0136] Figure 6 It can be seen that the core is that both the simulation platform and the shadow system call the same Agent, that is, in addition to being the unified agent of the HU's TPU service, the Agent is also the core input for large language model training; the biggest benefit brought by the consistency of the Agent is that the tasks generated by the trained model are parseable and executable on the vehicle side, meeting the two characteristics of consistent training environment and high maintainability of model training.

[0137] In order to automatically identify all the capabilities of the Agent during large model training, realize automatic labeling, and ensure the scalability of model training, it is necessary to semanticize the interfaces of each business plug-in in the Agent and provide the model with understanding and training capabilities in the form of an interface description file; it is integrated into the plug-in as a configuration file. The description should contain information such as function introduction, URI, parameters, return value, error code, example, etc. Figure 7The following is a semantic description of a weather query interface.

[0138] In addition, the Agent should also maintain a set of domain name related information of the Plugin, which is called the business description file here and integrated into the Agent in the form of an index file; when the Agent receives a task (including URI), the Intent Matching Engine (IME) inside the Agent can quickly distribute the URI to the corresponding Plugin for execution; the file should contain information such as business introduction and URI domain name configuration.

[0139] The large language model can significantly reduce analysis and decision-making time by continuously learning and understanding the semantic information provided by business description files and interface description files, and has stronger scalability and higher training efficiency.

[0140] In this embodiment, the parsing and execution of DSL information can be performed as follows: Figure 8 As shown, the specific process may include the following:

[0141] Step 1: The large language model sends DSL information to the TPU service of HU for unified processing. TPU calls the DSL parser to parse, distribute and manage the tasks;

[0142] Step 2: The task engine is responsible for generating a task list and scheduling and executing tasks;

[0143] Step 3: The Agent distributes the intent to the Intent Matching Engine (IME) module and manages the execution status of the Plugin in a unified manner.

[0144] Step 4: The IME module in the Agent is responsible for operations such as address matching and intent distribution. For example, the IME module matches the interactive function (business side) set by the perception function according to the user's intent information, and addresses and calls the corresponding Plugin.

[0145] Steps 5 and 6: The plugin and the business side (the APP used to perceive function settings) perform logic execution through remote calls;

[0146] Step 7: After the task meets the triggering conditions, if there is a new task to be executed, it needs to be forwarded to the task engine and the task list is regenerated until the task is completed and the cycle ends;

[0147] Step 8: The display of UI components will be presented by generative interaction;

[0148] Step 9, the task engine sends the layout display intent to the generative interaction for presentation; the layout display intent may include a layout description of the interface content, and a description of the interface dynamic effects, etc. The generative interaction can generate a corresponding interactive interface based on these descriptions. For example, by identifying and parsing the interface description information, the basic elements, attributes, and association rules between the basic elements in the final user interface can be determined. Then, based on the basic elements, the attributes of the basic elements, and the association rules between the basic elements, a user interface is generated. The basic elements here can be regarded as components, as the basic units of display and processing, for example, the basic element e1 is a button, and the basic element e2 is a picture. Attributes can be exemplified by the width or length of the basic element e1, or the content of the picture, etc. The association rule can be an association relationship or a progressive relationship between multiple basic elements, or it can be a rule. The reliability of the user interface generation process can be guaranteed in the above manner.

[0149] Step 10: When user interaction triggers a new Action request, the task needs to be forwarded to the task engine and the task list is regenerated until the task is completed and the cycle ends.

[0150] All vehicle-side services are registered in the Agent in the form of Plugins, which realizes unified management of external services and prevents logical conflicts caused by concurrent business calls. The Agent provides unified interface capabilities for the HU-side TPU service, shadow system, and simulation environment. The consistency enables seamless migration of pre-trained models. By semanticizing the atomic service capabilities of the Plugins in the Agent, large models have the ability to automatically annotate interfaces. At the same time, the IME module inside the Agent also benefits from the semanticization of interfaces to establish a unified URI addressing mechanism, making business Plugin matching and distribution more efficient.

[0151] For the input of user interaction data, for the input type, such as Figure 8 As shown. The interactive data used for input should include all perceptible and detectable data, which we collectively refer to as fusion input here, including but not limited to image perception, acoustic perception, touch perception, geographic information, vehicle status, user behavior habits and environmental information (smell, light, etc.) from inside and outside the vehicle. These data are used as multi-modal raw data input, processed by the large language model, and form interactive data described in DSL language, which is passed to the HU side.

[0152] For the output of user interaction data, the interaction data described in DSL language, including but not limited to UI, operation and behavior, is sent to each Plugin in DSL form for execution and presentation. Output type: including but not limited to generative display UI, action execution, TTS, etc.; generative interaction mainly includes UI layout, animation and action.

[0153] In this embodiment, generative interaction is a technology based on a large language model that automatically generates interfaces and interactive elements by learning user behavior and data. Compared with traditional applications, generative interaction has the following core values:

[0154] It has product value and improves user experience: Generative interaction can automatically generate personalized interfaces and interactive elements based on user needs and habits, making user use smoother, more natural and more enjoyable. Improve development efficiency: Generative interaction reduces a lot of manual design and programming workload, which can significantly shorten the development cycle and improve development efficiency. Reduce labor costs: Traditional applications require a lot of manpower for design and development, while generative interaction can reduce these costs and reduce development and maintenance costs. Optimize user behavior: Generative interaction can optimize the user experience and improve user satisfaction and loyalty by learning user behavior and data. Better respond to market demand: Generative interaction can respond to market demand more promptly, quickly generate applications that meet user needs, and help companies better meet market demand and improve competitiveness.

[0155] It has interactive efficiency value. Taking the addition, deletion and editing tasks as an example, generative interaction can automatically generate interfaces and interactive elements based on the user's historical behavior, environmental perception and other data, allowing users to use the application more easily. The interactive benefits brought include but are not limited to: automatic filling: when the user enters the task intention, generative interaction can automatically fill in the identified task conditions and action information, such as seat occupancy, speed limit, sunroof opening, air conditioning, etc. This not only saves the user's time, but also reduces the possibility of user input errors; local update: when the new generative interaction is issued and presented, it can be targeted for the local content of the page. Update, without the need to reload the entire page; perceptible is said: visible interactive elements, as well as invisible perceptual elements, can be located through language intelligence through a unified URI form matching to achieve interaction; recommended tasks: based on user behavior and associated perception data, generative interaction can automatically recommend task conditions and actions that users may add, and recommend multiple tasks in the form of task cards in the dialogue area. In this way, users can make decisions more quickly about the task data that needs to be added; Deletion confirmation: When deleting a task, generative interaction can provide a confirmation prompt for the deletion information to prevent irreparable losses caused by user errors; Editing task information: Generative interaction can provide intuitive and easy-to-understand interfaces and interactive elements based on user natural language input, such as highlighting the content being modified, to help users easily modify task information and automatically save the modified data to avoid problems caused by omissions due to manual saving of information.

[0156] Through the above interactive features, generative interaction allows users to complete the operations of adding, deleting and editing tasks more quickly and efficiently, improves the usability and user experience of the application, greatly improves the interaction efficiency, and thus increases user stickiness and loyalty.

[0157] The following is a detailed introduction to the generative interaction solution.

[0158] Generative interactive layout DSL, abstracts standard controls or card-style components through DSL, connects with algorithms to achieve generation capabilities, and outputs and presents solution processes, such as Fig. 9 As shown in the figure, there are several processes of the generative UI framework:

[0159] UI layout parser: A logical module used to parse the layout DSL language. It converts the DSL code into element objects or attributes of the UI system, including layout type, layout attributes, size, position and other information for subsequent processing. The parser needs to understand the DSL syntax and convert the DSL code into a restrictive layout tree according to the rules.

[0160] UI tree structure generation: The process of creating a UI tree based on the restrictive layout tree generated by the parser. In this process, in order to make the layout more concise and effective, the framework will package the corresponding UI components into meaningful components. For example, a button composed of multiple images can become a separate component for reuse in multiple pages.

[0161] UI layout optimization: A process designed to optimize the hierarchical structure of the UI layout. In this process, the framework analyzes the generated UI tree structure, finds out the levels that can be reduced, and merges multiple UI components with the same level into one component, thereby making the hierarchical structure of the UI interface more concise and improving rendering efficiency.

[0162] UI interactive presentation: The final process is to convert the UI tree structure into an interface and present it to the user. This process includes steps such as UI style processing, layout calculation, animation processing, and view rendering. Finally, the generative UI framework will render the generated results to the user interface so that the user can interact accordingly.

[0163] Generative animation relies on the existing animation engine architecture, and mainly includes the animation description statement Effect DSL, the animation parser Effect Parser and the animation loader Effect Loader. The Effect DSL description includes the animation process description and rendering effect description of the control, which are parsed by the animation parser (Effect Parser). The Effect Loader combines the animation transformation (translation, scaling, rotation, transparency, etc.) and rendering effects (blur, color mode) with the ability support of the animation engine to apply the animation effects to the presentation of the view.

[0164] Animation template library: provides descriptions of basic animation attributes that can be generated, such as position change, transparency change, and Gaussian blur configuration; Animation generator: is responsible for combining the data output by the large language model to generate a DSL description that matches the template library, including a single animation attribute, or a combination of multiple animation attributes; it can be a static rendering effect or a responsive animation / effect change; Animation parser: parses the DSL description and creates an animation Builder type, which will rebuild the animation attributes before the generator is generated, disassemble and associate the parameters of each animation, and can combine a group of animations into a composite animation according to the corresponding syntax; Animation loader: instantiates the corresponding animation in the Builder type into the corresponding animation / rendering effect in the operating system, and loads it to the corresponding UI layout.

[0165] For UI action execution, action execution is an important capability of interactive output, including but not limited to vehicle control and vehicle setting capability calls based on DSL description, travel capability calls, TTS broadcast / copy display capability calls, task addition, deletion and modification capability calls, etc., as well as various other business capability calls. For example: opening a specific page and calling the capability of a specific plugin. Action can be attached to any node in the tree structure, and Action can point to a new generative UI or an atomic service to complete the function. DSL is roughly divided into two dimensions, one is the view description of View type, and the other is interactive action and task execution. View can be bound to Action or Function, and Function can be executed alone. The architecture of View is basically composed of containers and standard controls. Standard controls can be regarded as the smallest display unit of UI. It must be placed in a container. Different standard controls and containers are combined to form the final UI display effect.

[0166] For example, the form presented to the user includes what is seen and heard (core UI, important multi-modal interaction form). Text / image display: Use the display device to directly display text information / image information to the user. Through the interface rendering engine, the acquired DSL data is rendered into an interface that the user can understand and presented to the user. Animation effect: Provide visual feedback by controlling image changes and animation effects. Through the motion effect engine, the acquired DSL motion effect data is rendered into motion effects and applied to the specified UI controls.

[0167] And voice feedback can be made: the result is converted into voice using speech synthesis technology, and played to the user through the audio output device, and speech synthesis is realized through speech synthesis engine, speech synthesis algorithm or cloud service; tactile feedback: motor-based vibration feedback: by installing a vibration motor on the seat / steering wheel, controlling the vibration intensity and frequency of the motor to achieve the vibration of the seat / steering wheel; touch feedback based on pressure sensor: by installing a special pressure sensor and controller under the touch screen, when the user touches the screen with his finger, the controller will send instructions to the pressure sensor to generate some vibration and sound to simulate the feedback of clicking a physical button. Olfactory feedback: based on the smell release device installed in the car, such as fragrance, etc., by controlling the switch of the device and the type and amount of smell release, the user can smell a specific smell; vehicle control feedback: by sending control instructions to the corresponding controllers of vehicle equipment such as windows, air conditioning, seat heating, ventilation and massage, the corresponding functions can be adjusted, switched on and off, etc.; other forms of feedback: in addition to the feedback forms listed above, it includes but is not limited to other devices triggered by lines or machinery, signals, external forces, etc. to provide user interactive feedback.

[0168] For example, the interactive interface can also include an AI dialogue interface and an interactive interface. Specifically, the right half of the generated user interface is an interactive interface generated based on the user's control input information and combined with the driving environment information and historical interactive behavior information. The left half also has an AI dialogue part for the user to confirm with the user whether the interactive interface currently generated on the right is consistent with the content that the user needs to interact with. For example, the specific display effect can be as follows Fig.10 As shown, the left resident AI interaction area contains the recognized user dialogue input; in-car sensory information input, such as gear position, seat, seat belt, gesture, etc.; AI makes dialogue feedback or execution suggestions based on the input. The right dynamic application content display area, the content of this area is given by AI based on user and sensory input through large model analysis, and the content is composed of the following parts or displayed exclusively:

[0169] The single application display is similar to the traditional opening of independent applications; it contains various functional contents of layout data, such as navigation, destination weather, vehicle information, merchant details, intersection or traffic light countdown prompts, etc.

[0170] The page to be provided in this embodiment displays the programming results in real time according to the DSL data sent by the big model, identifies the instruction actions by parsing the DSL data, and makes real-time changes to the page information according to the identified instructions, such as adding, deleting and modifying conditional actions. After the user manually edits the conditional actions, the editing results are synchronized to the big model in real time.

[0171] Further, as Figures 1 to 3 The specific implementation of the method shown in this embodiment provides a device for generating an interactive interface, such as Fig.11 As shown, the device includes: an acquisition unit 41, an identification unit 42, and a generation unit 43.

[0172] An acquisition unit 41 is configured to acquire interactive input information of a user, wherein the interactive input information includes multimodal feature data;

[0173] The recognition unit 42 is configured to input the multimodal feature data as input data into the large language model, so as to recognize the language description information corresponding to the interactive input information through the large language model; and analyze the language description information to obtain the interface description information of the interactive interface;

[0174] The generating unit 43 is configured to generate an interactive interface based on the interface description information.

[0175] In some embodiments, the recognition unit 42 is specifically configured to parse the language description information to obtain the user's intention information; perform analysis based on the intention information to obtain interface description information of the interactive function matching the intention information;

[0176] The generating unit 43 is specifically configured to generate an interactive interface of the interactive function based on the interface description information of the interactive function.

[0177] In some embodiments, the identification unit 42 is specifically configured to analyze the intention information to obtain interface content layout information and interface dynamic effect information of the interactive function matching the intention information;

[0178] The generating unit 43 is specifically configured to generate an interactive interface of the interactive function according to the interface content layout information and the interface dynamic effect information.

[0179] In some embodiments, the interactive function is to set a perception function of the cockpit system, and the interactive interface for setting the perception function includes a trigger condition interaction element of the perception function and a corresponding execution action interaction element.

[0180] In some embodiments, the recognition unit 42 is specifically configured to recognize natural language text information corresponding to the interactive input information; combine the user's historical behavior data and car usage habit data, analyze the natural language text information through a large language model, and obtain the language description information.

[0181] In some embodiments, the generation unit 43 is further configured to output an interactive interface for the interactive function; obtain setting result information of the interactive function by the user in the interactive interface; and update and train the large language model according to the setting result information.

[0182] In some embodiments, the identification unit 42 is further configured to perform a functional check on the legality and / or security of the target interactive function corresponding to the setting result information; if the functional check passes, the user is allowed to use the target interactive function.

[0183] In some embodiments, the identification unit 42 is further configured to perform a functional check on the legality and / or security of the target interactive function in combination with the vehicle's driving environment information and the user's function usage permission information.

[0184] It should be noted that for other corresponding descriptions of the functional units involved in the device for generating an interactive interface provided in this embodiment, reference can be made to Figures 1 to 3 The corresponding description in will not be repeated here.

[0185] Based on the above Figures 1 to 3 The method shown in the embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figures 1 to 3 The method shown.

[0186] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0187] Based on the above Figures 1 to 3 The method shown, and Fig.11 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides an electronic device, which can be configured on a vehicle (such as a new energy vehicle) or a server, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figures 1 to 3 The method shown.

[0188] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0189] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0190] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0191] Furthermore, based on the above Figures 1 to 3 The method shown, and Fig.11 The virtual device embodiment shown in the figure, as well as the electronic device embodiment described above, this embodiment also provides a vehicle, which may include the electronic device described above, and may further perform the following Figures 1 to 3 The method shown.

[0192] Through the description of the above implementation methods, the technical personnel in this field can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the scheme of this embodiment, compared with the current prior art, this embodiment accurately identifies the interactive interface required by the user through the user's interactive input, such as simple voice input, and generates a corresponding interactive interface according to the interface description information of the interactive interface, which reduces the user's manual search operations, facilitates the user to interact directly, and improves the interaction efficiency. For example, an interactive interface for the perception function setting is generated, and the user can directly adjust the perception function according to the perception function setting information in the interaction interface to create the perception function they need. In the whole process, the user's operation can be effectively reduced, and there is no need to search for the relevant setting pages layer by layer in the vehicle system. The perception function of the cockpit system can be quickly set, which can improve the efficiency of the perception function setting of the cockpit system, and thus improve the efficiency and experience of the user using the cockpit system.

[0193] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0194] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

Claims

1. A method for generating an interactive interface, characterized in that: include: Acquiring interactive input information of a user, wherein the interactive input information includes multimodal feature data; Inputting the multimodal feature data as input data into a large language model, so as to identify language description information corresponding to the interactive input information through the large language model; Analyze the language description information to obtain interface description information of the interactive interface; An interactive interface is generated based on the interface description information.

2. The method according to claim 1, characterized in that The analyzing according to the language description information to obtain the interface description information of the interactive interface includes: Parsing the language description information to obtain the user's intention information; Analyze the intent information to obtain interface description information of the interactive function matching the intent information; The generating an interactive interface based on the interface description information includes: Based on the interface description information of the interactive function, an interactive interface of the interactive function is generated.

3. The method according to claim 2, characterized in that The analyzing according to the intent information to obtain interface description information of the interactive function matching the intent information includes: Analyze the intention information to obtain interface content layout information and interface dynamic effect information of the interactive function matching the intention information; The generating the interactive interface of the interactive function based on the interface description information of the interactive function includes: An interactive interface for the interactive function is generated according to the interface content layout information and the interface dynamic effect information.

4. The method according to claim 3, characterized in that The interactive function is to set the perception function of the cockpit system, and the interactive interface for setting the perception function includes a trigger condition interaction element of the perception function and a corresponding execution action interaction element.

5. The method according to claim 4, characterized in that The identifying, by using a large language model, language description information corresponding to the interactive input information includes: Identifying natural language text information corresponding to the interactive input information; The natural language text information is analyzed by a large language model in combination with the user's historical behavior data and car-using habit data to obtain the language description information.

6. The method according to any one of claims 2 to 5, characterized in that After generating the interactive interface of the interactive function, the method further includes: Outputting an interactive interface of the interactive function; Obtaining information about the result of the user setting the interactive function in the interactive interface; According to the setting result information, the large language model is updated and trained.

7. The method according to claim 6, characterized in that After obtaining the setting result information of the interactive function by the user in the interactive interface, the method further includes: Performing a function check on the legality and / or security of the target interactive function corresponding to the setting result information; If the function check is passed, the user is allowed to use the target interactive function.

8. The method according to claim 7, characterized in that The function verification of the legality and / or security of the target interactive function corresponding to the setting result information includes: In combination with the vehicle's driving environment information and the user's function usage permission information, the target interactive function is subjected to a functional verification of legality and / or security.

9. A device for generating an interactive interface, characterized in that: include: An acquisition unit, configured to acquire interactive input information of a user, wherein the interactive input information includes multimodal feature data; an identification unit configured to input the multimodal feature data as input data into a large language model, so as to identify language description information corresponding to the interactive input information through the large language model; and to analyze the language description information to obtain interface description information of the interactive interface; The generating unit is configured to generate an interactive interface based on the interface description information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

12. A vehicle, characterized in that: include: The device as claimed in claim 9, or the electronic device as claimed in claim 11.

Images