Vehicle-mounted man-machine interaction method and device and electronic equipment

Abstract

The invention discloses a vehicle-mounted human-computer interaction method and device and electronic equipment, and belongs to the technical field of human-computer interaction. The method specifically comprises the following steps: collecting vehicle-mounted scene information; the vehicle-mounted scene information comprises vehicle environment information and / or user behavior information; determining a target interface element corresponding to the vehicle-mounted scene information, and generating target three-dimensional data corresponding to the target interface element according to the vehicle-mounted scene information; the target interface element is an existing interface element, or the target interface element is a newly added interface element; rendering the target three-dimensional data to obtain a three-dimensional HMI interface corresponding to the target interface element; and displaying the three-dimensional HMI interface. According to the method, the display mode, the layout or the interaction priority of the interface elements in the three-dimensional HMI interface can be automatically adapted based on the vehicle environment or the user behavior reflected by the vehicle-mounted scene information.

Description

Technical Field

[0001] This application belongs to the field of human-computer interaction, specifically relating to an in-vehicle human-computer interaction method, device, and electronic device. Background Technology

[0002] With the continuous improvement of vehicle intelligence, in-vehicle HMIs (Human-Machine Interfaces) have become a crucial bridge for information exchange between users and vehicle systems. To address the increasingly complex needs of human-vehicle interaction, 3D HMI technology has emerged. This technology constructs an interactive 3D visualization interface, transforming information such as vehicle functions, operating status, and the surrounding environment into a 3D scene composed of various interface elements, enabling users to interact with the vehicle in a way that more closely resembles real-world perception.

[0003] Current in-vehicle human-machine interaction methods typically begin by constructing a 3D scene based on predefined interface elements, and then rendering this 3D scene into a 3D HMI interface. These interface elements specifically include a 3D vehicle model and its surrounding environmental elements. The 3D vehicle model serves as the core of the interaction, allowing users to adjust vehicle functions through operations, such as adjusting seat angles and controlling window opening and closing. The surrounding environmental elements present traffic information around the vehicle, such as lane markings and intersection locations, assisting the driver in understanding the driving environment and anticipating potential risks, thereby improving driving safety.

[0004] However, the 3D scenes currently constructed are essentially static. The interface elements they contain (such as 3D vehicle models and surrounding environmental elements) are usually predefined and have fixed attributes. This results in a lack of flexibility and adaptability in the content presented by the 3D HMI interface, making it difficult to meet the complex and ever-changing human-vehicle interaction needs. For example, the appearance attributes (such as color and shape) and internal components (such as dashboard style and seat shape) of vehicle model objects usually adopt fixed configuration schemes; similarly, surrounding environmental elements (such as road topology, traffic signs, building appearance, etc.) also mostly rely on pre-configured static content. Summary of the Invention

[0005] The purpose of this application is to provide an in-vehicle human-machine interaction method, device, and electronic device that can solve the problem that the content presented by the three-dimensional HMI interface lacks flexibility and adaptability, making it difficult to meet the complex and ever-changing human-vehicle interaction needs.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide an in-vehicle human-machine interaction method, the method comprising: collecting in-vehicle scene information; the in-vehicle scene information including: vehicle environment information and / or user behavior information; The target interface element corresponding to the vehicle scene information is determined, and target 3D data corresponding to the target interface element is generated based on the vehicle scene information; the target interface element is an existing interface element, or the target interface element is a newly added interface element; the target 3D data is rendered to obtain the 3D HMI interface corresponding to the target interface element; the 3D HMI interface is displayed.

[0007] Secondly, embodiments of this application provide an in-vehicle human-machine interaction device, the device comprising: a data acquisition module for acquiring in-vehicle scene information; the in-vehicle scene information including: vehicle environment information and / or user behavior information; an interface element determination module for determining a target interface element corresponding to the in-vehicle scene information; a three-dimensional data generation module for generating target three-dimensional data corresponding to the target interface element based on the in-vehicle scene information; the target interface element being an existing interface element, or the target interface element being a newly added interface element; a rendering module for rendering the target three-dimensional data to obtain a three-dimensional HMI interface corresponding to the target interface element; and a display module for displaying the three-dimensional HMI interface.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] This application provides a mechanism for generating interface content driven by in-vehicle scene information. This mechanism no longer relies on completely pre-set static content, but instead determines the corresponding target interface element (which can be an existing or newly added interface element) based on real-time collected in-vehicle scene information (including vehicle environment information and / or user behavior information), generates target 3D data corresponding to the target interface element, and renders the target 3D data to obtain the corresponding 3D HMI interface.

[0011] Because the aforementioned mechanism enables the transformation of interface content from preset to generated, the embodiments of this application can transform the 3D HMI interface from a static, uniform display into an intelligent interface that can dynamically adjust according to the vehicle environment or user behavior reflected by in-vehicle scene information. Based on the vehicle environment or user behavior reflected by in-vehicle scene information, the embodiments of this application can automatically adapt the display method, layout, or interaction priority of interface elements in the 3D HMI interface, thereby providing flexible and precise interaction support in diverse driving scenarios, effectively improving the adaptability and operational efficiency of human-vehicle interaction, and enhancing driving safety. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the steps of an embodiment of the in-vehicle human-machine interaction method of this application; Figure 2 This is a structural block diagram of an embodiment of an in-vehicle human-machine interaction device according to this application; Figure 3 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] The in-vehicle human-machine interaction method, device, and electronic device provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0016] In this embodiment, in-vehicle human-machine interaction (HMI) can be used to realize two-way information transmission (i.e., the user issues commands to the vehicle, and the vehicle provides feedback and prompts to the user) and command execution between the user and the vehicle. Specifically, the aforementioned in-vehicle HMI can present the vehicle status, driving environment, and interactive controls in a three-dimensional and visual form through a 3D HMI interface. Users can use this for immersive scene interaction, such as directly clicking and dragging the vehicle model, thereby achieving precise control of vehicle functions (such as windows and seats). This in-vehicle HMI method transforms complex vehicle operation into simple and intuitive natural interaction, significantly improving the efficiency of human-vehicle interaction and driving safety.

[0017] To address the technical problem that the content presented by 3D HMI interfaces in related technologies lacks flexibility and adaptability, making it difficult to meet the complex and ever-changing human-vehicle interaction needs, this application provides an in-vehicle human-machine interaction method. The method specifically includes: collecting in-vehicle scene information; the in-vehicle scene information includes: vehicle environment information and / or user behavior information; determining the target interface element corresponding to the in-vehicle scene information, and generating target 3D data corresponding to the target interface element based on the in-vehicle scene information; the target interface element being an existing interface element, or the target interface element being a newly added interface element; rendering the target 3D data to obtain the 3D HMI interface corresponding to the target interface element; and displaying the 3D HMI interface.

[0018] This application provides a mechanism for generating interface content driven by in-vehicle scene information. This mechanism no longer relies on completely pre-set static content, but instead determines the corresponding target interface element (which can be an existing or newly added interface element) based on real-time collected in-vehicle scene information (including vehicle environment information and / or user behavior information), generates target 3D data corresponding to the target interface element, and renders the target 3D data to obtain the corresponding 3D HMI interface.

[0019] Because the aforementioned mechanism enables the transformation of interface content from preset to generated, the embodiments of this application can transform the 3D HMI interface from a static, uniform display into an intelligent interface that can dynamically adjust according to the vehicle environment or user behavior reflected by in-vehicle scene information. Based on the vehicle environment or user behavior reflected by in-vehicle scene information, the embodiments of this application can automatically adapt the display method, layout, or interaction priority of interface elements in the 3D HMI interface, thereby providing flexible and precise interaction support in diverse driving scenarios, effectively improving the adaptability and operational efficiency of human-vehicle interaction, and enhancing driving safety.

[0020] The in-vehicle human-machine interaction method provided in this application can enhance the intelligence and personalization experience of smart cockpits. This method endows smart cockpits with dynamic interactive capabilities by generating a 3D HMI interface adapted to in-vehicle scene information in real time. Specifically, it allows the smart cockpit to proactively adjust the interface content and interaction logic represented by target interface elements based on real-time in-vehicle scene information, thereby transforming traditional static information display into dynamic and precise interactive services.

[0021] The in-vehicle human-machine interaction method of this application embodiment can be executed by an electronic device. This electronic device may be an in-vehicle central control unit, a cockpit domain controller, an intelligent driving domain controller, an in-vehicle infotainment system host, or an in-vehicle central computing platform with computing capabilities, etc.

[0022] The in-vehicle human-machine interaction method of this application will be described below through specific embodiments.

[0023] Reference Figure 1 It illustrates a flowchart of a vehicle-mounted human-machine interaction method according to an embodiment of this application. The method specifically includes the following steps: Step 101: Collect in-vehicle scene information; the in-vehicle scene information specifically includes: vehicle environment information and / or user behavior information; Step 102: Determine the target interface element corresponding to the vehicle scene information, and generate the target 3D data corresponding to the target interface element based on the vehicle scene information; the target interface element can be an existing interface element, or the target interface element can be a newly added interface element. Step 103: Render the target 3D data to obtain the 3D HMI interface corresponding to the target interface element; Step 104: Display the three-dimensional HMI interface.

[0024] In step 101, in-vehicle scenario information refers to all information that reflects the vehicle's status, environmental conditions, and user behavior during vehicle operation. It is a crucial basis for the in-vehicle human-machine interaction system to make decisions. It is a comprehensive set of information that provides fundamental data support for achieving a more intelligent and convenient interactive experience.

[0025] Vehicle environmental information is a collection of information encompassing various aspects such as the vehicle's internal and external status, and its spatiotemporal location. It includes environmental data such as temperature and air quality inside the vehicle, as well as external conditions such as weather and road conditions. It also includes the vehicle's spatiotemporal information, such as its latitude, longitude, and altitude at a specific time, all of which reflect the surrounding environment in which the vehicle is operating.

[0026] User behavior information refers to the data generated by various user actions within the vehicle. Examples include voice commands, touchscreen operations, adjustments to in-vehicle equipment, and selected multimedia entertainment content. This information reflects user habits and immediate needs, helping the in-vehicle human-machine interface system better predict user demands and respond to user actions.

[0027] This application embodiment can utilize a data acquisition module to collect in-vehicle scene information. The aforementioned data acquisition module may include, but is not limited to: in-vehicle and external cameras, microphones, GPS (Global Positioning System) or BeiDou-based positioning modules, vehicle status sensors (such as vehicle speed, gear position, etc.), and user behavior data recording units. This data acquisition module is responsible for acquiring in-vehicle scene information such as vehicle environmental information and / or user behavior information in real-time or near real-time.

[0028] Of course, in addition to acquiring user behavior information in real-time or near real-time, historical user behavior data can also be obtained. This historical user behavior data can reflect users' usage habits and preferences. Specific user habits include: the types of music users frequently listen to, their preferred navigation routes, and their preferred in-car temperature settings, etc.

[0029] In step 102, the in-vehicle scene information can be analyzed, and the target interface elements corresponding to the in-vehicle scene information can be determined based on the analysis results. Specifically, if the analysis results include interaction requirements for the in-vehicle scene information, the interaction requirements are mapped to target interface elements in the 3D HMI interface; or, if the analysis results include environmental objects in the physical environment, the environmental objects in the physical environment are mapped to target interface elements in the 3D HMI interface. Mapping the analyzed interaction requirements to target interface elements in a 3D HMI interface effectively improves the convenience and efficiency of interaction. For example, by analyzing historical user behavior data, a pattern A can be identified: "Users frequently use the vehicle's air conditioning off function when entering the community on weekday evenings." When the vehicle's spatiotemporal information matches the spatiotemporal conditions corresponding to pattern A, the interaction requirement of "conveniently turning off the air conditioning" can be automatically mapped to a target interface element of "one-click air conditioning off." This process achieves scenario-driven intelligent interaction.

[0030] The aforementioned environmental objects may be dynamic entities that temporarily appear in the physical environment and affect driving safety, such as temporary roadside stalls or construction barriers; or they may be special obstacles, such as potholes on the road, scattered goods, or animals that have wandered onto the road. By mapping these environmental objects from the physical environment to target interface elements in the 3D HMI interface, and using interface elements such as visual highlighting or tactile warnings, environmental threats can be transformed into intuitive interactive prompts, thereby improving situational awareness and driving safety.

[0031] It should be noted that the target interface element can be an existing interface element, or it can be a newly added interface element. In other words, embodiments of this application can adjust the appearance attributes (such as size, color, or shape) of existing interface elements to meet the user's personalized needs; or they can dynamically create new interface elements to accurately meet the interaction needs triggered by in-vehicle scene information.

[0032] After determining the target interface elements, embodiments of this application can utilize 3D generation technology to generate target 3D data corresponding to the target interface elements based on the vehicle scene information. Specifically, the 3D generation technology may include: text-to-3D technology, image-to-3D technology, or multimodal 3D generation technology that integrates multiple types of input information.

[0033] The implementation of the 3D generation technology relies on a pre-trained 3D generation model, which may specifically include text-generated 3D models or image-generated 3D models.

[0034] In practical implementation, the input information used to drive the 3D generation model can be determined based on the analysis results corresponding to the vehicle scene information. This input information can be the analysis results themselves, or the attribute information of the target interface elements further extracted based on the analysis results.

[0035] If text-based 3D technology is used, the input information can be transformed into descriptive text prompts (such as "a semi-transparent flashing environmental object warning icon"), and the text-based 3D model can then generate target 3D data that conforms to the semantic description.

[0036] If image-generated 3D technology is used, the image corresponding to the input information (such as an image of icy road surface) can be input into the image-generated 3D model. The image-generated 3D model can reconstruct the corresponding target 3D data by recognizing the geometric structure and visual features in the image.

[0037] Furthermore, embodiments of this application can also combine input information from multiple modalities such as text, images, and depth information to construct target 3D data that is highly adapted to the scene through multimodal 3D generation technology.

[0038] This application embodiment can provide the following technical solution for determining the target interface element corresponding to the vehicle scene information, and generating the target three-dimensional data corresponding to the target interface element based on the vehicle scene information: Technical Solution 1 In technical solution 1, the in-vehicle scenario information includes: user behavior information; the user behavior information includes: user input information for VPA (Virtual Personal Assistant); The process of determining the target interface element corresponding to the vehicle scene information specifically includes: The VPA interface element is determined as the target interface element corresponding to the input information; The step of generating target 3D data corresponding to the target interface element based on the vehicle scene information includes: Based on the user's input information for VPA, VPA 3D model data with skeletal rigging and animation is generated, which serves as the target 3D data corresponding to the VPA interface elements.

[0039] VPA is a software program based on artificial intelligence and natural language processing technology. It usually has a visual representation of the program's running status, can understand user needs through voice dialogue, and can proactively perform tasks, provide information or control devices to achieve personalized services.

[0040] In traditional technologies, the visual appearance of virtual assistants can usually only be selected from one or a few preset models, making it impossible to achieve true personalized customization.

[0041] To overcome this limitation, this application provides a VPA customization solution. Specifically, based on user input information for the VPA, the VPA interface elements can be determined as the target interface elements corresponding to the input information. Furthermore, based on the user input information for the VPA, VPA 3D model data with skeletal rigging and animation is generated as the target 3D data corresponding to the VPA interface elements. The skeletal system provides movable joint structures for the 3D model, serving as the basis for driving the model to make facial expressions, lip movements, and body gestures. The animation data defines the motion rules of the skeleton, enabling the VPA avatar to perform vivid and natural dynamic interactive behaviors such as nodding, waving, and speaking.

[0042] In one example, after the user enters the vehicle, they activate the VPA customization function via the settings menu on the central control screen or a voice command. The user's input for VPA could be a voice command A: "Hello, Little A, please change your avatar to a cute astronaut kitten." The data acquisition module (microphone) in this embodiment captures the voice command A and converts it into command text. This embodiment can analyze the command text, and the analysis results can include attribute information of the VPA interface elements, such as the avatar name and style of the VPA interface elements. This embodiment can then input the attribute information of the VPA interface elements into the Wensheng 3D model.

[0043] Wensheng 3D Model can be deployed on cloud or local high-performance computing units, and can generate a 3D cat model with an astronaut appearance, basic skeletal binding, and pre-set corresponding animations in real time based on the input attribute information.

[0044] It is understood that the above voice commands are merely an example of user input for the VPA. In practical applications, user input is not limited to voice; it can also include directly entered text commands (such as typing "change the image to a sci-fi robot" on the in-vehicle screen) or provided reference images (such as uploading a picture and instructing "set the VPA image to resemble the character in this picture"). By supporting multimodal input methods, this solution can more flexibly meet users' personalized customization needs for the VPA image.

[0045] Technical Solution 2 In technical solution 2, the vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle spatiotemporal information; The process of determining the target interface element corresponding to the vehicle scene information specifically includes: determining the functional interface element corresponding to the vehicle spatiotemporal information based on the historical user behavior data corresponding to the vehicle spatiotemporal information, and using it as the target interface element corresponding to the vehicle spatiotemporal information; the functional interface element corresponds to first attribute information; the first attribute information includes: interaction response logic, which is used to respond to the user's trigger operation on the functional interface element and display the functional content corresponding to the functional interface element. The process of generating target 3D data corresponding to the target interface element based on the vehicle scene information specifically includes: generating target 3D data corresponding to the functional interface element based on the first attribute information of the functional interface element.

[0046] Vehicle spatiotemporal information refers to data that includes both timestamps and geographic coordinates, used to describe the changes in a vehicle's position and state in continuous spatiotemporal space.

[0047] Technical Solution 2 determines user interaction patterns based on historical user behavior data corresponding to vehicle spatiotemporal information. When the vehicle spatiotemporal information conforms to the spatiotemporal conditions corresponding to the pattern, the interaction requirements are mapped to functional interface elements in the 3D HMI interface, which can effectively improve the convenience and efficiency of interaction.

[0048] For example, by analyzing historical user behavior data, a pattern B can be identified: "After parking their car in the company parking lot around 9 a.m. every day, there is an 80% probability that the user will open the calendar and meeting apps." Thus, when the vehicle's spatiotemporal information matches the spatiotemporal conditions corresponding to this pattern B (time: around 9 a.m. every day, location: company parking lot), the interaction requirement of "opening the calendar and meeting apps" can be automatically mapped to a functional interface element that serves as the "entry point for the calendar and meeting apps."

[0049] In practical implementation, the first step is to collect and organize massive amounts of historical user behavior data. This data covers various operation records of users using vehicles at different times and locations, including but not limited to vehicle routes, stopping locations, stopping durations, and operations on the in-vehicle system during those times. After collecting the historical user behavior data, data analysis algorithms, such as association rule mining algorithms in machine learning, are used to conduct in-depth analysis of the historical user behavior data to identify potential user interaction patterns. Taking the pattern B, "Users have an 80% probability of opening the calendar and meeting applications after parking their cars in the company parking lot around 9 am every day," as an example, long-term monitoring of user behavior data can identify this fixed operating pattern of users in a specific time (around 9 am) and space (company parking lot) environment.

[0050] After collecting real-time vehicle spatiotemporal information, it is compared with identified user interaction patterns. Once the vehicle spatiotemporal information matches the spatiotemporal conditions corresponding to pattern B—that is, the time is around 9 AM and the vehicle is in the company parking lot—the corresponding interaction mapping mechanism is triggered. In the design of the 3D HMI interface, the interaction requirement of "opening the calendar and meeting applications" is mapped to a functional interface element called "Calendar and Meeting Application Function Entry." This function entry is presented in a prominent way on the 3D HMI interface, for example, in an area where the user's gaze is easily focused, with a large icon size and a unique color identifier.

[0051] Technical Solution 2, based on the analysis of historical data, enables the system to know the user's operational intentions in a specific scenario in advance, avoiding the tedious search and operation of the user in the complex vehicle system interface.

[0052] Furthermore, when the vehicle's spatiotemporal information meets the spatiotemporal conditions, the target interface elements can be displayed, reducing user waiting time and improving operational efficiency.

[0053] Furthermore, the attribute information of functional interface elements is transformed into the structural parameters and action logic of a 3D model. By leveraging text-to-3D and image-to-3D generation technologies, these elements are endowed with richer visual effects and interactive dynamics. For example, when a user clicks the "Schedule and Meeting Application Function Entry" icon, the icon produces a slight scaling and rotation animation, providing intuitive operational feedback. This three-dimensional and intuitive presentation not only enhances the user experience of the in-vehicle system but also makes the interactive operation more intuitive, significantly improving the intelligence of the in-vehicle system interface and the convenience of user operation.

[0054] The primary attribute information of functional interface elements specifically includes: interaction response logic, visual display requirements, size specifications, and hierarchical relationships. Interaction response logic clarifies the feedback pattern after a user performs a triggered action on the element, such as expanding a submenu, navigating to a new page, or executing a specific function after clicking the element. Visual display requirements determine the appearance of the 3D model corresponding to the target 3D data, including color, material, and lighting effects. Size specifications define the size and proportion of the 3D model in 3D space. Hierarchical relationships clarify the hierarchy and relationships between functional interface elements and other interface elements. These various attribute information complement each other, collectively forming the key basis for generating the target 3D data.

[0055] In the context of 3D data generation, the main structure, as a key component, forms the basic architecture of the target 3D data. This main structure can include core elements such as the geometry of the 3D model, vertex positions, and edge-face connections, determining the approximate outline and basic form of the 3D model.

[0056] Regarding the relationship between interactive response logic and 3D data, there are two implementation methods in the process of generating 3D data.

[0057] In one implementation, the interactive response logic can be directly embedded into the main structure of the 3D data, becoming an indispensable part of the target 3D data. In this way, the interactive response logic contained in the 3D model can be directly called during the rendering process.

[0058] Another approach is to bind the interactive response logic to the 3D model during the rendering process. Binding the interactive response logic to the rendering process allows for flexible adjustments to the interactive logic without altering the main structure of the 3D data, facilitating subsequent feature updates and optimizations.

[0059] The aforementioned first attribute information may also include the presentation position of the functional interface element. During the rendering process, it can be placed at the corresponding position on the 3D HMI interface based on the aforementioned presentation position. For example, the application's function entry point can be displayed at a designated screen position. Specifically, the designated screen position could be the lower left corner of the screen. Considering that most drivers frequently operate the steering wheel or gearshift lever with their right hand when using the in-vehicle system, and their left hand moves less, the lower left area is easier to reach. Placing the function entry point here facilitates quick operation for the driver. Alternatively, the designated screen position could be slightly below the center of the screen. This position is below the driver's line of sight, so it won't obstruct important driving information.

[0060] This application does not limit the layout of the 3D HMI interface. Those skilled in the art can arrange the interface elements, such as vehicle environment interface elements, vehicle interface elements, and functional interface elements, according to actual application needs. The vehicle environment interface elements are used to present the surrounding environment of the vehicle, the vehicle interface elements are used to present the 3D model of the vehicle, and the functional interface elements are used to present function access points.

[0061] In one example, the 3D HMI interface adopts a spatial hierarchy layout of far-field, mid-field, and close-up views. The far-field layer's environment presentation area presents simplified vehicle environment elements as a background; the mid-field layer's core visual area displays the core 3D vehicle model in the center and provides real-time status feedback; the close-up layer's function presentation area overlays functional interface elements such as vehicle speed, navigation prompts, and function entrances corresponding to interactive needs in the form of floating cards in front.

[0062] In the core visual area, the 3D model of the vehicle is located in the center of the screen. The 3D model of the vehicle can reflect the real status of the vehicle in real time (such as the opening status of the doors and windows, and the flashing of the turn signals).

[0063] In the environment rendering area, vehicle environment interface elements (such as roads, traffic signs, and environmental objects) serve as background extensions, providing context for the vehicle's 3D model, but are appropriately blurred and simplified to avoid interference. In the functional presentation area, functional interface elements (such as vehicle speed, gear, range cards, or function entrances for office applications) are displayed as floating cards around the vehicle's 3D model.

[0064] Technical Solution 3 In technical solution 3, the vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle interior and exterior environment information; the vehicle interior and exterior environment information includes: vehicle environment image; The process of determining the target interface element corresponding to the vehicle scene information specifically includes: determining the vehicle environment interface element as the target interface element corresponding to the vehicle environment image. The process of generating target 3D data corresponding to the target interface element based on the vehicle scene information specifically includes: determining the theme corresponding to the vehicle environment image; generating descriptive information of the vehicle environment interface element based on the theme corresponding to the vehicle environment image; and generating vehicle environment 3D data based on the descriptive information of the vehicle environment interface element.

[0065] Vehicle environment interface elements are key carriers connecting the physical world and the virtual interactive interface. Their core function is to transform the real environmental information perceived by the vehicle (such as themes, objects, and atmosphere) into an intuitive and immersive visual presentation inside the vehicle through three-dimensional visualization, thereby enhancing the driver's perception of the environment and improving the interactive experience.

[0066] Technical solution 3 first determines the theme corresponding to the vehicle environment image, then generates descriptive information for the vehicle environment interface elements based on the identified theme, and then uses 3D generation technology to generate 3D data of the vehicle environment based on the descriptive information, thereby constructing a 3D scene adapted to the theme of the vehicle environment image. This embodiment of the application can transform real-world vehicle environment images into themed 3D scenes, significantly enhancing the immersive experience of the interface and providing a visual foundation for contextual awareness and personalized interaction.

[0067] In this embodiment, the corresponding scene theme is determined by recognizing and analyzing vehicle environment images, or by combining vehicle location information with environmental images. Specifically: Image recognition scenario: Using computer vision algorithms to perform semantic segmentation, object detection, and scene understanding on vehicle environment images. For example, identifying elements such as forests, campfires, and tents in the image determines it as an "outdoor camping" theme; if features such as glass curtain walls and office building outlines are detected, it is determined to be an "office" theme.

[0068] Image and location information fusion scenario: When vehicle location information (e.g., obtained via GPS) indicates "forest park campsite," combining visual features such as trees and camping facilities in the vehicle's environmental image strengthens the determination of the "outdoor camping" theme. Alternatively, if the vehicle location information indicates an "office building" area, matching elements such as office buildings and parking lots in the vehicle's environmental image accurately identifies the "office" theme. This mechanism improves the accuracy and robustness of theme recognition through multi-source data fusion, providing a more precise semantic foundation for subsequent 3D scene generation.

[0069] In one example, when the vehicle enters the "Forest Park Campsite" area, the in-vehicle human-machine interaction system will call 3D generation technology to generate a 3D scene that highly matches the "outdoor camping" theme, such as a campfire campsite under a tranquil starry sky, or a virtual forest dotted with fireflies.

[0070] In another example, when the vehicle enters the "office building" area, the in-vehicle human-machine interaction system uses 3D generation technology to combine environmental image features of the office scene (such as glass curtain walls, parking lot signs, building outlines, etc.) to generate a 3D scene that matches the "office" theme. For example, it presents a virtual lobby of a modern office building, displaying interface elements such as elevator floor guidance and company name signs.

[0071] It should be noted that technical solutions 2 and 3 in this application embodiment can be used in combination. Specifically, when the vehicle's spatiotemporal information conforms to the spatiotemporal conditions corresponding to the user interaction pattern, the interaction requirements are mapped to functional interface elements in the 3D HMI interface, and the functional interface elements are displayed in the 3D scene corresponding to the vehicle environment 3D data.

[0072] For example, when a vehicle enters an office building area, the in-vehicle human-machine interaction system can identify the "office" theme based on environmental images and automatically call 3D generation technology to construct a matching virtual office scene (such as presenting a 3D environment with glass curtain walls and building outlines). At the same time, functional interface elements such as "schedule and meeting applications" are integrated into this virtual office scene to achieve intelligent linkage between the in-vehicle interface and the office scene, thereby improving the immersiveness and efficiency of the scene-based interaction.

[0073] Technical Solution 4 In technical solution 4, the vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle interior and exterior environment information; the vehicle interior and exterior environment information includes: vehicle environment image; The process of determining the target interface element corresponding to the vehicle scene information specifically includes: determining the target interface element corresponding to the vehicle environment image as an environment object interface element based on the environment objects in the vehicle environment image; the environment object interface element corresponds to second attribute information; The process of generating target 3D data corresponding to the target interface element based on the vehicle scene information specifically includes: generating target 3D data corresponding to the environmental object interface element based on the second attribute information corresponding to the environmental object interface element.

[0074] Technical solution 4 is based on the analysis of in-vehicle environment images. The analysis results include environmental objects in the physical environment, and the environmental objects in the physical environment are mapped to vehicle environment interface elements in the 3D HMI interface.

[0075] The aforementioned environmental objects may be dynamic entities that temporarily appear in the physical environment and affect driving safety, such as temporary roadside stalls. By mapping these objects from the physical environment to vehicle environment interface elements in a 3D HMI interface, and using interface elements such as visual highlighting or tactile warnings, environmental threats can be transformed into intuitive interactive prompts, thereby improving situational awareness and driving safety.

[0076] The aforementioned second attribute information includes: a warning attribute, which causes the environmental object interface elements to display a warning effect.

[0077] In specific implementations, the warning attributes include one or more of the following attributes: The highlight attribute is used to indicate that the display brightness of the target interface element is higher than a set threshold. The pulsation attribute is used to characterize the periodic changes in the brightness or size parameters of the target interface element; The color blinking attribute is used to characterize the periodic switching of the color or brightness of the target interface element between two different values.

[0078] The highlight attribute increases the display brightness of target interface elements to a level significantly higher than the normal environmental value (set threshold), making them stand out against the visual background and thus attracting the driver's attention, achieving a warning effect.

[0079] The pulsation attribute creates a rhythmic visual signal similar to a biological heartbeat by causing the brightness or size parameters (such as size) of the target interface elements to change in a regular cycle. This dynamic change continuously guides the driver's attention and achieves a warning effect.

[0080] The color flashing attribute creates a strong visual flashing effect by rapidly and periodically switching the color or brightness of a target interface element between two contrasting values ​​(such as red and black, highlight and constant). This conveys urgent information with the highest priority and achieves a warning effect.

[0081] In this embodiment, the second attribute can be mapped to generation parameters. After receiving these generation parameters, the 3D generation model parses them and generates target 3D data corresponding to the interface elements of the environment objects based on the corresponding parsing results.

[0082] Among them, the generated parameters are the specific technical means to implement the warning attributes, and there is a clear mapping relationship between them: The generation parameters corresponding to the highlight attribute can be material optical parameters, such as luminous intensity parameters. By significantly increasing the brightness of the target's 3D data, environmental objects and interface elements stand out in the visual environment.

[0083] The generation parameters corresponding to the pulsation attribute can be dynamically changing parameters. Dynamically changing parameters define the frequency of change (e.g., 2Hz, which means flashing twice per second) and the amplitude curve (e.g., a sine curve, which controls the brightness or size to fluctuate smoothly and periodically).

[0084] The generation parameters for the color blinking attribute can include: material optical parameters (defining two basic color values ​​that alternate, such as red and black) and dynamic change parameters (defining the frequency of color switching).

[0085] The generation parameters determine the initial state and properties of the target 3D data for the 3D generated model: Material optical parameters determine the underlying texture map and material visual properties of the generated data. For example, a higher "luminescence intensity" parameter will generate a texture with high brightness and self-illumination. The dynamically changing parameters determine whether the generated data contains a predefined animation sequence. For example, specific "frequency" and "amplitude curve" parameters will generate an animated 3D mesh with vertex positions deformed according to a predetermined pattern.

[0086] In addition to the warning attribute, the second attribute can also include basic visual attributes such as geometric attributes and material attributes. Among them, geometric attributes are used to define the shape and spatial structure of the target interface element, while material attributes are used to define the visual texture and optical properties of the target interface element's surface.

[0087] For example, for a temporary roadblock that cannot be categorized, its geometric properties can be defined as a cube to quickly match the occupied volume of unknown obstacles; its material properties can be defined as a semi-transparent gray material to provide a clear outline cues while avoiding completely obscuring the real environment.

[0088] This application embodiment can combine the aforementioned geometric and material properties (to generate a semi-transparent cube) and further overlay warning properties (such as color flashing) to create a corresponding semi-transparent 3D block model with a warning flashing effect from the 3D generation model. This 3D block model is overlaid and displayed with other interface elements in the 3D HMI interface to achieve a universal and highly perceptible reminder to the driver.

[0089] Technical Solution 5 In technical solution 5, the process of determining the target interface element corresponding to the vehicle scene information specifically includes: when the vehicle scene information meets the preset scenario conditions, determining the target interface element corresponding to the vehicle scene information as a scenario interface element; the scenario interface element corresponds to descriptive information. The process of generating target 3D data corresponding to the target interface element based on the vehicle scene information specifically includes: generating target 3D data corresponding to the context interface element based on the description information corresponding to the context interface element. The target 3D data corresponding to the context interface element may include: 3D special effects data or character animation data.

[0090] Special effects can refer to special effects, which are usually created by computer software and would not normally occur in reality.

[0091] The 3D special effects data in this application embodiment corresponds to particle effects. Particle effects are visual effects commonly used to simulate natural phenomena, such as flames, smoke, explosions, rain, and snow. They consist of a large number of tiny particles, and complex visual effects are achieved by controlling the behavior, color, size, speed, and other attributes of these particles. Of course, this application embodiment does not limit the specific type of special effects.

[0092] 3D effects data can be a collection of data created by a 3D generative model for rendering particle systems. For example, to create a festive atmosphere, a 3D generative model can generate a data package containing parameters such as the initial position, velocity, lifespan, and color change trajectory of particles. After being parsed by the rendering engine, it can render the corresponding fireworks or snow effects.

[0093] Character animation data specifically refers to the data set used to drive a single character model (such as a human or animal) to perform actions. For example, generating animation data for a virtual assistant waving and greeting requires including the hierarchical relationship of bones, weight mapping, and transformation information of each bone in the animation sequence.

[0094] Particle effects data controls the statistical behavior of a group (such as a snowflake), while character animation data controls the precise movements of an individual (such as a butterfly).

[0095] In one exemplary implementation, when a soothing type of music is detected as currently playing, the generation and display of character animation data can be triggered. Specifically, in the corner of the 3D HMI interface, a butterfly model driven by character animation data will quietly appear above a virtual plant, flying naturally. The butterfly displays realistic wing flapping and a natural flight trajectory, and its timing and rhythm of movement match the musical atmosphere, aiming to create a natural and immersive surprise experience for the user.

[0096] Technical solution 5 intelligently identifies in-vehicle scene information by pre-setting contextual conditions. When the in-vehicle scene information matches a specific context (such as "birthday" or "anniversary"), it automatically triggers the generation process of contextual interface elements: the descriptive information corresponding to the contextual interface element (such as "romantic fireworks celebration") is converted into generation effect parameters, driving the 3D generation model to create 3D effect data (such as particle fireworks animation) that matches the effect parameters, and finally rendering immersive contextual effects in the cabin, thereby transforming the vehicle into an emotional space that is deeply integrated with the scene, significantly enhancing the fun and emotional resonance of the driving experience.

[0097] In a specific implementation, the in-vehicle scenario information includes: in-vehicle time information, music playback information, and user emotion information corresponding to user behavior information; wherein, the above-mentioned in-vehicle scenario information conforms to preset context conditions, specifically including: The in-vehicle scene information includes in-vehicle time information that meets preset holiday conditions; or The in-vehicle scene information includes music playback information that meets preset music conditions; or The user's emotional information corresponding to the user's behavior information meets the preset emotional conditions.

[0098] For preset holiday conditions, real-time in-vehicle time information (such as date) can be matched with holiday time information (such as the time information of Chinese New Year and Christmas) in a preset holiday database. When the in-vehicle time information matches the time information of a specific holiday, it can be considered to meet the preset holiday conditions. In this case, 3D special effects that match the holiday atmosphere (such as virtual lanterns for Chinese New Year and snow effects for Christmas) are automatically generated, so that the in-vehicle interface changes accordingly, creating a strong sense of holiday ritual and immersive experience.

[0099] For preset music conditions, the metadata (such as genre and rhythm) or audio characteristics of the currently playing music are analyzed to identify whether it belongs to a predefined category (such as "classical music" or "rock music") or a specific playlist (such as "driving focus"), thereby triggering the corresponding preset music conditions.

[0100] In this case, visual effects that are dynamically coordinated with the music style can be generated (such as generating soothing light particles when playing classical music and generating vibrant spectrum particles when playing rock music), achieving audio-visual synergy and enhancing the entertainment experience.

[0101] In one example, when the currently playing music is soothing, a butterfly can be quietly generated on a virtual plant in the corner of the screen, creating an unexpected surprise for the user.

[0102] For preset emotional conditions, user emotional state information (such as "pleasure" or "fatigue") is identified by analyzing user behavior information (such as voice tone and facial expressions). This is triggered when the user's emotional state information matches a preset emotional condition (such as "needs a boost"). Preset emotional conditions can generate 3D effects designed to regulate or respond to user emotions (such as generating an invigorating sunrise effect when fatigue is detected), enabling the 3D HMI interface to possess emotional perception and feedback capabilities, providing more caring and personalized interactions.

[0103] This application embodiment describes in detail the process of determining the target interface element corresponding to the vehicle scene information and generating the target three-dimensional data corresponding to the target interface element based on the vehicle scene information through technical solutions 1 to 5. It can be understood that those skilled in the art can adopt any or a combination of technical solutions 1 to 5 according to actual application needs, or they can adopt other technical solutions according to actual application needs.

[0104] For example, in another technical solution, the user behavior information includes: an input image of a three-dimensional vehicle model by the user; then, in this embodiment of the application, the vehicle interface element can be determined as the target interface element corresponding to the input image; and based on the input image, vehicle three-dimensional model data with skeletal binding and animation can be generated as the target three-dimensional data corresponding to the vehicle interface element.

[0105] The input image may originate from photographs of actual vehicles or from meticulously designed vehicle blueprints. This embodiment analyzes the input image, obtaining vehicle information such as shape and structure. Subsequently, based on this vehicle information, a 3D vehicle model with skeletal rigging and animation is generated. The generated 3D vehicle model visually matches the vehicle presented in the input image. For example, if a user inputs a photo of a typical family sedan, showing features such as blue paint, square headlights, and a conventional three-box structure, the generated 3D vehicle model will have a paint color that closely matches the blue in the photo, accurately reproduce the size and shape of the square headlights, and maintain the same proportions as the vehicle in the photo. From all angles, it will highly match the input image.

[0106] Furthermore, thanks to skeletal binding and animation effects, the generated 3D vehicle model conforms to the expected interaction logic based on the input image when performing actions such as opening and closing doors and turning on headlights, bringing users a highly personalized and realistic in-vehicle interaction experience.

[0107] In step 103, the target 3D data and the preset 3D scene data can be fused first to obtain fused 3D data; then, the fused 3D data can be rendered to obtain the 3D HMI interface corresponding to the target interface element.

[0108] The process of fusing the target 3D data with the preset 3D scene data specifically includes: spatially aligning and unifying the target 3D data (such as 3D data of environmental objects) with the preset 3D scene data (such as data of a 3D vehicle model), so that the position, scale and orientation of the target 3D data and the preset 3D scene data are unified.

[0109] Pre-defined 3D scene data refers to relatively fixed 3D scene data that is generated and stored in advance. For example, pre-defined 3D scene data may include: vehicle 3D model data used as a reference, or vehicle environment 3D data pre-constructed based on map data, etc.

[0110] The rendering engine (such as an automotive graphics engine) performs graphics pipeline calculations (vertex transformation, lighting, shading, etc.) on the fused 3D data, generates 2D pixel frames of all interface elements, and synthesizes the 2D pixel frames of all interface elements based on depth information, ultimately generating a 3D HMI interface adapted to the automotive scene information.

[0111] In step 104, the image frames corresponding to the 3D HMI interface output by the rendering engine can be sent to the display screen, and the display screen displays the image frames corresponding to the 3D HMI interface.

[0112] In its implementation, after the rendering engine completes the drawing and compositing of all 3D elements, it submits the generated image frames to the system frame buffer. The display controller reads the image frames from the frame buffer according to the screen refresh rate and transmits them to the display screen through the vehicle display interface. The display controller follows the timing sequence of the vertical synchronization signal to control the display screen to scan the transmitted image frame data line by line. Finally, the drive circuit activates the pixel units of the display screen to emit light, achieving a stable, low-latency dynamic display of the 3D HMI interface.

[0113] In summary, this application embodiment realizes the transformation of interface content from preset to generated. Therefore, this application embodiment can transform the 3D HMI interface from a static and uniform display into an intelligent interface that can be dynamically adjusted according to the vehicle environment or user behavior reflected by in-vehicle scene information. This application embodiment can automatically adapt the display mode, layout, or interaction priority of interface elements in the 3D HMI interface based on the vehicle environment or user behavior reflected by in-vehicle scene information, thereby providing flexible and accurate interaction support in diverse driving scenarios, effectively improving the adaptability and operational efficiency of human-vehicle interaction, and enhancing driving safety.

[0114] First, the embodiments of this application realize the personalization and customization of VPA image and scene theme: users are no longer limited to preset options, and can generate unique VPA image and scene theme in real time according to their own preferences, which improves the user experience of smart cockpit.

[0115] Secondly, a scene-adaptive dynamic interactive interface has been created: it can dynamically generate the most suitable three-dimensional interactive scene based on the real physical environment of the vehicle and user habits, thereby improving the accuracy and efficiency of functional interaction.

[0116] Furthermore, it enhances the perception and interaction dimensions of intelligent driving: through real-time generative representation of unknown or unmodelable environmental objects, it enhances the visualization prompts for unknown objects, providing drivers with richer and more intuitive environmental information, thereby improving driving safety and trust.

[0117] In addition, it creates a warm and emotional experience: by introducing dynamically generated, unexpected contextual interface elements, it enhances the emotional resonance of human-computer interaction.

[0118] Furthermore, it expands the boundaries of the smart cockpit experience: the embodiments of this application transform the three-dimensional HMI interface from a fixed information display window into a dynamically generated interactive space, providing new possibilities for the evolution of future smart cockpits.

[0119] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0120] It should be noted that the vehicle-mounted human-machine interaction method provided in this application embodiment can be executed by a vehicle-mounted human-machine interaction system. Device Or, or the in-vehicle human-machine interaction Device The control module in this application is used to execute the loading method for in-vehicle human-machine interaction. In this embodiment, in-vehicle human-machine interaction is used. Device Taking the execution of the in-vehicle human-machine interaction method as an example, the in-vehicle human-machine interaction method provided in this application embodiment is explained.

[0121] Reference Figure 2 The diagram illustrates the structure of an in-vehicle human-machine interaction device according to an embodiment of this application. The device specifically includes the following modules: Data acquisition module 201 is used to collect in-vehicle scene information; the in-vehicle scene information includes: vehicle environment information and / or user behavior information; The interface element determination module 202 is used to determine the target interface element corresponding to the vehicle scene information; The 3D data generation module 203 is used to generate target 3D data corresponding to the target interface element based on the vehicle scene information; the target interface element is an existing interface element, or the target interface element is a newly added interface element. Rendering module 204 is used to render the target 3D data to obtain the 3D HMI interface corresponding to the target interface elements; The display module 205 is used to display the three-dimensional HMI interface.

[0122] Optionally, the interface element determination module 202 includes: The analysis module is used to analyze in-vehicle scene information and obtain corresponding analysis results; The mapping module is used to map the interaction requirements to target interface elements in the 3D HMI interface when the analysis results include interaction requirements for in-vehicle scene information; or, when the analysis results include environmental objects in the physical environment, to map environmental objects in the physical environment to target interface elements in the 3D HMI interface.

[0123] Optionally, the in-vehicle scenario information includes: user behavior information; the user behavior information includes: user input information for the VPA virtual personal assistant; The interface element determination module 202 includes: The first interface element determination module is used to determine the VPA interface element as the target interface element corresponding to the input information. The three-dimensional data generation module 203 includes: The first 3D data generation module is used to generate 3D model data of VPA with skeleton rigging and animation based on the user's input information for VPA, as the target 3D data corresponding to the VPA interface elements.

[0124] Optionally, the vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle spatiotemporal information; The interface element determination module 202 includes: The second interface element determination module is used to determine the functional interface element corresponding to the vehicle spatiotemporal information based on the historical user behavior data corresponding to the vehicle spatiotemporal information, and use it as the target interface element corresponding to the vehicle spatiotemporal information; the functional interface element corresponds to first attribute information; the first attribute information includes: interaction response logic, which is used to respond to the user's trigger operation on the functional interface element and display the functional content corresponding to the functional interface element. The three-dimensional data generation module 203 includes: The second 3D data generation module is used to generate target 3D data corresponding to the functional interface element based on the first attribute information of the functional interface element.

[0125] Optionally, the vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle interior and exterior environment information; the vehicle interior and exterior environment information includes: vehicle environment image; The interface element determination module 202 includes: The third interface element determination module is used to determine the vehicle environment interface elements as the target interface elements corresponding to the vehicle environment image. The three-dimensional data generation module 202 includes: The theme determination module is used to determine the theme corresponding to the vehicle environment image; The description generation module is used to generate description information of vehicle environment interface elements based on the theme corresponding to the vehicle environment image. The third 3D data generation module is used to generate 3D data of the vehicle environment based on the description information of the vehicle environment interface elements.

[0126] Optionally, the vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle interior and exterior environment information; the vehicle interior and exterior environment information includes: vehicle environment image; The interface element determination module 202 includes: The fourth interface element determination module is used to determine the target interface element corresponding to the vehicle environment image as an environment object interface element based on the environment objects in the vehicle environment image; the environment object interface element has corresponding second attribute information. The three-dimensional data generation module 202 includes: The fourth 3D data generation module is used to generate target 3D data corresponding to the environmental object interface element based on the second attribute information corresponding to the environmental object interface element.

[0127] Optionally, the second attribute information includes: a warning attribute, so that the environmental object interface elements present a warning effect; The warning attributes include one or more of the following attributes: The highlight attribute is used to indicate that the display brightness of the target interface element is higher than a set threshold. The pulsation attribute is used to characterize the periodic changes in the brightness or size parameters of the target interface element; The color blinking attribute is used to characterize the periodic switching of the color or brightness of the target interface element between two different values.

[0128] Optionally, the interface element determination module 202 includes: The fifth interface element determination module is used to determine the target interface element corresponding to the vehicle scene information as a context interface element when the vehicle scene information meets the preset context conditions; the context interface element has corresponding descriptive information. The three-dimensional data generation module 202 includes: The fifth 3D data generation module is used to generate target 3D data corresponding to the context interface element based on the description information corresponding to the context interface element.

[0129] Optionally, the in-vehicle scenario information includes: in-vehicle time information, music playback information, and user emotion information corresponding to user behavior information; wherein, the in-vehicle scenario information conforms to preset scenario conditions, including: The in-vehicle scene information includes in-vehicle time information that meets preset holiday conditions; or The in-vehicle scene information includes music playback information that meets preset music conditions; or The user's emotional information corresponding to the user's behavior information meets the preset emotional conditions.

[0130] Optionally, the rendering module includes: The fusion module is used to spatially align and coordinate system unify the target 3D data with the preset 3D scene data to obtain fused 3D data. The interface generation module is used to render the fused 3D data to obtain the 3D HMI interface corresponding to the target interface element.

[0131] In-vehicle human-machine interaction in the embodiments of this application DeviceIt can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. The embodiments in this application do not impose specific limitations.

[0132] The in-vehicle human-machine interaction device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0133] The vehicle-mounted human-machine interaction device provided in this application embodiment can achieve... Figure 1 The various processes implemented by the in-vehicle human-machine interaction device in the method embodiment will not be described again here to avoid repetition.

[0134] Optionally, this application also provides an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described vehicle-mounted human-machine interaction method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0135] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0136] Figure 3 A schematic diagram of the hardware structure of an electronic device 1100 according to an embodiment of this application.

[0137] Electronic device 1100 has one or more processors 1102, a control module (chipset) 1104 coupled to at least one of the processors 1102, a memory 1106 coupled to the control module 1104, a non-volatile memory / storage device 1108 coupled to the control module 1104, one or more input / output devices 1110 coupled to the control module 1104, and a network interface 1112 coupled to the control module 1104.

[0138] Processor 1102 may include one or more single-core or multi-core processors, and processor 1102 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, electronic device 1100 can serve as a terminal device, server (cluster), or other device as described in the embodiments of this application.

[0139] In some embodiments, electronic device 1100 may include one or more computer-readable media (e.g., memory 1106 or non-volatile memory / storage device 1108) having instructions 1114 and one or more processors 1102 that are combined with the one or more computer-readable media and configured to execute instructions 1114 to implement modules and thus perform the actions described in this disclosure.

[0140] In one embodiment, the control module 1104 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1102 and / or any suitable device or component communicating with the control module 1104.

[0141] The control module 1104 may include a memory controller module to provide an interface to the memory 1106. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0142] Memory 1106 may be used, for example, to load and store data and / or instructions 1114 for electronic device 1100. In one embodiment, memory 1106 may include any suitable volatile memory, such as suitable DRAM (Dynamic Random Access Memory). In some embodiments, memory 1106 may include double data rate type quad synchronous dynamic random access memory.

[0143] In one embodiment, the control module 1104 may include one or more input / output controllers to provide an interface to the non-volatile memory / storage device 1108 and (one or more) input / output devices 1110.

[0144] For example, non-volatile memory / storage device 1108 may be used to store data and / or instructions 1114. Non-volatile memory / storage device 1108 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives, one or more optical disk drives, and / or one or more digital universal optical disk drives).

[0145] The non-volatile memory / storage device 1108 may include storage resources that are physically part of a device on which the electronic device 1100 is mounted, or that can be accessed by the device without being part of the device. For example, the non-volatile memory / storage device 1108 may be accessed via a network via one or more input / output devices 1110.

[0146] One or more input / output devices 1110 may provide an interface for electronic device 1100 to communicate with any other suitable device. Input / output devices 1110 may include communication components, audio components, sensor components, etc. Network interface 1112 may provide an interface for electronic device 1100 to communicate via one or more networks. Electronic device 1100 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing wireless networks based on communication standards, such as WiFi (Wireless Fidelity), 2G (2-Generation wireless telephone technology), 3G (3-Generation wireless telephone technology), 4G (4-Generation wireless telephone technology), 5G (5-Generation wireless telephone technology), etc., or combinations thereof.

[0147] In one embodiment, at least one of the processors 1102 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 1104. In one embodiment, at least one of the processors 1102 may be logically packaged with one or more controllers of the control module 1104 to form a system-in-package. In one embodiment, at least one of the processors 1102 may be integrated with the logic of one or more controllers of the control module 1104 on the same die. In one embodiment, at least one of the processors 1102 may be integrated with the logic of one or more controllers of the control module 1104 on the same die to form a system-on-a-chip.

[0148] In various embodiments, electronic device 1100 may be, but is not limited to, a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, touchscreen device, netbook, etc.). In various embodiments, electronic device 1100 may have more or fewer components and / or different architectures. For example, in some embodiments, electronic device 1100 includes one or more cameras, a keyboard, a liquid crystal display screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0149] In the aforementioned electronic device 1100, a main control chip may be used as a processor or control module, sensor data, position information, etc. are stored in a memory or non-volatile memory / storage device, the sensor group may be used as an input / output device, and the communication interface may include a network interface.

[0150] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described vehicle-mounted human-machine interaction method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0151] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0152] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0154] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A vehicle-mounted human-computer interaction method, characterized in that, The method includes: Collect in-vehicle scene information; the in-vehicle scene information includes: vehicle environment information and / or user behavior information; The target interface element corresponding to the vehicle scene information is determined, and the target three-dimensional data corresponding to the target interface element is generated based on the vehicle scene information; the target interface element is an existing interface element, or the target interface element is a newly added interface element. The target 3D data is rendered to obtain the 3D HMI interface corresponding to the target interface element; The three-dimensional HMI interface is then displayed.

2. The method according to claim 1, characterized in that, Determining the target interface element corresponding to the vehicle scene information includes: The in-vehicle scene information is analyzed to obtain the corresponding analysis results; If the analysis results include interactive requirements for in-vehicle scene information, the interactive requirements are mapped to target interface elements in the 3D HMI interface; or, if the analysis results include environmental objects in the physical environment, the environmental objects in the physical environment are mapped to target interface elements in the 3D HMI interface.

3. The method according to claim 1, characterized in that, The in-vehicle scenario information includes: user behavior information; the user behavior information includes: user input information for the VPA virtual personal assistant; Determining the target interface element corresponding to the vehicle scene information includes: The VPA interface element is determined as the target interface element corresponding to the input information; The step of generating target 3D data corresponding to the target interface element based on the vehicle scene information includes: Based on the user's input information for VPA, VPA 3D model data with skeletal rigging and animation is generated, which serves as the target 3D data corresponding to the VPA interface elements.

4. The method according to any one of claims 1-3, characterized in that, The in-vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle spatiotemporal information. Determining the target interface element corresponding to the vehicle scene information includes: Based on the historical user behavior data corresponding to the vehicle spatiotemporal information, the functional interface elements corresponding to the vehicle spatiotemporal information are determined as the target interface elements corresponding to the vehicle spatiotemporal information; the functional interface elements correspond to first attribute information; the first attribute information includes: interactive response logic, which is used to respond to the user's trigger operation on the functional interface elements and display the functional content corresponding to the functional interface elements. The step of generating target 3D data corresponding to the target interface element based on the vehicle scene information includes: Based on the first attribute information of the functional interface element, the target three-dimensional data corresponding to the functional interface element is generated.

5. The method according to any one of claims 1-3, characterized in that, The vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle interior and exterior environment information; the vehicle interior and exterior environment information includes: vehicle environment image; Determining the target interface element corresponding to the vehicle scene information includes: The vehicle environment interface elements are defined as the target interface elements corresponding to the vehicle environment image. The step of generating target 3D data corresponding to the target interface element based on the vehicle scene information includes: Determine the subject corresponding to the vehicle environment image; Based on the theme corresponding to the vehicle environment image, generate descriptive information for the vehicle environment interface elements; Based on the description information of the vehicle environment interface elements, generate three-dimensional data of the vehicle environment.

6. The method according to any one of claims 1-3, characterized in that, The vehicle scene information includes: vehicle environment information; the vehicle environment information includes: vehicle interior and exterior environment information; the vehicle interior and exterior environment information includes: vehicle environment image; Determining the target interface element corresponding to the vehicle scene information includes: Based on the environmental objects in the vehicle environment image, the target interface element corresponding to the vehicle environment image is determined as the environmental object interface element; the environmental object interface element has corresponding second attribute information. The step of generating target 3D data corresponding to the target interface element based on the vehicle scene information includes: Based on the second attribute information corresponding to the environmental object interface element, the target three-dimensional data corresponding to the environmental object interface element is generated.

7. The method according to claim 6, characterized in that, The second attribute information includes: a warning attribute, which causes the environmental object interface element to display a warning effect; wherein, the warning attribute includes one or more of the following attributes: The highlight attribute is used to indicate that the display brightness of the target interface element is higher than a set threshold. The pulsation attribute is used to characterize the periodic changes in the brightness or size parameters of the target interface element; The color blinking attribute is used to characterize the periodic switching of the color or brightness of the target interface element between two different values.

8. The method according to any one of claims 1-3, characterized in that, Determining the target interface element corresponding to the vehicle scene information includes: When the vehicle scene information meets the preset scenario conditions, the target interface element corresponding to the vehicle scene information is determined as the scenario interface element; the scenario interface element has corresponding descriptive information. The step of generating target 3D data corresponding to the target interface element based on the vehicle scene information includes: Based on the description information corresponding to the context interface element, generate the target 3D data corresponding to the context interface element.

9. The method according to claim 8, characterized in that, The in-vehicle scenario information includes: in-vehicle time information, music playback information, and user emotion information corresponding to user behavior information; wherein, the in-vehicle scenario information conforms to preset scenario conditions, including: The in-vehicle scene information includes in-vehicle time information that meets preset holiday conditions; or The in-vehicle scene information includes music playback information that meets preset music conditions; or The user's emotional information corresponding to the user's behavior information meets the preset emotional conditions.

10. The method according to any one of claims 1 to 3, characterized in that, The rendering of the target 3D data includes: The target 3D data is spatially aligned and coordinate system unified with the preset 3D scene data to obtain fused 3D data. The fused 3D data is rendered to obtain the 3D HMI interface corresponding to the target interface element.

11. A vehicle-mounted human-machine interaction device, characterized in that, The device includes: The data acquisition module is used to collect in-vehicle scene information; the in-vehicle scene information includes: vehicle environment information and / or user behavior information; The interface element determination module is used to determine the target interface element corresponding to the vehicle scene information; A 3D data generation module is used to generate target 3D data corresponding to the target interface element based on the vehicle scene information; the target interface element is an existing interface element, or the target interface element is a newly added interface element. The rendering module is used to render the target 3D data to obtain the 3D HMI interface corresponding to the target interface elements; The display module is used to display the three-dimensional HMI interface.

12. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in claims 1-10.

13. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in claims 1-10.

Images