Cabin 3D-HMI multi-screen collaborative interaction method, system, medium and equipment

Abstract

The invention relates to a cockpit 3D-HMI multi-screen collaborative interaction method, system, medium and equipment, and relates to the technical field of automobile man-machine interaction. The method comprises the following steps: constructing a 3D-HMI dynamic scene map based on a preset resource library and a vehicle real-time state; based on the 3D-HMI dynamic scene map, performing differential rendering on a plurality of display terminals at different positions in the vehicle cabin to obtain display content of each display terminal; in response to a user interaction instruction received on any display terminal, updating the 3D-HMI dynamic scene map; and synchronously updating the display content on the corresponding display terminal based on the updated 3D-HMI dynamic scene map. Compared with the prior art, the immersion feeling of all passengers in the cabin is greatly enhanced, and the operation intuition and safety are improved; the space in the vehicle is converted into a continuous entertainment and information platform through the multi-screen collaborative 3D HMI, a bottom layer interaction framework is provided for future vehicle-mounted application, and high expandability is achieved.

Description

Technical Field

[0001] This application relates to the field of automotive human-machine interaction technology, and in particular to a cockpit 3D-HMI multi-screen collaborative interaction method, system, medium and device. Background Technology

[0002] With the development of smart cockpit technology, modern vehicles are typically equipped with multiple display screens, such as head-up displays, central control screens, and rear entertainment screens. Currently, the content displayed and interactive functions of these screens are often independent, or only involve simple content mapping, resulting in the following drawbacks: The different 2D interfaces displayed on each screen have inconsistent styles and interaction logic, failing to create an immersive and coherent experience for drivers and passengers.

[0003] Each screen renders its content independently, increasing the system's computing power overhead, which is particularly noticeable on hardware platforms with limited computing power, such as the 8797 chip.

[0004] It failed to fully utilize the "same three-dimensional scene" as a unified interactive space to achieve a collaborative interactive experience between front and back rows and between active and passive participants.

[0005] Current multi-screen cockpits are essentially "physical stacking of multiple smart terminals in one space," rather than "a seamless, integrated intelligent entity." This not only limits the enhancement of user experience but also brings high hardware costs and rigid product definitions to automakers. Therefore, a new technological paradigm is urgently needed, capable of achieving distributed display with a unified 3D scene at its core, enabling one-time rendering and multi-device adaptation, and supporting natural interaction between front and rear seats, as well as active and passive passenger interaction, thereby truly unleashing the full potential of the smart cockpit as a "mobile living space." Summary of the Invention

[0006] Based on this, it is necessary to address the technical problems mentioned above, such as the independent display content and interactive functions of multiple display screens in existing cockpits, or only simple content mapping, which leads to increased system overhead due to independent rendering of each screen, inconsistent UI style and interaction logic causing fragmented user experience, and lack of collaborative interaction between multiple screens in a unified scene. A cockpit 3D-HMI (3D Human-Computer Interface) multi-screen collaborative interaction method, system, medium, and device should be provided to enhance immersion, optimize interaction efficiency, and improve system performance and cost advantages.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: In a first aspect, the present invention provides a cockpit 3D-HMI multi-screen collaborative interaction method, comprising: A 3D-HMI dynamic scene map is constructed based on a pre-set resource library and real-time vehicle status. Based on the 3D-HMI dynamic scene map, the display terminals in different locations in the vehicle cabin are rendered differently to obtain the display content of each display terminal. In response to a user interaction command received on any of the display terminals, the 3D-HMI dynamic scene map is updated; The content displayed on the corresponding display terminal is updated synchronously based on the updated 3D-HMI dynamic scene map.

[0008] Secondly, the present invention also provides a cockpit 3D-HMI multi-screen collaborative interaction system, utilizing a cockpit 3D-HMI multi-screen collaborative interaction method as described above, comprising: The scene building module is used to build a 3D-HMI dynamic scene map based on a preset resource library and the real-time status of the vehicle. The display rendering module is used to perform differentiated rendering on multiple display terminals in different locations in the vehicle cabin based on the 3D-HMI dynamic scene map, so as to obtain the display content of each display terminal. An interactive response module is used to update the 3D-HMI dynamic scene map in response to a user interaction command received on any of the display terminals. The display update module is used to synchronously update the display content on the corresponding display terminal based on the updated 3D-HMI dynamic scene map.

[0009] Thirdly, the present invention also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to realize a cockpit 3D-HMI multi-screen collaborative interaction method as described above.

[0010] Fourthly, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of a cockpit 3D-HMI multi-screen collaborative interaction method as described above are performed.

[0011] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention constructs a visually unified digital world by having multiple display terminals share the same 3D map scene. Combined with a customized visual design, it greatly enhances the immersive experience for all cabin occupants, realizing the concept of a "third mobile space" within the cabin. The flat and game-like interaction logic reduces the learning cost for users. Multi-screen collaborative interaction based on the same data source and scene rules seamlessly connects road information, central control operations, and rear-seat entertainment information, improving operational intuitiveness and safety. Through deep optimization for the Snapdragon 8797 chip, stable multi-screen output in complex scenarios is achieved without relying on expensive hardware upgrades. This provides a key advantage for mass production and cost control. Deeply integrating the concept of a metaverse, the multi-screen collaborative 3D HMI transforms the in-vehicle space into a continuous entertainment and information platform, providing a foundational interaction framework for future in-vehicle applications (such as social and office applications) with high scalability. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a cockpit 3D-HMI multi-screen collaborative interaction method in some embodiments of this application; Figure 2 This is a schematic diagram of the initialization interface of a cockpit 3D-HMI multi-screen collaborative interaction method in a specific implementation process according to some embodiments of this application; Figure 3 This is a schematic diagram of the entertainment service area interface in the specific implementation process of a cockpit 3D-HMI multi-screen collaborative interaction method in some embodiments of this application; Figure 4 This is a schematic diagram of the music player interface in the specific implementation process of a cockpit 3D-HMI multi-screen collaborative interaction method in some embodiments of this application; Figure 5 This is a flowchart illustrating a cockpit 3D-HMI multi-screen collaborative interaction system in some embodiments of this application. Detailed Implementation

[0013] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses. The term "determine" broadly covers a wide variety of actions, including acquiring, calculating, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), probing, and similar actions; it may also include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), and similar actions; it may also include generating, creating, establishing, and similar actions; and parsing, selecting, choosing, and similar actions, etc. Definitions of other terms will be given in the following description.

[0014] It should be noted that when one element is considered to be "connected" to another element, it can be directly connected to the other element or connected to the other element through an intermediary element. Furthermore, in the following embodiments, "connection" should be understood as "electrical connection," "communication connection," etc., if there is transmission of electrical signals or data between the connected objects.

[0015] It should be emphasized that the acquisition, transmission, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations.

[0016] In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0017] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0018] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0019] Example 1 This embodiment provides a cockpit 3D-HMI multi-screen collaborative interaction method, see reference. Figure 1 ,include: A 3D-HMI dynamic scene map is constructed based on a pre-set resource library and real-time vehicle status. Based on the 3D-HMI dynamic scene map, the display terminals in different locations in the vehicle cabin are rendered differently to obtain the display content of each display terminal. In response to a user interaction command received on any of the display terminals, the 3D-HMI dynamic scene map is updated; The content displayed on the corresponding display terminal is updated synchronously based on the updated 3D-HMI dynamic scene map.

[0020] Compared to existing technologies, this method constructs a visually unified digital world by sharing the same 3D map scene across multiple display terminals. Combined with a customized visual design, it greatly enhances the immersive experience for all cabin occupants, realizing the concept of a "third mobile space in the cabin." The collaborative interaction of multiple screens based on the same data source and scene rules enables seamless integration of road information, central control operations, and rear entertainment information, improving operational intuitiveness and safety. It deeply integrates the concept of the metaverse, transforming the in-vehicle space into a continuous entertainment and information platform through multi-screen collaborative 3D HMI, providing an underlying interactive framework for future in-vehicle applications (such as social networking and office work), and possessing high scalability.

[0021] In some preferred embodiments, the resource library includes custom-styled 3D scene models, interactive object models, preset dynamic interaction logic scripts, audio resources, and animation resources; The real-time vehicle status includes any one or more of vehicle speed, steering angle, and gear position, which are used to drive the movement of the vehicle model in the 3D-HMI dynamic scene map, so that it is consistent with the operation of a real vehicle.

[0022] In this embodiment, the entire 3D-HMI dynamic scene map is organized through a hierarchical, node-based scene graph. Unlike traditional methods that rely on real geographic coordinates, the root node of this method's scene graph is a self-defined virtual spatial coordinate system. It includes: Virtual environment nodes: These nodes host self-designed 3D scene models with custom styles (such as futuristic city blocks, digital tunnels, and sci-fi landmarks). These models are created independently by the team and do not rely on real map data.

[0023] Interactive element nodes: manage various interactive object models distributed in the virtual space, such as navigation target points, information panels, entertainment function entrances, and dynamic effect triggers.

[0024] Dynamic logic nodes: These nodes carry the game script logic and are used to control the event sequence, animation playback, state transitions, and user flow in the scene, enabling "game script-style" guidance and exploration.

[0025] The core innovation lies in abstracting and mapping real-world vehicle and user states into a fictional virtual environment. The input is no longer a high-precision map, but rather: Real-time vehicle status data, such as vehicle speed, steering angle, and gear position, is used to drive the movement of the vehicle model in the virtual world, making it consistent with the handling feel of a real vehicle.

[0026] User interaction intents, such as navigation destination settings and entertainment function selections, are translated into target points or triggering events in the virtual world and then processed by the game script engine.

[0027] The resource library includes a set of custom-style shaders, materials, 3D models, and audio resource libraries.

[0028] Furthermore, by integrating a lightweight scripting engine to parse and execute pre-written game scripts, these scripts define the user's interaction flow (e.g., triggering a specific animation upon entering a virtual area) and seamless switching between functions (e.g., switching from navigation mode to entertainment exploration mode), achieving a seamless, immersive experience rather than the abrupt interface transitions of traditional HMIs.

[0029] In some preferred embodiments, the display terminal includes: a HUD display terminal, a central control screen, and a rear entertainment screen.

[0030] In some preferred embodiments, based on the 3D-HMI dynamic scene map, differentiated rendering is performed on multiple display terminals at different locations within the vehicle cabin to obtain the display content of each display terminal, specifically including: For the HUD display terminal, core information is extracted from the 3D-HMI dynamic scene map and mapped into simplified graphics for rendering on the HUD display terminal to obtain the first display content; For the central control screen, the 3D-HMI dynamic scene map is fully rendered and displayed on the central control screen to obtain the second display content; For the rear entertainment screen, environmental information is extracted from the 3D-HMI dynamic scene map, and the highest quality picture is rendered on the rear entertainment screen based on the environmental information to obtain the third display content; The rendering priority of the third display content is lower than that of the first display content and the second display content.

[0031] In this embodiment, key navigation and safety information, such as preset route guidance, next destination markers, and vehicle status icons, is extracted from the virtual scene and converted into simplified graphic elements that meet the requirements of HUD perspective projection. A minimalist rendering strategy is adopted, disabling complex effects and prioritizing high contrast, low latency, and high refresh rate of the graphics to ensure driving safety.

[0032] The content displayed on the central control screen serves as the main interactive view, fully rendering the virtual world and allowing users to rotate, zoom, and explore the scene through touch and gestures. It also enables complete custom-style visual effects (dynamic lighting, glow, particle system).

[0033] The rear entertainment screen provides an explorer's view, such as a third-person perspective following the vehicle or flying freely, focusing on showcasing the environmental art and dynamic details of the virtual world for passenger entertainment. The entertainment screen is rendered with the highest quality visuals, but its rendering priority is lower than the previous two. When system resources are strained, the frame rate can be appropriately reduced to ensure a better experience for the driver through resource scheduling.

[0034] In some preferred embodiments, the update synchronization relationship between the display content on the plurality of display terminals is determined according to preset scene rules; The scenario rules include any one of the following: global collaborative update rules, independent update rules, and mutually exclusive update rules.

[0035] In some preferred embodiments, the global collaborative update rule includes: when the user interaction command is a preset global collaborative command (such as telephone, nap mode, vehicle core settings, etc.), the operation of any display terminal triggers all display terminals to synchronously update the display content; The independent update rule includes: when the user interaction command is a preset entertainment command (such as music, games, movies, etc.), each display terminal independently updates its display content; The mutually exclusive update rules include: when the vehicle cabin is in a specific scenario (such as focusing on navigation driving), pausing the update of the content displayed on the rear entertainment screen, and concentrating resources and interaction focus on the driver's side.

[0036] In addition, multiple independent screens can be linked together into a continuous display canvas in a specific mode (such as nap mode), and the script engine can drive coherent animations and interactions across the three screens, greatly enhancing the sense of spatial immersion.

[0037] This embodiment also uses the process of switching from "travel service area" to "entertainment service area" and launching a music application as an example to illustrate the working process of the present invention.

[0038] The method is run on a typical smart cockpit hardware platform, which is configured as follows: Main control chip: High-performance automotive-grade SoC, such as Qualcomm SA8797P.

[0039] Display units include a P-HUD based on DLP technology, a 12.3-inch central control touchscreen, and a 12.3-inch rear entertainment touchscreen.

[0040] Memory: Equipped with no less than 8GB of LPDDR5 memory to ensure smooth scheduling of large 3D scenes and resources.

[0041] Software environment: The operating system is based on QNX Neutrino RTOS to ensure system real-time performance and reliability. 3D graphics rendering is implemented based on the Vulkan SC API. The core software modules of this invention all run on this environment as upper-layer applications or system middleware.

[0042] When the vehicle starts, the system initializes. It loads a self-built 3D-HMI dynamic scene map, which includes four functional service areas (such as the travel service area and the entertainment service area). Initially, the system defaults to displaying the "travel service area" scene. (See attached image.) Figure 2 The central control screen serves as the main view, fully rendering the 3D environment of the area; the P-HUD displays core information related to travel; and the rear screens can simultaneously display the central control view or remain in standby mode.

[0043] The user intends to use the music function, located in the "Entertainment Service Area." The user issues a scene switching command on the central control screen using a specific gesture (such as rotating a 3D Rubik's Cube control). The scene switching command is responded to and determined as a global scene switching event according to preset scene rules.

[0044] The script is received by the game script engine and then a preset "scene transition animation script" is launched.

[0045] The script controls a virtual camera to produce a one-shot transition animation that smoothly flies through the "Travel Service Area" and finally arrives at the "Entertainment Service Area" in the virtual world.

[0046] This transition animation is rendered in real time on both the central control screen and the rear entertainment screen, creating a strong sense of immersion and gaming. During this time, the P-HUD displays simplified status prompts (such as "Switching to entertainment mode").

[0047] After the animation ends, the virtual camera stabilizes within the "Entertainment Service Area." A specific 3D music application icon appears directly in front of the user's field of vision. (See attached image.) Figure 3 .

[0048] The user clicks the icon. The game script engine responds again, triggering another script that controls the camera position to zoom in and out a second time, as if the user has "entered" the music application. The interface then transforms into the music player's custom 3D UI. (See also...) Figure 4 .

[0049] At this point, based on preset scenario rules, the music application is determined to be an independent entertainment function. Therefore, when the music application is launched on the central control screen, other applications (such as games) can run independently on the rear entertainment screens, and the two do not interfere with each other.

[0050] Throughout the process, the method demonstrates a flexible multi-screen collaboration strategy: when switching scenes, the three screens are synchronized to create a unified sense of immersion; at the functional application level, the central control screen and the rear screen are independent to improve freedom and system efficiency.

[0051] This invention creates a self-constructed, unified 3D-HMI dynamic scene map, decoupled from real-world geographic information, as the sole interaction context. Based on this, a game script engine is introduced as the core controller for the interaction logic. All user interactions (such as unfolding a Rubik's Cube to switch scenes, clicking to enter a functional area) no longer trigger traditional interface transitions, but are instead mapped to the execution of predefined game scripts. These scripts control the virtual camera's perspective, UI elements, and scene objects, achieving seamless transitions and continuous dynamic interactions in a single shot, thus transforming discrete functional operations into a coherent, narrative-driven, immersive experience. Applying game scripting logic to in-vehicle HMI process control is the core method for achieving non-linear, cinematic interaction sequences.

[0052] This invention abandons the traditional flat menu or icon grid layout, innovatively spatializing and partitioning in-vehicle functions within a unified virtual 3D world (such as a travel service area and an entertainment service area). Users switch between function areas at a macro level by manipulating specific 3D controls (such as a Rubik's Cube), and "enter" or "exit" specific functions at a micro level by moving and advancing the viewpoint. This hierarchical visual and interactive design of "world-area-function" makes function navigation itself a three-dimensional spatial exploration activity, greatly enhancing the sense of exploration and intuition. Based on this, the aim is to protect the visual design and interaction model that spatially organizes in-vehicle functions using virtual 3D function areas and uses three-dimensional viewpoint transformation as the core navigation method.

[0053] Based on the aforementioned unified 3D-HMI dynamic scene map, this invention proposes a dynamic, scene-rule-based multi-screen state management strategy. According to the currently active application scene rules, the display and interaction relationships between the P-HUD, central control screen, and rear entertainment screen are dynamically defined, including state synchronization (such as global scene switching, nap mode), state independence (such as central control navigation and rear entertainment not affecting each other), and state mutual exclusion (such as locking non-core interactions while driving). This allows the multi-screen terminal to present a highly unified visual narrative while also achieving flexible personalized interaction, resolving the contradiction between immersion and functional independence.

[0054] Example 2 This embodiment further provides a cockpit 3D-HMI multi-screen collaborative interaction system based on Embodiment 1, utilizing a cockpit 3D-HMI multi-screen collaborative interaction method as described above. (See reference...) Figure 5 ,include: The scene building module is used to build a 3D-HMI dynamic scene map based on a preset resource library and the real-time status of the vehicle. The display rendering module is used to perform differentiated rendering on multiple display terminals in different locations in the vehicle cabin based on the 3D-HMI dynamic scene map, so as to obtain the display content of each display terminal. An interactive response module is used to update the 3D-HMI dynamic scene map in response to a user interaction command received on any of the display terminals. The display update module is used to synchronously update the display content on the corresponding display terminal based on the updated 3D-HMI dynamic scene map.

[0055] In some preferred embodiments, the system runs on an in-vehicle hardware platform based on a Snapdragon 8797 chip, and the Snapdragon 8797 chip is optimized through dynamic level of detail technology.

[0056] This system runs on an in-vehicle hardware platform based on the 8797 chip. Its core idea is to build a unified, centralized 3D-HMI dynamic scene map, and use this map to drive the graphics rendering and interaction logic of all display terminals in the cabin (including P-HUD, central control screen, and rear entertainment screen).

[0057] Once the system starts, the scene construction module builds a complete, custom-styled 3D-HMI dynamic scene map in memory based on the vehicle's current location, navigation information, and a metaverse-style material library. The display rendering module then "extracts" or "derives" the most suitable view for each of the P-HUD, central control screen, and rear entertainment screen based on their physical characteristics (such as size, resolution, viewing distance, and projection method) and functional priorities, and performs differentiated rendering optimization. The interaction response module and display update module handle user input (such as touch and voice) and feed back interaction events to the 3D-HMI dynamic scene map, ensuring that operations on any screen update the display content of all screens in real time and synchronously, thus achieving a truly immersive and integrated experience.

[0058] It is understood that the system in this embodiment corresponds to the method in Embodiment 1 above, and the options in Embodiment 1 above are also applicable to this embodiment, so they will not be described again here.

[0059] Example 3 This embodiment provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor, causing the processor to perform some or all of the steps of the method provided in Embodiment 1 of this application.

[0060] It is understood that the storage medium can be transient or non-transient. Exemplarily, the storage medium includes, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0061] By way of example, the processor may be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0062] By way of example, the read-only memory includes, but is not limited to, MASK ROM, PROM, EPROM, EEPROM, Flash, etc.

[0063] By way of example, the random access memory includes, but is not limited to, DRAM, SRAM, SDRAM, DDR SDRAM, etc.

[0064] In some examples, a computer program product is provided, which can be implemented by hardware, software, or a combination thereof. As a non-limiting example, the computer program product can be embodied in the storage medium, or it can be embodied in a software product, such as an SDK (Software Development Kit).

[0065] As a non-limiting example, a computer program product is provided, comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium, and executes the computer-executable instructions, causing the electronic device to perform some or all of the steps of the method described in the embodiments of this application.

[0066] In some examples, a computer program is provided, including computer-readable code, wherein, when the computer-readable code is run in a computer device, a processor in the computer device performs some or all of the steps for implementing the method.

[0067] This embodiment also proposes an electronic device, including a memory and a processor. The memory stores at least one instruction, at least one program, code set, or instruction set. When the processor executes the at least one instruction, at least one program, code set, or instruction set, it implements some or all of the steps of the method described in Embodiment 1.

[0068] In some examples, a hardware entity of the electronic device is provided, including: a processor, a memory, and a communication interface; wherein the processor typically controls the overall operation of the electronic device; the communication interface is used to enable the electronic device to communicate with other terminals or servers via a network; the memory is configured to store instructions and applications executable by the processor, and may also cache data to be processed or already processed (including but not limited to image data, audio data, voice communication data, and video communication data) to be processed by the processor and various modules in the electronic device, and may be implemented using flash memory (FLASH), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or random access memory (RAM).

[0069] A processor may include one or more processing elements. Therefore, a processor may include one or more integrated circuits (ICs) configured to perform the functions of the processor. Furthermore, each integrated circuit may include circuitry (e.g., a first circuit, a second circuit, and other circuitry) configured to perform the functions of the processor.

[0070] Furthermore, data can be transferred between the processor, communication interface, and memory via a bus, which can include any number of interconnected buses and bridges, connecting various circuits of one or more processors and memories together.

[0071] It is understood that the options in Embodiment 1 above also apply to this embodiment, so they will not be described again here.

[0072] The same or similar labels correspond to the same or similar parts; The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this application. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0073] In different specific implementations, the methods or systems described in this application can be implemented in software, hardware, or a combination thereof. Furthermore, the order of the method steps can be changed, and various elements can be added, reordered, combined, omitted, or modified.

[0074] Obviously, the above embodiments of this application are merely examples for clearly illustrating this application, and are not intended to limit the implementation of this application, nor are they intended to limit this application. For those skilled in the art, other variations or modifications can be made based on the above description. The separate structural / functional modules or units can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. The structure and function of the separate components can be implemented as a combined structure or component. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of the claims of this application.

Claims

1. A cockpit 3D-HMI multi-screen collaborative interaction method, characterized in that, include: A 3D-HMI dynamic scene map is constructed based on a pre-set resource library and real-time vehicle status. Based on the 3D-HMI dynamic scene map, the display terminals in different locations in the vehicle cabin are rendered differently to obtain the display content of each display terminal. In response to a user interaction command received on any of the display terminals, the 3D-HMI dynamic scene map is updated; The content displayed on the corresponding display terminal is updated synchronously based on the updated 3D-HMI dynamic scene map.

2. The cockpit 3D-HMI multi-screen collaborative interaction method according to claim 1, characterized in that, The resource library includes custom-styled 3D scene models, interactive object models, preset dynamic interaction logic scripts, audio resources, and animation resources. The real-time vehicle status includes any one or more of the following: vehicle speed, steering angle, and gear position.

3. The cockpit 3D-HMI multi-screen collaborative interaction method according to claim 1, characterized in that, The display terminals include: a HUD display terminal, a central control screen, and a rear entertainment screen.

4. The cockpit 3D-HMI multi-screen collaborative interaction method according to claim 3, characterized in that, Based on the 3D-HMI dynamic scene map, differentiated rendering is performed on multiple display terminals at different locations within the vehicle cabin to obtain the display content of each display terminal, specifically including: For the HUD display terminal, core information is extracted from the 3D-HMI dynamic scene map and mapped into simplified graphics for rendering on the HUD display terminal to obtain the first display content; For the central control screen, the 3D-HMI dynamic scene map is fully rendered and displayed on the central control screen to obtain the second display content; For the rear entertainment screen, environmental information is extracted from the 3D-HMI dynamic scene map, and the highest quality picture is rendered on the rear entertainment screen based on the environmental information to obtain the third display content; The rendering priority of the third display content is lower than that of the first display content and the second display content.

5. The cockpit 3D-HMI multi-screen collaborative interaction method according to claim 3, characterized in that, The update synchronization relationship between the displayed content on the multiple display terminals is determined according to preset scenario rules; The scenario rules include any one of the following: global collaborative update rules, independent update rules, and mutually exclusive update rules.

6. The cockpit 3D-HMI multi-screen collaborative interaction method according to claim 5, characterized in that, The global collaborative update rule includes: when the user interaction command is a preset global collaborative command, the operation of any display terminal triggers all display terminals to synchronously update the display content; The independent update rule includes: when the user interaction instruction is a preset entertainment instruction, each display terminal independently updates its display content; The mutually exclusive update rule includes: when the vehicle cabin is in a specific scenario, pausing the update of the content displayed on the rear entertainment screen.

7. A cockpit 3D-HMI multi-screen collaborative interaction system, utilizing a cockpit 3D-HMI multi-screen collaborative interaction method as described in any one of claims 1-6, characterized in that, include: The scene building module is used to build a 3D-HMI dynamic scene map based on a preset resource library and the real-time status of the vehicle. The display rendering module is used to perform differentiated rendering on multiple display terminals in different locations in the vehicle cabin based on the 3D-HMI dynamic scene map, so as to obtain the display content of each display terminal. An interactive response module is used to update the 3D-HMI dynamic scene map in response to a user interaction command received on any of the display terminals. The display update module is used to synchronously update the display content on the corresponding display terminal based on the updated 3D-HMI dynamic scene map.

8. A cockpit 3D-HMI multi-screen collaborative interaction system according to claim 7, characterized in that, The system runs on an in-vehicle hardware platform with the Snapdragon 8797 chip at its core, and optimizes the Snapdragon 8797 chip through dynamic detail level technology.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the method as described in any one of claims 1-6.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions, which, when executed by the processor, perform the steps of the method as described in any one of claims 1-6.

Images