An intelligent cockpit implementation method and system based on holographic projection voice interaction

Abstract

This invention provides a method and system for implementing a smart cockpit based on holographic projection and voice interaction, belonging to the field of automotive cockpit technology. The method includes: acquiring voice commands from the driver and passengers, and acquiring vehicle data; integrating the voice commands and vehicle data, performing logical judgments, and generating corresponding control commands and display content; sending the control commands to the corresponding electronic control unit for execution, and sending the display content to the holographic projection display unit to display a three-dimensional image. This invention uses holographic images in a vacuum to make information interaction more technologically advanced and engaging. This invention integrates display, interaction, and control into a unified and intelligent cockpit ecosystem. The concrete virtual assistant and emotional voice interaction of this invention enhance the emotional bond between the user and the vehicle.

Description

Technical Field

[0001] This invention relates to the field of automotive cockpit technology, and in particular to a method and system for realizing a smart cockpit with voice interaction based on holographic projection. Background Technology

[0002] With the development of intelligent connected vehicles, traditional cockpit infotainment systems and interaction methods are no longer able to meet users' needs for intelligent, immersive, and emotional experiences.

[0003] Specifically, existing cockpit technology has the following problems:

[0004] 1) The interactive information is presented in a flat way: most of the information is displayed on a two-dimensional screen, which lacks a sense of space and three-dimensionality, and the displayed content is limited.

[0005] 2) Lack of emotional communication: The cold interface and voice feedback make it difficult to provide a human-like sense of companionship.

[0006] Therefore, there is an urgent need for a new cockpit system that can provide three-dimensional visual presentation, natural voice interaction, and be deeply integrated into vehicle control and services. Summary of the Invention

[0007] The present invention aims to solve at least one of the above-mentioned problems in the prior art, and constructs an intelligent cockpit with holographic projection and voice interaction functions. By using voice as the main input, manual operation by the driver is reduced, and voice and gesture interaction are convenient. Through holographic projection, key information and virtual assistant images are presented in a three-dimensional floating form, enhancing the information presentation effect.

[0008] In a first aspect, embodiments of the present invention provide a method for implementing a smart cockpit based on holographic projection and capable of voice interaction, including:

[0009] Obtain voice commands from drivers and passengers, and acquire vehicle data;

[0010] The system integrates the voice commands and vehicle data, performs logical judgments, and generates corresponding control commands and display content.

[0011] The control commands are sent to the corresponding electronic control unit for execution, and the display content is sent to the holographic projection display unit to display a three-dimensional image.

[0012] In a preferred embodiment, it further includes:

[0013] In response to a custom request, the virtual assistant's image is edited, and the virtual assistant's image is a predefined 3D cartoon or anthropomorphic image.

[0014] In a preferred embodiment, it further includes:

[0015] The system recognizes the hand gestures of the driver and passengers, generates corresponding hand gesture commands, updates the displayed content in real time according to the hand gesture commands, and sends the updated displayed content to the holographic projection display unit to update the displayed three-dimensional image.

[0016] In a preferred embodiment, the step of integrating the voice command and the vehicle data, performing logical judgments, and generating corresponding control commands and display content includes:

[0017] The control commands include voice broadcast commands with emotions, the displayed content is a virtual assistant, and the voice with emotions is synchronized with the lip movements and facial expressions of the virtual assistant.

[0018] In a preferred embodiment, the step of integrating the voice command and the vehicle data, performing logical judgments, and generating corresponding control commands and display content includes:

[0019] When it is determined that the vehicle is being used, a welcome voice broadcast command is generated, and a virtual assistant welcome performance animation is displayed.

[0020] In a preferred embodiment, the step of integrating the voice command and the vehicle data, performing logical judgments, and generating corresponding control commands and display content includes:

[0021] When the voice command is a navigation command, the destination of the navigation command is obtained, the current GPS location of the vehicle is obtained, and the destination information is queried and calculated in combination with the cloud map service. The display content of the 3D navigation interface is generated, the voice broadcast command of the navigation type and the display content of the virtual assistant navigation gesture animation are generated, and the navigation command is generated.

[0022] In a preferred embodiment, the step of integrating the voice command and the vehicle data, performing logical judgments, and generating corresponding control commands and display content includes:

[0023] When the voice command is to display the vehicle status, the system determines whether a component is faulty based on the DTC fault code information of each component in the vehicle data. Based on the remaining fuel / battery level, door status, and tire pressure information in the vehicle data, the system generates 3D vehicle display content. The 3D vehicle display content shows the remaining fuel / battery level, door status, and tire pressure information, while highlighting the location of the faulty component.

[0024] When the voice command is a user search, the built-in model splicing processing network is used to obtain the image corresponding to the user's search intent and generate the display content of the 3D image.

[0025] In a second aspect, embodiments of the present invention provide a smart cockpit system based on holographic projection and capable of voice interaction, configured to implement any of the methods described in the first aspect, the system comprising:

[0026] The acquisition module is used to acquire voice commands from drivers and passengers and to acquire vehicle data.

[0027] The processing and control module is used to integrate the voice commands and vehicle data, perform logical judgments, and generate corresponding control commands and display content.

[0028] The sending module is used to send the control commands to the corresponding electronic control unit for execution, and to send the display content to the holographic projection display unit to display a three-dimensional image.

[0029] Thirdly, embodiments of the present invention provide an electronic device, including:

[0030] One or more processors;

[0031] Memory, used to store one or more programs;

[0032] When the one or more programs are executed by the one or more processors, the one or more processors implement the methods as described in any of the first aspects.

[0033] Fourthly, embodiments of the present invention provide a computer-readable medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0034] Beneficial effects of this invention:

[0035] Immersive experience: This invention uses holographic images in a vacuum to break the boundaries of the screen, making information interaction more technological and fun.

[0036] Highly integrated: This invention integrates display, interaction, and control into a unified and intelligent cockpit ecosystem.

[0037] Emotional Connection: The present invention’s tangible virtual assistant and emotional voice interaction enhance the emotional bond between the user and the vehicle. Attached Figure Description

[0038] Figure 1 This is a block diagram of the overall architecture of a smart cockpit system based on holographic projection and voice interaction, provided as an embodiment of the present invention.

[0039] Figure 2 This is a schematic diagram of the virtual assistant image customization scenario provided in an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram of the welcoming performance scene provided in an embodiment of the present invention.

[0041] Figure 4 This is a schematic diagram of a user navigation scenario provided in an embodiment of the present invention.

[0042] Figure 5 This is a schematic diagram of a vehicle status display scenario provided in an embodiment of the present invention.

[0043] Figure 6 This is a schematic diagram illustrating the user search intent presentation scenario provided in an embodiment of the present invention.

[0044] Figure 7 This is a schematic flowchart illustrating a method for implementing a smart cockpit based on holographic projection and voice interaction, as provided in an embodiment of the present invention.

[0045] Figure 8 This is a schematic diagram of the overall process of a smart cockpit implementation method based on holographic projection and voice interaction, provided in an embodiment of the present invention.

[0046] Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0047] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0048] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0049] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0050] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0051] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0052] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0053] This invention proposes a smart cockpit solution based on holographic projection and voice interaction to address the shortcomings of existing technologies. Its core objective is:

[0054] 1) Convenience of voice and gesture interaction: Voice is used as the main input, reducing manual operation by the driver. At the same time, gestures can be used to enlarge, shrink, and rotate the holographic projection.

[0055] 2) Enhance information presentation: Through holographic projection, key information (such as navigation and vehicle status) and virtual assistant images are presented in a three-dimensional floating form. It can also integrate images of people, buildings, and landscapes obtained from online searches and convert them into 3D presentations to enhance the immersive experience.

[0056] 3) Deep integration of vehicle control: Connect the interactive system with the vehicle's core control system to enable voice control of vehicle functions;

[0057] 4) Emotionalized cockpit interaction: The holographic assistant's voice, lip movements, and actions are anthropomorphized.

[0058] Figure 1 This is a system architecture block diagram of a smart cockpit based on holographic projection and capable of voice interaction, provided as an embodiment of the present invention. (See diagram below.) Figure 1 As shown, the system includes: an in-vehicle entertainment display module, a holographic projection display module, a multimodal voice interaction module, a central processing and control module, a vehicle data bus interface module, and a gesture recognition module.

[0059] The core components of the in-vehicle entertainment display module include an in-vehicle entertainment display screen and controller. Its functions include: installing a holographic projection virtual assistant image customization app, allowing users to customize the virtual image's hair, eyes, eyebrows, mouth, nose, glasses, top, bottom, shoes, socks, gloves, etc., to create images with different facial features and decorations.

[0060] The core components of the holographic projection display module use light field display technology or laser scanning projection technology, combined with a special holographic display medium (such as a transparent holographic film, a spiral rotating mirror, etc.), to generate a holographic image in a vacuum in a specific space inside the cockpit (such as above the center console) without the need for wearing any equipment.

[0061] The holographic projection display module displays the following content: Virtual assistant avatar: a customizable 3D cartoon or anthropomorphic avatar, serving as the visual medium for system-user interaction; 3D navigation map: a floating 3D map displaying real-time road and traffic conditions, and POI information points; Vehicle status information: displaying vehicle battery / fuel levels, tire pressure, door status, etc., in 3D model form, with faulty parts highlighted; User intent presentation: displaying 3D information such as people, animals, and scenery that the user wants to see.

[0062] The holographic projection display module is responsible for receiving instructions from the central processing unit, rendering and generating realistic three-dimensional dynamic images.

[0063] The core components of the multimodal voice interaction module include a high-sensitivity microphone array, speakers, and a natural language processing unit. Key technologies of the multimodal voice interaction module include: far-field voice wake-up and recognition: employing beamforming technology to accurately pick up voice commands from drivers or passengers while suppressing environmental noise; natural language understanding and dialogue management: a built-in large language model capable of understanding context, processing ambiguous commands, and conducting multi-turn dialogues; and speech synthesis and emotional expression: the generated speech can carry emotions (such as happiness or playfulness) and is synchronized with the lip movements and facial expressions of the holographic virtual assistant.

[0064] The multimodal voice interaction module is designed to collect, recognize, and understand user voice messages, and generate human-like voice feedback.

[0065] The core components of the central processing and control module utilize a high-performance onboard computing platform (integrating GPU and NPU). The functions of the central processing and control module include: system brain: coordinating and controlling the work of all other modules; data fusion and decision-making: integrating commands from the voice module and data from the vehicle bus, performing logical judgments, and generating corresponding control commands and display content; 3D image generation: built-in models that can stitch and integrate image information from web searches to generate 3D graphics; and 3D graphics rendering: rendering high-quality 3D graphics in real time for the holographic projection display module.

[0066] The core component of the vehicle data bus interface module adopts a CAN / FlexRay / Ethernet gateway. The functions of the vehicle data bus interface module include: data acquisition: real-time acquisition of vehicle speed, fuel consumption, door status, GPS location, sensor data, etc., from networks such as the vehicle CAN bus; command issuance: conversion of voice control commands issued by the central processing unit (such as "turn on the air conditioning to 25 degrees" or "navigate to the company") into standard vehicle control commands, which are then sent to the corresponding electronic control units for execution via the bus.

[0067] The core components of the gesture recognition module are sensors (ToF camera and infrared camera). The key technology of the gesture recognition module is that AI algorithms (computer vision and deep learning models) identify key points of the hand (such as joints and fingertips) and track their movement trajectory in real time. The function of the gesture recognition module is to match the tracked hand movement trajectory with a preset "gesture library" to determine whether the user's intention is "swipe" or "rotate," etc.

[0068] The intelligent cockpit system based on holographic projection and capable of voice interaction provided in the above embodiments of the present invention can be used in the following scenarios.

[0069] Scenario 1: Customization of Virtual Assistant Image

[0070] The virtual assistant avatar customization scenario includes: custom facial creation. Users open the app on the in-car entertainment display screen. The app categorizes virtual avatar appearances, including hair, eyes, eyebrows, mouth, nose, glasses, top, bottom, shoes, socks, gloves, etc. Each category includes multiple different decorations. Users can customize their avatar's appearance, and after confirmation, the holographic virtual assistant avatar will change accordingly. For example... Figure 2 As shown, the virtual assistant image customization process includes: setting the virtual assistant image and wake-up word on the in-vehicle entertainment controller; rendering the 3D virtual assistant image according to the settings on the central processing and control module; and displaying the virtual assistant image on the holographic projection display module.

[0071] Scene 2: Welcoming Performance

[0072] The welcoming performance scene includes: after the user opens the car door, gets in, and starts the vehicle, the central processing and control module receives CAN signal data from the vehicle data bus interface module, determines that the user has started using the vehicle, and the holographic projection display module plays a welcome holographic projection animation, along with a voice announcement: "Hi, I am your intelligent assistant [nickname], it's nice to see you again, and I wish you a pleasant journey today." Figure 3 As shown, the welcoming performance scene process includes: the vehicle data bus interface module obtains the door unlock and vehicle ignition signals from the vehicle CAN bus and sends them to the central processing and control module; the central processing and control module renders the virtual assistant 3D welcome animation and voice commands and sends them to the holographic projection display module and the multimodal voice interaction module; the holographic projection display module plays the welcoming animation; and the multimodal voice interaction module plays the welcoming voice.

[0073] Scenario 3: User Navigation

[0074] like Figure 4 As shown, the user navigation scenario process includes:

[0075] Wake-up: The user says the wake-up phrase "Hello, Xiao V", and the microphone array picks up the voice;

[0076] Recognition and understanding: The multimodal voice interaction module performs noise reduction and recognition, and sends commands such as "navigate to the nearest charging station" to the central processing and control module. The engine understands the "navigation" intent and identifies the target location as "the nearest charging station".

[0077] Data processing: The central processing and control module obtains the current GPS location through the vehicle data bus interface and, in conjunction with cloud map services, queries and calculates a list of nearby charging stations;

[0078] Holographic presentation and interaction: The central processing and control module instructs the holographic projection display module to project a 3D navigation interface into the cockpit. A virtual assistant appears and says in voice and gesture: "Found 3 charging stations, of which station A is the closest and has 2 available charging spots. Do you want to navigate to this station?" At the same time, these three options will be highlighted on the 3D map.

[0079] Confirmation and Execution: After the user answers "yes", the system confirms again via voice and sends the navigation command to the vehicle navigation ECU through the vehicle bus to start navigation. The holographic interface changes to 3D arrows and road guidance.

[0080] Scenario 4: Vehicle Status Display

[0081] like Figure 5 As shown, the vehicle status display scenario flow includes:

[0082] Wake-up: The user says the wake-up phrase "Hello, Xiao V", and the microphone array picks up the voice;

[0083] Recognition and Understanding: The multimodal voice interaction module performs noise reduction and recognition, and sends the command "Display vehicle status" to the central processing and control module;

[0084] Data processing: The central processing and control module obtains DTC fault code information of each component through the vehicle data bus interface module to determine whether the component is faulty; at the same time, it obtains information such as remaining fuel / battery, door status, and tire pressure.

[0085] Holographic presentation: The central processing and control module instructs the holographic projection display module to project a 3D vehicle into the cockpit, displaying information such as remaining battery / fuel level, door closure status, and tire pressure. At the same time, the location of faulty parts is highlighted. Users can zoom in, zoom out, and select the 3D vehicle and view the vehicle model from 360 degrees.

[0086] Scenario 5: Presentation of User Search Intent

[0087] Example: User search intent: "I want to see what a giraffe looks like," "I want to see the autumn scenery of Lhasa." For example... Figure 6 As shown, the vehicle status display scenario flow includes:

[0088] Wake-up: The user says the wake-up phrase "Hello, Xiao V", and the microphone array picks up the voice;

[0089] Recognition and understanding: The multimodal voice interaction module performs noise reduction and recognition, and sends the command "Show giraffe" to the central processing and control module;

[0090] Data processing: The central processing and control module uses a built-in model stitching process to process giraffe images acquired from the network and generate 3D images;

[0091] Holographic presentation: The central processing and control module instructs the holographic projection display module to project a 3D image of a giraffe into the cockpit, which users can rotate with gestures to view from multiple angles.

[0092] In this embodiment of the invention, for ease of description, the cockpit domain controller will be used as the execution subject in the following description. The execution subject can be other electronic devices, software modules, etc., that can achieve the following functions.

[0093] Figure 7 This is a schematic flowchart illustrating a method for implementing a smart cockpit based on holographic projection and voice interaction, as provided in an embodiment of the present invention. Figure 8 This is a schematic diagram illustrating the overall process of a method for implementing a voice-interactive smart cockpit based on holographic projection, as provided in an embodiment of the present invention. Figure 7 and Figure 8As shown, the method includes:

[0094] Step S1: Obtain voice commands from the driver and passengers, and acquire vehicle data;

[0095] Step S2: Integrate the voice commands and vehicle data, perform logical judgments, and generate corresponding control commands and display content;

[0096] Step S3: Send the control command to the corresponding electronic control unit for execution, and send the display content to the holographic projection display unit to display the three-dimensional image.

[0097] This invention integrates holographic display technology in a vacuum with the cockpit design through the above steps, solving the problem of stably generating high-quality holographic images in a confined and vibrating vehicle environment.

[0098] In some embodiments, the system further includes: step S0, in response to a custom request, editing the image of the virtual assistant, wherein the image of the virtual assistant is a predefined 3D cartoon or anthropomorphic image, serving as a visual carrier for interaction between the system and the user.

[0099] In some embodiments, the method further includes: step S4, recognizing the hand gestures of the driver and passengers, generating corresponding hand gesture commands, updating the display content in real time according to the hand gesture commands, and sending the updated display content to the holographic projection display unit to update the displayed three-dimensional image.

[0100] If the displayed content is a 3D image and the gesture is a zoom-in, zoom-out, or rotation gesture, after receiving the gesture command, the 3D image is re-rendered and sent to the holographic projection display unit to update the displayed 3D image.

[0101] The present invention combines holographic projection with gesture interaction through the above steps: the viewing angle of the holographic image can be adjusted by gesture.

[0102] In some embodiments, in step S2, which involves integrating the voice command and the vehicle data, performing logical judgments, and generating corresponding control commands and display content:

[0103] The control commands include voice broadcast commands with emotions (such as happiness or playfulness), the displayed content is a virtual assistant, and the voice with emotions is synchronized with the lip movements and facial expressions of the virtual assistant.

[0104] In some embodiments, in step S2, which involves integrating the voice command and the vehicle data, performing logical judgments, and generating corresponding control commands and display content:

[0105] When it is determined that the vehicle is being used, a welcome voice broadcast command is generated, and a virtual assistant welcome performance animation is displayed.

[0106] Among them, obtaining door unlock and vehicle ignition signals from the vehicle's CAN bus can be used to determine when a user starts using the vehicle.

[0107] In some embodiments, in step S2, which involves integrating the voice command and the vehicle data, performing logical judgments, and generating corresponding control commands and display content:

[0108] When the voice command is a navigation command, the destination of the navigation command is obtained, the current GPS location of the vehicle is obtained, and the destination information is queried and calculated in combination with the cloud map service. The display content of the 3D navigation interface is generated, the voice broadcast command of the navigation type and the display content of the virtual assistant navigation gesture animation are generated, and the navigation command is generated.

[0109] For example, navigation commands are sent to the vehicle navigation ECU via the vehicle bus to start navigation.

[0110] In some embodiments, in step S2, which involves integrating the voice command and the vehicle data, performing logical judgments, and generating corresponding control commands and display content:

[0111] When the voice command is to display the vehicle status, the system determines whether a component is faulty based on the DTC fault code information of each component in the vehicle data. Based on the remaining fuel / battery level, door status, and tire pressure information in the vehicle data, the system generates 3D vehicle display content. The 3D vehicle display content shows the remaining fuel / battery level, door status, and tire pressure information, while highlighting the location of the faulty component.

[0112] When the voice command is a user search, the built-in model splicing processing network is used to obtain the image corresponding to the user's search intent and generate the display content of the 3D image.

[0113] This invention achieves deep integration of multimodal interaction through the above steps, enabling real-time synchronization and linkage of voice, vision (virtual assistant expressions / gestures), and vehicle context data, providing a unified interactive experience.

[0114] Based on the same inventive concept, embodiments of the present invention also provide a smart cockpit implementation system based on holographic projection and capable of voice interaction, configured to implement any of the methods described in the above embodiments, the system comprising:

[0115] The acquisition module is used to acquire voice commands from drivers and passengers and to acquire vehicle data.

[0116] The processing and control module is used to integrate the voice commands and vehicle data, perform logical judgments, and generate corresponding control commands and display content.

[0117] The sending module is used to send the control commands to the corresponding electronic control unit for execution, and to send the display content to the holographic projection display unit to display a three-dimensional image.

[0118] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 9 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0119] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0120] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0121] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0122] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable medium. This computer-readable medium stores a computer program, wherein, when executed by a processor, the program implements the steps of any of the methods described in the above embodiments. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0123] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0124] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0125] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0126] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0127] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0128] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0129] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0130] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0132] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A method for implementing a smart cockpit with voice interaction based on holographic projection, characterized in that, include: Obtain voice commands from drivers and passengers, and acquire vehicle data; The system integrates the voice commands and vehicle data, performs logical judgments, and generates corresponding control commands and display content. The control commands are sent to the corresponding electronic control unit for execution, and the display content is sent to the holographic projection display unit to display a three-dimensional image.

2. The method according to claim 1, characterized in that, Also includes: In response to a custom request, the virtual assistant's image is edited, and the virtual assistant's image is a predefined 3D cartoon or anthropomorphic image.

3. The method according to claim 1, characterized in that, Also includes: The system recognizes the hand gestures of the driver and passengers, generates corresponding hand gesture commands, updates the displayed content in real time according to the hand gesture commands, and sends the updated displayed content to the holographic projection display unit to update the displayed three-dimensional image.

4. The method according to claim 1, characterized in that, In the step of integrating the voice commands and vehicle data, performing logical judgments, and generating corresponding control commands and display content: The control commands include voice broadcast commands with emotions, the displayed content is a virtual assistant, and the voice with emotions is synchronized with the lip movements and facial expressions of the virtual assistant.

5. The method according to claim 4, characterized in that, In the step of integrating the voice commands and vehicle data, performing logical judgments, and generating corresponding control commands and display content: When it is determined that the vehicle is being used, a welcome voice broadcast command is generated, and a virtual assistant welcome performance animation is displayed.

6. The method according to claim 1, characterized in that, In the step of integrating the voice commands and vehicle data, performing logical judgments, and generating corresponding control commands and display content: When the voice command is a navigation command, the destination of the navigation command is obtained, the current GPS location of the vehicle is obtained, and the destination information is queried and calculated in combination with the cloud map service. The display content of the 3D navigation interface is generated, the voice broadcast command of the navigation type and the display content of the virtual assistant navigation gesture animation are generated, and the navigation command is generated.

7. The method according to claim 1, characterized in that, In the step of integrating the voice commands and vehicle data, performing logical judgments, and generating corresponding control commands and display content: When the voice command is to display the vehicle status, the system determines whether a component is faulty based on the DTC fault code information of each component in the vehicle data. Based on the remaining fuel / battery level, door status, and tire pressure information in the vehicle data, the system generates 3D vehicle display content. The 3D vehicle display content shows the remaining fuel / battery level, door status, and tire pressure information, while highlighting the location of the faulty component. When the voice command is a user search, the built-in model splicing processing network is used to obtain the image corresponding to the user's search intent and generate the display content of the 3D image.

8. A smart cockpit system based on holographic projection and capable of voice interaction, characterized in that, The system, configured to implement the method as described in any one of claims 1 to 7, comprises: The acquisition module is used to acquire voice commands from drivers and passengers and to acquire vehicle data. The processing and control module is used to integrate the voice commands and vehicle data, perform logical judgments, and generate corresponding control commands and display content. The sending module is used to send the control commands to the corresponding electronic control unit for execution, and to send the display content to the holographic projection display unit to display a three-dimensional image.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images