Holographic projection interaction method, vehicle-mounted terminal, vehicle and readable storage medium

Abstract

The invention belongs to the technical field of vehicles, and particularly relates to a holographic projection interaction method, a vehicle-mounted terminal, a vehicle and a readable storage medium, and the holographic projection interaction method comprises the step of projecting a 3D interaction object in the vehicle through a holographic three-dimensional projection technology. And sensing user operation information through interaction sensing equipment, and / or receiving interaction information sent by an associated terminal. And controlling the 3D interaction object to perform interaction operation based on the user operation information and / or the interaction information. Therefore, the vehicle can interact with the user through the 3D interaction object, a novel human-vehicle interaction mode is provided, and the use experience of the user can be greatly improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a holographic projection interaction method, a vehicle-mounted terminal, a vehicle and a readable storage medium. Background Art

[0002] With the development of vehicle technology, vehicles are becoming more intelligent and convenient, and people's lives are becoming more and more convenient. Vehicles are moving towards intelligence. Smart vehicles are the key to competition in the second half of the vehicle industry's transformation, and are a key link before technology iteration and commercial application.

[0003] Among some current technologies, the interaction technology between vehicle-mounted terminals and people has been widely used in vehicles. Most of the interactive display or feedback methods between vehicle-mounted terminals and people are through screen displays, voice broadcasts, etc., which has caused consumers to become tired of using them. Therefore, how to provide a novel way for vehicle-mounted terminals to interact with people and enhance user experience is an issue that technical personnel urgently need to solve. Summary of the invention

[0004] In response to the above technical problems, the present application provides a holographic projection interaction method, a vehicle-mounted terminal, a vehicle and a readable storage medium, which can enable the vehicle to interact with the user through 3D interactive objects. The interaction method is novel and can greatly enhance the user experience.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a holographic projection interaction method, comprising: projecting a 3D interactive object in a car through holographic three-dimensional projection technology. Sensing user operation information through an interactive sensing device, and / or receiving interaction information sent by an associated terminal. Based on the user operation information and / or interaction information, controlling the 3D interactive object to perform interactive operations.

[0006] Optionally, the step of controlling the 3D interactive object to perform interactive operation based on user operation information and / or interaction information includes: controlling the 3D interactive object to perform action changes in response to user operation information and / or interaction information through chat robot technology, and the action changes include at least one of facial expression changes, body movement changes, and lip movement changes.

[0007] Optionally, the associated terminal is an AR device. The step of controlling the 3D interactive object to perform interactive operation based on the user operation information and / or the interaction information includes: controlling the 3D interactive object to change the display form based on the interaction information output by the AR device.

[0008] Optionally, the interaction information includes image data or operation instructions. Based on the user operation information and / or interaction information, the step of controlling the 3D interactive object to perform an interactive operation includes: performing an image projection operation based on the image data, and controlling the 3D interactive object to perform a movie viewing interactive action. Or, based on the operation instruction, controlling the 3D interactive object to perform an action.

[0009] Optionally, the step of projecting a 3D interactive object in the vehicle by using the holographic three-dimensional projection technology includes: when it is detected that the vehicle is in a stationary state, projecting the 3D interactive object in the vehicle by using the holographic three-dimensional projection technology.

[0010] Optionally, the step of projecting a 3D interactive object in the vehicle by means of holographic 3D projection technology includes: when a vehicle fault is detected, obtaining fault handling guidance information according to the fault attribute. Based on the fault handling guidance information, projecting a 3D interactive object in the vehicle by means of holographic 3D projection technology to display the fault handling action, and / or outputting a corresponding voice prompt.

[0011] Optionally, the step of controlling the 3D interactive object to perform interactive operations based on user operation information and / or interaction information includes: based on the user operation information, adjusting the viewing of the fault handling action displayed by the 3D interactive object, the viewing adjustment including at least one of pause, zoom in, zoom out, fast forward, rewind, and adjust angle.

[0012] A second aspect of the present application provides a vehicle-mounted terminal, comprising: a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, a holographic projection interaction method as described in any of the above items is implemented.

[0013] A third aspect of the present application provides a vehicle, comprising the aforementioned vehicle-mounted terminal.

[0014] A fourth aspect of the present application provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the holographic projection interaction method as described in any of the above items is implemented.

[0015] The present application provides a holographic projection interaction method, including: projecting a 3D interactive object in a vehicle through holographic three-dimensional projection technology. Sensing user operation information through an interactive sensing device, and / or receiving interaction information sent by an associated terminal. Based on the user operation information and / or interaction information, controlling the 3D interactive object to perform interactive operations. Therefore, the present application can enable the vehicle to interact with the user through the 3D interactive object, providing a novel human-vehicle interaction method, which can greatly enhance the user experience.

[0016] Furthermore, the present application can add 3D interactive objects for viewing interaction when the user is watching a movie on the screen in the vehicle, thereby enhancing the user's viewing pleasure in the car.

[0017] Furthermore, the present application can also integrate AR devices to perform operations such as adjusting the display image of 3D interactive objects, creating a feeling of breaking through the barriers between reality and virtuality and enhancing the user experience.

[0018] Furthermore, when a vehicle breaks down, the present application can also display fault handling actions in a three-dimensional image in an intuitive and vivid manner through 3D interactive objects, making it easier for users to learn fault handling methods and further enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0020] Figure 1 It is a flowchart of a first holographic projection interaction method provided in the first embodiment of the present application;

[0021] Figure 2 It is a flowchart of the second holographic projection interaction method provided in the first embodiment of the present application;

[0022] Figure 3 It is a flowchart of the third holographic projection interaction method provided in the first embodiment of the present application;

[0023] Figure 4 is a first structural schematic diagram of a vehicle-mounted terminal provided in a second embodiment of the present application;

[0024] Figure 5 It is a second structural schematic diagram of the vehicle-mounted terminal provided in the second embodiment of the present application.

[0025] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0027] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. Optionally, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context in the specific embodiment.

[0028] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0029] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0030] It should be noted that, in this article, step codes such as S10 and S11 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing the step, those skilled in the art may execute S10 first and then S11, etc., but these should all be within the scope of protection of this application.

[0031] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0033] First embodiment

[0034] Figure 1 It is a flowchart of a first holographic projection interaction method provided in the first embodiment of the present application; Figure 2 It is a flowchart of the second holographic projection interaction method provided in the first embodiment of the present application; Figure 3 is a flow chart of the third holographic projection interaction method provided in the first embodiment of the present application. In order to clearly describe the holographic projection interaction method provided in the first embodiment of the present application, please refer to Figures 1 to 3 .

[0035] See also Figure 1 The first embodiment of the present application provides a holographic projection interaction method, including:

[0036] S10: Through holographic 3D projection technology, 3D interactive objects are projected inside the car.

[0037] In one embodiment, a 3D interactive object is projected in the vehicle through a 3D projection device, which may be, but is not limited to, a combination of a projector and a stereoscopic imaging prism, or a combination of a projector and a water mist device.

[0038] In one embodiment, the 3D interactive object may be, but is not limited to, a human figure, a cartoon figure, etc. The 3D interactive object is driven by AI technology. Optionally, the AI ​​technology may be the latest chatbot technology, such as ChatGPT, etc.

[0039] In one embodiment, the location where the 3D interactive object is projected may be on the center island or other areas in the vehicle.

[0040] In one embodiment, the step of projecting a 3D interactive object in the vehicle by using holographic 3D projection technology includes: when the vehicle is detected to be stationary, projecting the 3D interactive object in the vehicle by using holographic 3D projection technology. Therefore, the present application projects the 3D interactive object when the vehicle is detected to be stationary, but not in driving state, so as to ensure driving safety.

[0041] In one embodiment, the step of projecting a 3D interactive object in a vehicle through holographic three-dimensional projection technology may include, but is not limited to: when a vehicle fault is detected, obtaining fault handling guidance information according to the fault attributes. Based on the fault handling guidance information, a 3D interactive object is projected in the vehicle through holographic three-dimensional projection technology to display fault handling actions, and / or output corresponding voice prompts. For example, when a tire fault is detected, tire fault handling guidance information is obtained, and a three-dimensional image of a 3D interactive object repairing a tire is displayed according to the handling guidance information. The present application can also display fault handling actions in a three-dimensional image format through a 3D interactive object in an intuitive and vivid manner when a vehicle fault occurs, making it easier for users to learn fault handling methods and further enhancing the user's user experience.

[0042] In one implementation, the present application may pre-store fault handling guidance information for a variety of faults, such as tire failure, light failure, brake failure, etc.

[0043] In one embodiment, the fault handling action image of the 3D interactive object can be generated instantly based on AI technology and fault handling guidance information. Optionally, the fault handling action image of the 3D interactive object can be included or associated with the monthly setting in the fault handling guidance information.

[0044] In one embodiment, the present application may also include receiving image data transmitted by an associated terminal and displaying it through a 3D projection device. In one embodiment, the image data may be video data in a terminal such as a mobile phone or game screen data in an AR device (such as AR glasses). When the game screen data in an AR device (such as AR glasses) is displayed through a 3D projection device, multiple people in the car can interact with each other in the game.

[0045] S11: sensing user operation information through an interaction sensing device, and / or receiving interaction information sent by an associated terminal.

[0046] In one embodiment, step S11: sensing user operation information through an interactive sensing device, and / or receiving interactive information sent by an associated terminal, may include: capturing the user's hand motion information through a gesture tracking detection device in the vehicle to obtain user operation information.

[0047] In other implementations, other biological information sensing devices may be used to sense biological actions to obtain user operation information, for example, detecting head movements, eyeglass movements, etc.

[0048] Therefore, when the user turns on the holographic projection, he can touch it directly. A large number of cameras and other sensing devices are installed in the car, which can use algorithms to calculate the volume and motion trajectory of the collision, control 3D interactive objects, and achieve the purpose of immersive experience.

[0049] In one embodiment, before the step of receiving the interaction information sent by the associated terminal, the step may include: associating with the associated terminal through remote communication technology or near field communication technology. In one embodiment, the near field communication technology may include but is not limited to Bluetooth communication, WiFi communication, radio frequency communication, etc.

[0050] S12: Based on the user operation information and / or interaction information, control the 3D interactive object to perform interactive operations.

[0051] In one embodiment, step S12: based on user operation information and / or interaction information, controlling the 3D interactive object to perform interactive operation may include: controlling the 3D interactive object to perform action changes in response to user operation information and / or interaction information through chat robot technology, and the action changes include at least one of expression changes, body movement changes, and lip movement changes. For example, the user operation information is converted into language information, and the language information is processed through chat robot technology to obtain feedback language information and emotional information, thereby controlling the 3D interactive object to perform action changes based on the feedback language information and emotional information. For another example, the non-language interactive information sent by the associated terminal is converted into language information, and the language information is processed through chat robot technology to obtain feedback language information and emotional information, thereby controlling the 3D interactive object to perform action changes based on the feedback language information and emotional information. Therefore, the present application can control the 3D interactive object to perform anthropomorphic action feedback based on various user operation information and / or interaction information, thereby improving the user's use experience. Optionally, when the 3D interactive object performs lip movement changes in a speaking scene, the corresponding text lip shape can be transformed based on the voice content of the voice display device.

[0052] In one embodiment, the interactive information may include, but is not limited to, image data, operating instructions, or language information. When the interactive information includes operating instructions, the 3D interactive object can be controlled accordingly based on the operating instructions. For example, when the front row user can input user operating information based on the gesture tracking detection device, and the rear row passenger is not in the detection area of ​​the gesture tracking detection area device, a control instruction can be sent through the associated terminal to control the 3D interactive object accordingly. It should be understood that this is only an example, and does not mean that the detection area of ​​the gesture tracking detection device can only be in the front row, and of course it can also cover all areas in the car.

[0053] In one embodiment, the associated terminal is an AR device. Step S12: Based on the user operation information and / or interaction information, controlling the 3D interactive object to perform interactive operations may include: based on the interaction information output by the AR device, controlling the 3D interactive object to change the display form. Therefore, the present application can also integrate the AR device to perform operations such as display image adjustment on the 3D interactive object, creating a feeling of breaking through the barriers between reality and virtuality, and improving the user's experience.

[0054] In one embodiment, an AR device such as AR glasses can realize many functions and can be regarded as a miniature mobile phone. It can determine the user's state by tracking the eye gaze trajectory and activate corresponding functions.

[0055] In one embodiment, when the AR device acquires a 3D interactive object, it can display control instructions for the 3D interactive object, and when receiving a user-selected control instruction, it generates and outputs interactive information to control the 3D interactive object to perform interactive operations. For example, when a user pinches the face or shapes the 3D interactive object through the AR device, the interactive information corresponding to the user operation can be output in real time to control the 3D interactive object to change the display form, or the interactive information can be output after the final form is confirmed to change the display form of the 3D interactive object.

[0056] In other implementations, based on the interactive information output by the AR device, when controlling the 3D interactive object to perform interactive operations, other operations besides shaping may be performed, which can be set according to user needs.

[0057] In one embodiment, step S12: the interactive information includes image data. Based on the user operation information and / or the interactive information, controlling the 3D interactive object to perform interactive operations may include: performing image projection operations based on the image data, and controlling the 3D interactive object to perform interactive viewing actions. Or, based on the operation instructions, controlling the 3D interactive object to perform actions. This application can add 3D interactive objects for interactive viewing when the user is watching a movie on the screen in the vehicle, thereby enhancing the user's viewing pleasure in the vehicle.

[0058] In one embodiment, based on the image data, an image projection operation is performed, and the 3D interactive object is controlled to perform interactive actions for watching movies. For example, the language information of the image data is detected, and the feedback language information and the corresponding 3D interactive object action control information are obtained according to the language information, so as to control the 3D interactive object to perform actions according to the voice broadcast corresponding to the feedback language information, so that when watching movies or TV, the 3D interactive object can achieve the effect of accompanying, chatting, and expressing bullet comments, thereby improving the user's experience. For example, the image data can correspond to the bullet information sent by the associated terminal, and after the image projection operation is performed, the hot comment bullet in the bullet information corresponding to the current screen is detected, and the feedback language information and the corresponding 3D interactive object action control information are obtained according to the hot comment bullet control, so as to control the 3D interactive object to perform actions according to the voice broadcast corresponding to the feedback language information, so that good Guanyin interaction can be achieved. Optionally, when detecting the hot comment bullet in the bullet information corresponding to the current screen, the bullet related to the image screen is selected as the hot comment bullet according to the preset rules.

[0059] In one embodiment, when a vehicle fault is detected, step S12: based on user operation information and / or interaction information, controlling the 3D interactive object to perform interactive operations may include: based on the user operation information, adjusting the viewing of the fault handling action displayed by the 3D interactive object, and the viewing adjustment may include but is not limited to at least one of pause, zoom in, zoom out, fast forward, rewind, and adjust the angle.

[0060] In order to make it easier to understand the above scheme, an example is given below for reference:

[0061] Example 1, see Figure 2 , providing a holographic projection interaction method, comprising:

[0062] S10: Through holographic 3D projection technology, 3D interactive objects are projected inside the car.

[0063] S10a: Based on the interactive information outputted after the AR device performs a virtual object shaping operation, control the 3D interactive object to change the display form.

[0064] S11a: sensing user operation information through an interactive sensing device;

[0065] S12a: Based on the user operation information, control the 3D interactive object to perform interactive operations.

[0066] Therefore, this example 1 integrates AR devices to perform operations such as adjusting the display image of 3D interactive objects, creating a feeling of breaking through the barriers between reality and virtuality and enhancing the user experience.

[0067] Example 2, see Figure 3, providing a holographic projection interaction method, comprising:

[0068] S10: Through holographic 3D projection technology, 3D interactive objects are projected inside the car.

[0069] S10b: Associating with an associated terminal via Bluetooth communication technology;

[0070] S11b: receiving interaction information sent by an associated terminal, where the interaction information includes image data;

[0071] S12b: Based on the image data, perform image projection operations and control the 3D interactive object to perform interactive actions for watching a movie.

[0072] Therefore, in this second example, when the user is watching a movie on the screen in the vehicle, 3D interactive objects are added for viewing interaction, thereby enhancing the user's viewing pleasure in the vehicle.

[0073] The first embodiment of the present application provides a holographic projection interaction method, including: S10: Projecting a 3D interactive object in a vehicle through holographic three-dimensional projection technology. S11: Sensing user operation information through an interactive sensing device, and / or receiving interaction information sent by an associated terminal. S13: Based on user operation information and / or interaction information, controlling the 3D interactive object to perform interactive operations. Therefore, the present application can enable the vehicle to interact with the user through a 3D interactive object, providing a novel human-vehicle interaction method, which can greatly enhance the user experience.

[0074] Second embodiment:

[0075] Figure 4 is a first structural schematic diagram of the vehicle-mounted terminal 1 provided in the second embodiment of the present application; Figure 5 1 is a second structural diagram of the vehicle-mounted terminal 1 provided in the second embodiment of the present application. In order to clearly describe the vehicle-mounted terminal 1 provided in the second embodiment of the present application, please refer to Figure 4 or Figure 5 .

[0076] The vehicle-mounted terminal 1 provided in the second embodiment of the present application includes: a processor A101 and a memory A201. Optionally, the processor A101 is used to execute a computer program A6 stored in the memory A201 to implement the steps of the holographic projection interaction method described in the first embodiment or the second embodiment.

[0077] Optionally, the vehicle-mounted terminal 1 provided in this embodiment may include at least one processor A101 and at least one memory A201. Optionally, the at least one processor A101 may be referred to as a processing unit A1, and the at least one memory A201 may be referred to as a storage unit A2. Optionally, the storage unit A2 stores a computer program A6, and when the computer program A6 is executed by the processing unit A1, the vehicle-mounted terminal 1 provided in this embodiment implements the steps of the holographic projection interaction method described in the first embodiment, for example, Figure 1 10: Projecting a 3D interactive object in the car through holographic three-dimensional projection technology. S11: Sensing user operation information through an interactive sensing device, and / or receiving interactive information sent by an associated terminal. S13: Based on the user operation information and / or interactive information, controlling the 3D interactive object to perform interactive operations. Or, for example, Figure 2 Step S10 shown: projecting a 3D interactive object in the car through holographic three-dimensional projection technology. S10a: controlling the 3D interactive object to change its display form based on the interactive information output by the AR device after the virtual object shaping operation. S11a: sensing the user operation information through the interactive sensing device; S12a: controlling the 3D interactive object to perform interactive operation based on the user operation information. Or, for example, Figure 3 Step S10 shown: projecting a 3D interactive object in the car through holographic three-dimensional projection technology. S10b: associating with an associated terminal through Bluetooth communication technology; S11b: receiving interactive information sent by the associated terminal, the interactive information including image data; S12b: performing image projection operation based on the image data, and controlling the 3D interactive object to perform interactive viewing actions.

[0078] Optionally, the vehicle-mounted terminal 1 provided in this embodiment may include a plurality of memories A201 (referred to as storage units A2 for short).

[0079] Optionally, the storage unit A2 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Optionally, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The storage unit A2 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0080] Optionally, the vehicle terminal 1 further includes a bus connecting different components (eg, the processor A101 and the memory A201, the 3D projection device A3, the interactive sensing device, etc.).

[0081] Optionally, the vehicle-mounted terminal 1 in this embodiment may further include a communication interface (eg, I / O interface A4), which may be used to communicate with external devices.

[0082] In one embodiment, the vehicle-mounted terminal 1 can also be connected to an associated terminal (such as a mobile phone, an AR device) through a communication interface, or be externally connected to a 3D projection device A3, an interactive sensing device, etc.

[0083] In one embodiment, the interactive sensing device may include, but is not limited to, a plurality of cameras installed in the vehicle, a voice collection device, and the like.

[0084] In one embodiment, the external 3D projection device A3 can be installed at the center island in the vehicle, but is not limited to being installed therein. The 3D projection device A3 can be a combination of a projector and a stereoscopic imaging prism, or a combination of a projector and a water mist device, etc.

[0085] Optionally, the vehicle-mounted terminal 1 provided in this embodiment may further include a communication device A5.

[0086] The vehicle terminal 1 provided in the second embodiment of the present application includes a memory A101 and a processor A201, and the processor A101 is used to execute the computer program A6 stored in the memory A201 to implement the steps of the holographic projection interaction method as in the first embodiment or, therefore, the vehicle terminal 1 provided in this embodiment can realize the vehicle interacting with the user through the 3D interactive object, providing a novel human-vehicle interaction method, which can greatly enhance the user's experience. Further, the present application can add 3D interactive objects for viewing and interacting when the user is watching a movie on the vehicle, thereby enhancing the user's viewing pleasure in the vehicle. Further, the present application can also integrate AR devices, perform operations such as display image adjustment on 3D interactive objects, form a feeling of breaking through the barriers between reality and virtuality, and enhance the user's experience. Further, the present application can also display the fault handling action in a three-dimensional image in an intuitive and vivid way through 3D interactive objects when the vehicle fails, so as to facilitate users to learn the fault handling method and further enhance the user's experience.

[0087] The second embodiment of the present application also provides another vehicle-mounted terminal 100, see Figure 5, which is a schematic diagram of the hardware structure of a vehicle terminal for implementing various embodiments of the present application, the mobile vehicle terminal 100 may include: RF (Radio Frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 5 The structure of the vehicle-mounted terminal 100 shown in the figure does not constitute a limitation on the vehicle-mounted terminal. The vehicle-mounted terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0088] In one embodiment, the vehicle terminal 100 can be connected to an external 3D projection device, an interactive sensing device, etc.

[0089] Combine the following Figure 5 The various components of the vehicle-mounted terminal 100 are introduced in detail:

[0090] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or calls. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. Optionally, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution) and TDD-LTE (Time Division Duplexing-Long Term Evolution), etc.

[0091] WiFi is a short-range wireless transmission technology. The vehicle-mounted terminal 100 can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 102. It provides users with wireless broadband Internet access. Figure 5 The WiFi module 102 is shown, but it is understandable that it is not a necessary component of the vehicle terminal 100 and can be omitted as needed without changing the essence of the invention.

[0092] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile vehicle terminal 100 is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast receiving mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile vehicle terminal 100 (for example, a call signal receiving sound, a message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0093] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0094] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0095] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the vehicle terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. Optionally, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0096] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Then, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Figure 5 In the figure, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the vehicle terminal 100. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the vehicle terminal 100, which is not limited here.

[0097] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile vehicle terminal 100. For example, the external device may include a wired or wireless headset port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile vehicle terminal 100 or may be used to transmit data between the mobile vehicle terminal 100 and an external device.

[0098] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area. Optionally, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. Optionally, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0099] The processor 110 is the control center of the vehicle terminal. It uses various interfaces and lines to connect various parts of the entire vehicle terminal. By running or executing software programs and / or modules stored in the memory 109 and calling data stored in the memory 109, it executes various functions of the vehicle terminal and processes data, thereby monitoring the vehicle terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0100] although Figure 5 Not shown, the vehicle terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0101] The third embodiment of the present application also provides a vehicle, including the vehicle-mounted terminal described above. The present application provides a vehicle to interact with the user through a 3D interactive object, providing a novel way of human-vehicle interaction, which can greatly enhance the user's experience. Furthermore, when the user is watching a movie on the screen in the vehicle, a 3D interactive object can be added for movie-watching interaction to enhance the user's viewing pleasure in the car. Furthermore, the vehicle provided by the present application can also integrate AR equipment to perform operations such as display image adjustment on the 3D interactive object, creating a feeling of breaking through the barriers between reality and virtuality, and enhancing the user's experience. Furthermore, the present application can also intuitively and vividly display fault handling actions in the form of three-dimensional images through 3D interactive objects when the vehicle fails, which is convenient for users to learn fault handling methods and further enhance the user's experience.

[0102] The second embodiment of the present application further provides a computer-readable storage medium, which stores a computer program A6. When the computer program A6 is executed by the processor A101, the steps of the holographic projection interaction method described in the first embodiment are implemented. For example, Figure 1 10: Projecting a 3D interactive object in the car through holographic three-dimensional projection technology. S11: Sensing user operation information through an interactive sensing device, and / or receiving interactive information sent by an associated terminal. S13: Based on the user operation information and / or interactive information, controlling the 3D interactive object to perform interactive operations. Or, for example, Figure 2 Step S10 shown: projecting a 3D interactive object in the car through holographic three-dimensional projection technology. S10a: controlling the 3D interactive object to change its display form based on the interactive information output by the AR device after the virtual object shaping operation. S11a: sensing the user operation information through the interactive sensing device; S12a: controlling the 3D interactive object to perform interactive operation based on the user operation information. Or, for example, Figure 3 Step S10 shown: projecting a 3D interactive object in the car through holographic three-dimensional projection technology. S10b: associating with an associated terminal through Bluetooth communication technology; S11b: receiving interactive information sent by the associated terminal, the interactive information including image data; S12b: performing image projection operation based on the image data, and controlling the 3D interactive object to perform interactive viewing actions.

[0103] Optionally, the computer-readable storage medium provided by this embodiment may include any entity or device, recording medium that can carry computer program code, such as ROM, RAM, magnetic disk, optical disk, flash memory, etc.

[0104] The computer program A6 stored in the computer-readable storage medium provided in the second embodiment of the present application can interact with the user through 3D interactive objects when executed by the processor A101, providing a novel human-vehicle interaction method, which can greatly enhance the user's experience. Furthermore, when the user is watching a movie on the screen in the vehicle, a 3D interactive object can be added for movie watching interaction to enhance the user's viewing pleasure in the car. Furthermore, it is possible to integrate AR devices and perform operations such as adjusting the display image of 3D interactive objects to create a feeling of breaking through the barriers between reality and virtuality, thereby enhancing the user's experience. Furthermore, the present application can also realize that when a vehicle fails, the fault handling action is displayed in a three-dimensional image in an intuitive and vivid way through a 3D interactive object, which is convenient for users to learn the fault handling method and further enhance the user's experience.

[0105] Optionally, in the embodiments of the mobile terminal and the computer-readable storage medium provided in the present application, all technical features of the above-mentioned embodiments of the holographic projection interaction method are included, and the expansion and explanation content of the specification are basically the same as those of the above-mentioned embodiments of the holographic projection interaction method, and will not be repeated here.

[0106] The embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the methods in the above various possible implementation modes.

[0107] An embodiment of the present application also provides a chip, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods in various possible implementation modes as described above.

[0108] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0109] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0110] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0111] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0112] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0113] The various technical features of the technical solution of the present application can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0115] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium, (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)), etc.

[0116] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A holographic projection interaction method, characterized in that: include: Through holographic 3D projection technology, 3D interactive objects are projected inside the car; Sense user operation information through an interactive sensing device, and / or receive interactive information sent by an associated terminal; Based on the user operation information and / or the interaction information, the 3D interactive object is controlled to perform an interactive operation.

2. The holographic projection interaction method according to claim 1, characterized in that: The step of controlling the 3D interactive object to perform an interactive operation based on the user operation information and / or the interaction information includes: In response to the user operation information and / or the interaction information through chat robot technology, the 3D interactive object is controlled to perform action changes, and the action changes include at least one of facial expression changes, body movement changes, and lip movement changes.

3. The holographic projection interaction method according to claim 1, characterized in that: The associated terminal is an AR device; The step of controlling the 3D interactive object to perform an interactive operation based on the user operation information and / or the interaction information includes: Based on the interaction information output by the AR device, the 3D interactive object is controlled to change a display form.

4. The holographic projection interaction method according to claim 1, characterized in that: The interactive information includes image data or operation instructions; The step of controlling the 3D interactive object to perform an interactive operation based on the user operation information and / or the interaction information includes: Based on the image data, an image projection operation is performed, and the 3D interactive object is controlled to perform a movie viewing interactive action; or, Based on the operation instruction, the 3D interactive object is controlled to perform an action.

5. The holographic projection interaction method according to claim 1, characterized in that: The step of projecting a 3D interactive object in the vehicle by using holographic 3D projection technology includes: When it is detected that the vehicle is stationary, the 3D interactive object is projected inside the vehicle through holographic three-dimensional projection technology.

6. The holographic projection interaction method according to claim 1, characterized in that: The step of projecting a 3D interactive object in the vehicle by using holographic 3D projection technology includes: When a vehicle fault is detected, fault handling guidance information is obtained based on the fault attributes; Based on the fault handling guidance information, the 3D interactive object is projected in the vehicle through holographic three-dimensional projection technology to display the fault handling actions, and / or output corresponding voice prompts.

7. The holographic projection interaction method according to claim 6, characterized in that: The step of controlling the 3D interactive object to perform an interactive operation based on the user operation information and / or the interaction information includes: Based on the user operation information, a viewing adjustment is performed on the fault handling action displayed by the 3D interactive object, and the viewing adjustment includes at least one of pausing, zooming out, zooming in, fast forwarding, rewinding, and adjusting the angle.

8. A vehicle-mounted terminal, characterized in that: The vehicle-mounted terminal comprises: a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the holographic projection interaction method according to any one of claims 1 to 7 is implemented.

9. A vehicle, characterized in that: Comprising the vehicle-mounted terminal as claimed in claim 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the holographic projection interaction method according to any one of claims 1 to 7 is implemented.

Images