Human-machine interaction methods, devices and control systems for vehicle systems

Abstract

This application relates to a method, device, and control system for human-machine interaction in a vehicle system. By utilizing various in-vehicle devices and under the unified processing of the in-vehicle system software, a new approach to in-vehicle human-machine interaction is achieved. It introduces retrofittable holographic projection devices, which can be distributed at the desired display locations or centrally deployed at a central location where projection can reach the desired display area. In addition to utilizing common in-vehicle voice assistants and audio equipment, the introduction of holographic projection and other devices, combined with the increasingly powerful processing capabilities of in-vehicle systems, provides a more intuitive way to interact with the vehicle and offers greater interactive capabilities for subsequent in-vehicle entertainment experiences. For new or temporary car owners, this allows for quick understanding of in-vehicle functions, avoiding distractions caused by unfamiliarity with the equipment, which could seriously affect driving safety. Furthermore, the in-vehicle entertainment system offers a rich variety of interactive game experiences.

Description

Technical Field

[0001] This disclosure relates to the technical field, and in particular to a human-machine interaction method, apparatus and control system for an in-vehicle system. Background Technology

[0002] Currently, the most widely used form of human-computer interaction is through smart mobile terminals, such as smartphones and tablets, which are relied upon extensively for daily commutes. Limited by device size, while smartphones are increasingly achieving the so-called "full-screen" design, interaction methods have expanded from touch control to voice control, gestures, and other methods utilizing the device's external space. In the current era of rapid development of intelligent connected vehicles, in-vehicle infotainment systems still heavily borrow from mobile solutions. Even with the emphasis on voice control experience and priority while retaining traditional touch control, it essentially boils down to the interaction and experience of mobile voice assistants, and gestures and other operation methods are also forms of mobile interaction.

[0003] However, the in-vehicle environment differs from mobile devices. While mobile devices prioritize convenience, in the automotive field, with autonomous driving still in its early stages, the primary focus of the interactive experience is driving safety. This is also the area automakers consider most when designing in-vehicle infotainment systems. Lacking a better interaction method, automakers currently tacitly use paused interaction to address the conflict between in-vehicle infotainment and driving safety. For example, videos can be watched normally when parked, but automatically pause when driving (of course, the passenger and rear seat screens remain unaffected). In reality, in-vehicle interactions are often related to vehicle control. The feedback from the infotainment screen for these interactions also attracts the operator's attention, thus diverting focus from driving safety and failing to fundamentally change the in-vehicle interaction requirements prioritizing driving safety.

[0004] Especially for novice or temporary car owners, the inability to quickly understand the operation of the car's functions and the resulting distraction due to unfamiliarity with the equipment can seriously affect driving safety. In addition, the in-car entertainment system offers relatively few interactive games.

[0005] With the rapid development of the automotive industry in recent years, various hardware related to experience and interaction are being tested and implemented in vehicles. Besides in-vehicle infotainment screens, various extended display hardware, such as head-up displays (HUDs) and holographic projections, are also being integrated into the field of in-vehicle human-machine interaction. With the addition of more diverse and versatile hardware, solutions for in-vehicle human-machine interaction will undoubtedly be quickly implemented and presented to a wider range of users. Summary of the Invention

[0006] To address the aforementioned issues, this application proposes a human-machine interaction method, device, and control system for in-vehicle systems. By designing a novel human-machine interaction method using special hardware such as holographic projection in in-vehicle scenarios, it overcomes the shortcomings of traditional HMIs that display operation feedback on specific in-vehicle screens via touch or voice control, which negatively impacts driving safety. This provides a potentially safer and more intuitive path for various operations in in-vehicle scenarios.

[0007] This application proposes a human-machine interaction method for an in-vehicle system, characterized by comprising the following steps:

[0008] Predefined scenario trigger conditions are stored in the vehicle system;

[0009] Deploy holographic projection equipment in the vehicle and establish a connection with the vehicle system;

[0010] Obtain scene trigger information and determine whether the scene trigger information triggers the scene trigger condition:

[0011] If the scene triggering condition is triggered, a vehicle target image to be projected is generated and projected onto the corresponding position on the vehicle through the holographic projection device to guide the user's operation;

[0012] Awaiting user feedback.

[0013] As an optional implementation of this application, optionally, scenario triggering conditions are predefined and stored in the vehicle system, including:

[0014] Based on various types of data from the vehicle system, scene triggering conditions that are autonomously triggered by the vehicle are set and stored in the vehicle system; wherein, the scene triggering conditions are activated by the vehicle.

[0015] and / or

[0016] Based on the user's needs, a second scenario trigger condition is set and stored in the vehicle system; wherein the second scenario trigger condition is activated by the user.

[0017] As an optional implementation of this application, optionally, obtaining scene trigger information and determining whether the scene trigger information triggers the scene trigger condition includes:

[0018] Obtain the first scene trigger information sent from the vehicle system;

[0019] Determine whether the first scene trigger information triggers the first scene trigger condition:

[0020] If the first scenario triggering information triggers the first scenario triggering condition, the vehicle will autonomously issue a prompt message for the first scenario triggering information to prompt the user to perform an operation.

[0021] Waiting for the user to execute this prompt message.

[0022] As an optional implementation of this application, optionally, obtaining scene trigger information and determining whether the scene trigger information triggers the scene trigger condition includes:

[0023] The system obtains the second scenario trigger information sent by the user through the in-vehicle system;

[0024] Analyze and determine whether the trigger information of the second scene triggers the trigger condition of the second scene:

[0025] If the second scenario trigger information triggers the second scenario trigger condition, the vehicle system will send a prompt message to the user regarding the second scenario trigger information, prompting the user to perform an operation.

[0026] Waiting for the user to execute this prompt message.

[0027] As an optional implementation of this application, if the scene triggering condition is triggered, a vehicle-mounted target image to be projected is generated and projected onto the corresponding position on the vehicle using the holographic projection device to guide user operation, including:

[0028] Determine whether the user has executed the triggered scenario conditions:

[0029] If not executed, the operation ends;

[0030] If executed, the system obtains the user's confirmation instruction for the triggered scenario conditions and responds to the vehicle system.

[0031] The vehicle system receives the response and generates a vehicle target image to be projected based on the target information of the first scene trigger information or the second scene trigger information, and sends it to the holographic projection device.

[0032] The holographic projection device receives and projects the vehicle-mounted target image onto the target location of the vehicle-mounted device, guiding the user to operate according to the projection information of the target location.

[0033] As an optional implementation of this application, optionally, user feedback is awaited, including:

[0034] Confirm whether the user is operating manually:

[0035] If the user manually operates the on-board equipment at the target location, the control of the on-board equipment at the target location will be completed manually.

[0036] If the user does not manually operate the on-board equipment at the target location, the on-board system will execute and complete the control of the on-board equipment at the target location.

[0037] Execution complete. Awaiting feedback on the results.

[0038] As an optional implementation of this application, optionally, after the user completes the execution, it also includes:

[0039] Once the user has completed the task, the vehicle system receives the execution feedback signal and returns the execution feedback signal to the holographic projection device.

[0040] The holographic projection device receives the execution feedback signal, generates a corresponding execution result projection image, and projects the execution result projection image onto the vehicle system;

[0041] Display the execution results to the user.

[0042] As an optional implementation of this application, after displaying the execution results to the user, the method may further include:

[0043] The user determines whether the execution was correct based on the execution result;

[0044] If correct, execution ends;

[0045] If an error occurs, repeat the above steps for implementing the projection instruction.

[0046] In another aspect, this application proposes an apparatus for implementing the above-described in-vehicle system human-machine interaction method, comprising:

[0047] The scene definition module is used to predefine scene triggering conditions and save them in the vehicle system;

[0048] An association module is used to deploy holographic projection equipment on the vehicle and establish an association with the vehicle system;

[0049] The trigger determination module is used to obtain scene trigger information and determine whether the scene trigger information triggers the scene trigger condition.

[0050] If the scene triggering condition is triggered, a vehicle target image to be projected is generated and projected onto the corresponding position on the vehicle through the holographic projection device to guide the user's operation;

[0051] The feedback module is used to wait for user feedback.

[0052] In another aspect, this application also proposes a control system, comprising:

[0053] processor;

[0054] Memory used to store processor-executable instructions;

[0055] The processor is configured to implement the above-described in-vehicle system human-machine interaction method when executing the executable instructions.

[0056] Technical effects of the present invention:

[0057] This application utilizes various in-vehicle devices and, under the unified processing of the vehicle's infotainment system software, achieves a new method of in-vehicle human-machine interaction. It introduces retrofittable holographic projection equipment, which can be distributed at the desired display locations or centrally deployed in a central location that can project onto the desired display area. In addition to utilizing common in-vehicle voice assistants and audio equipment, the introduction of holographic projection and other devices, combined with the increasingly powerful processing capabilities of the vehicle's infotainment system, provides a more intuitive way for in-vehicle interaction and lays a foundation for more interactive experiences in subsequent in-vehicle entertainment. For new or temporary car owners, this allows for quick understanding of in-vehicle functions, avoiding distractions caused by unfamiliarity with the equipment, and preventing serious safety hazards. Furthermore, the in-vehicle entertainment system offers a wealth of interactive game experiences.

[0058] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0059] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0060] Figure 1 The diagram illustrates the implementation flow of the human-computer interaction method for the vehicle system of the present invention.

[0061] Figure 2 This diagram illustrates the scene interaction process for triggering the closing of the rear left side window in this embodiment.

[0062] Figure 3 This diagram illustrates how trigger information can be obtained via a voice assistant.

[0063] Figure 4 The diagram shows a holographic projection onto the left rear window.

[0064] Figure 5 The diagram shows a simulation of the closing of the left rear window on the system. Detailed Implementation

[0065] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0066] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0067] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0068] This technology, in addition to utilizing common in-vehicle voice assistants and audio equipment, introduces devices such as holographic projection. Combined with the increasingly powerful processing capabilities of in-vehicle infotainment systems, it provides a more intuitive way to interact with the vehicle and lays a foundation for more interactive experiences in subsequent in-vehicle entertainment. For new or temporary car owners, it allows them to quickly learn how to operate the vehicle's functions, avoiding distractions caused by unfamiliarity with the equipment, which could seriously affect driving safety. Furthermore, the in-vehicle entertainment system offers a wealth of interactive game experiences.

[0069] In this embodiment, when a user needs to operate a certain in-vehicle device / hardware, the user will send information to the vehicle platform. After parsing the information, the target location of the target in-vehicle device / hardware is obtained, and a corresponding projection image is generated. This projection image will be transmitted by a holographic projection device deployed on the vehicle to the corresponding target location, prompting the user to find the operation location of the in-vehicle device and allowing the user to quickly find the accurate location of the device / control key.

[0070] The generation of the projected image from the holographic projection device, as well as the target location information, will be calculated by the in-vehicle system or the cloud-based vehicle infotainment system. The choice of holographic projection device is determined by the user; the method by which the cloud server calculates and generates the target location information is implemented using a specific algorithm. For example, if the user voice-instructs them to open the left-side window, the left-side window's location information will be stored in the cloud during scene definition. After voice interpretation, the cloud retrieves the left-side window's location information based on the voice information and sends it to the holographic projection device, which then projects the image based on that location.

[0071] The projected image can be a voice assistant, which can play audio messages such as "The left window is here".

[0072] Example 1

[0073] This invention utilizes various in-vehicle devices to achieve a new method of in-vehicle human-machine interaction under the unified processing of the vehicle's infotainment system software. Existing in-vehicle devices that can be utilized include the in-vehicle infotainment system's voice assistant for collecting sound, and audio equipment located in various corners of the vehicle. Voice assistants are a basic feature of intelligent connected vehicles, while the number and distribution of audio equipment vary depending on the vehicle model. Even with a smaller number of audio devices, most interactive effects can be achieved.

[0074] This invention introduces a holographic projection device that can be mounted on the device, or it can be distributed at the location where it needs to be displayed, or it can be centrally deployed at the middle position where it can be projected onto the location where it needs to be displayed.

[0075] like Figure 1 As shown, this application proposes a human-machine interaction method for an in-vehicle system, comprising the following steps:

[0076] S1. Predefine scene trigger conditions and save them in the vehicle system;

[0077] First, there's scene triggering. Scene triggering can be initiated by passengers, including touchscreen activation, voice activation, etc., or it can be triggered autonomously by the vehicle's own status or data, such as excessively high or low temperature, or excessive ambient noise.

[0078] Secondly, there are scenario suggestions. Similarly, scenario suggestions can come directly from the people in the car or from the vehicle's infotainment system, and can be completed through interaction between the voice assistant and the people in the car.

[0079] As an optional implementation of this application, optionally, scenario triggering conditions are predefined and stored in the vehicle system, including:

[0080] Based on various types of data from the vehicle system, scene triggering conditions that are autonomously triggered by the vehicle are set and stored in the vehicle system; wherein, the scene triggering conditions are activated by the vehicle.

[0081] and / or

[0082] Based on the user's needs, a second scenario trigger condition is set and stored in the vehicle system; wherein the second scenario trigger condition is activated by the user.

[0083] Scene triggers can be triggered by feedback from the vehicle's infotainment system, such as if the rear left-side window has been left open for an extended period, in which case the infotainment system will activate the scene trigger conditions for that "rear left-side window". Alternatively, they can be trigger scenarios set based on user needs, such as user control of the windows or air conditioning.

[0084] The specific scenario definition is set by the user on the vehicle's infotainment system, or by the default definition set by the manufacturer when the vehicle's infotainment system is shipped.

[0085] The vehicle infotainment system implements a scene engine function in the cloud, which can preset trigger thresholds based on parameters obtained from on-vehicle sensors, and also provides a user editing entry on the vehicle infotainment application interface to customize the trigger thresholds.

[0086] The scene engine consists of two parts: scene trigger conditions and the actions to be performed after triggering. When the voice assistant collects a scene trigger command, such as "open the rear left window," it matches the scene trigger conditions defined in the scene engine to maximize the match (i.e., a complete match is better than a partial match). After the scene is matched, the system calls the operation interface according to the trigger action defined for the corresponding scene. Taking "open the rear left window" as an example, the system will then open the rear left window.

[0087] The above-mentioned package can be used to upgrade the vehicle infotainment system or to be embedded independently as an SDK application.

[0088] S2. Deploy a holographic projection device on the vehicle and establish a connection with the vehicle system;

[0089] The deployment location and number of holographic projection devices are not limited in this embodiment, but are determined by the defined number of scenes and the corresponding scene trigger execution locations.

[0090] The holographic projection device is linked to the vehicle's in-vehicle system. When the in-vehicle system determines a triggered scenario, such as opening the rear left-side window, the cloud retrieves the projection target of the rear left-side window and sends it to the holographic projection device.

[0091] In the scene definition, the location information of each scene will be established and saved under holographic projection coordinates.

[0092] S3. Obtain scene trigger information and determine whether the scene trigger information triggers the scene trigger condition:

[0093] If the scene triggering condition is triggered, a vehicle target image to be projected is generated and projected onto the corresponding position on the vehicle through the holographic projection device to guide the user's operation;

[0094] The scene trigger information can be triggered automatically by the vehicle's infotainment system, such as when the interior temperature is too high, triggering the scene condition for opening the rear left-side window. At this time, the system will project a message onto the rear left-side window, prompting the user to open it.

[0095] As an optional implementation of this application, optionally, obtaining scene trigger information and determining whether the scene trigger information triggers the scene trigger condition includes:

[0096] Obtain the first scene trigger information sent from the vehicle system;

[0097] Determine whether the first scene trigger information triggers the first scene trigger condition:

[0098] If the first scenario triggering information triggers the first scenario triggering condition, the vehicle will autonomously issue a prompt message for the first scenario triggering information to prompt the user to perform an operation.

[0099] Waiting for the user to execute this prompt message.

[0100] The above describes a scenario triggered automatically by the in-vehicle system (e.g., the rear left-side window not being closed for an extended period). When the in-vehicle system detects a defined scenario, it issues a first scenario trigger message (e.g., the rear left-side window not being closed for more than 3 hours) and determines whether the system's custom first scenario trigger condition (e.g., the rear left-side window not being closed for more than 3 hours) has been triggered. At this point, the system issues a "rear left-side window not closed for more than 3 hours" prompt, reminding the user to close the rear left-side window. The user then chooses whether to close it based on this prompt. During the prompt phase, the holographic projection device projects the voice assistant onto the marked position on the rear left-side window based on its projection coordinates, prompting the user.

[0101] As an optional implementation of this application, optionally, obtaining scene trigger information and determining whether the scene trigger information triggers the scene trigger condition includes:

[0102] The system obtains the second scenario trigger information sent by the user through the in-vehicle system;

[0103] Analyze and determine whether the trigger information of the second scene triggers the trigger condition of the second scene:

[0104] If the second scenario trigger information triggers the second scenario trigger condition, the vehicle system will send a prompt message to the user regarding the second scenario trigger information, prompting the user to perform an operation.

[0105] Waiting for the user to execute this prompt message.

[0106] When a user actively intervenes, the voice assistant can detect the user's intention, such as "close the left rear window." The user issues a voice message, which is then parsed and the user's intention is recognized. The trigger condition for this second scenario can be customized by the user; the trigger information is the user information obtained through the voice assistant, such as "close the left rear window." For details on the projection method, please refer to the above-mentioned in-vehicle infotainment system projection.

[0107] As an optional implementation of this application, if the scene triggering condition is triggered, a vehicle-mounted target image to be projected is generated and projected onto the corresponding position on the vehicle using the holographic projection device to guide user operation, including:

[0108] Determine whether the user has executed the triggered scenario conditions:

[0109] If not executed, the operation ends;

[0110] If executed, the system obtains the user's confirmation instruction for the triggered scenario conditions and responds to the vehicle system.

[0111] The vehicle system receives the response and generates a vehicle target image to be projected based on the target information of the first scene trigger information or the second scene trigger information, and sends it to the holographic projection device.

[0112] The holographic projection device receives and projects the vehicle-mounted target image onto the target location of the vehicle-mounted device, guiding the user to operate according to the projection information of the target location.

[0113] S4. Wait for user feedback.

[0114] As an optional implementation of this application, optionally, user feedback is awaited, including:

[0115] Confirm whether the user is operating manually:

[0116] If the user manually operates the on-board equipment at the target location, the control of the on-board equipment at the target location will be completed manually.

[0117] If the user does not manually operate the on-board equipment at the target location, the on-board system will execute and complete the control of the on-board equipment at the target location.

[0118] Execution complete. Awaiting feedback on the results.

[0119] As an optional implementation of this application, optionally, after the user completes the execution, it also includes:

[0120] Once the user has completed the task, the vehicle system receives the execution feedback signal and returns the execution feedback signal to the holographic projection device.

[0121] The holographic projection device receives the execution feedback signal, generates a corresponding execution result projection image, and projects the execution result projection image onto the vehicle system;

[0122] Display the execution results to the user.

[0123] As an optional implementation of this application, after displaying the execution results to the user, the method may further include:

[0124] The user determines whether the execution was correct based on the execution result;

[0125] If correct, execution ends;

[0126] If an error occurs, repeat the above steps for implementing the projection instruction.

[0127] Taking closing the left rear window as an example, as shown in the attached document... Figure 2 As shown, this is the scene interaction flow that triggers the action of closing the left rear window.

[0128] For detailed procedures, please refer to the appendix. Figure 2 .

[0129] like Figure 3 As shown, once the vehicle assistant learns that the user has confirmed closing the left rear window, it will display "Close the left rear window" on the vehicle system display screen via voice assistant.

[0130] like Figure 4 As shown, the image of the voice assistant is projected onto the operating terminal through holographic projection. The projection process can include voice, sound effects, and animations to prompt the user with the target projection location and corresponding operation instructions.

[0131] Regardless of whether the actual operation is performed by the people in the vehicle or by the system control hardware, the real-time status can be presented through the vehicle's infotainment screen or voice assistant.

[0132] like Figure 5 As shown, if the user performs a hardware operation corresponding to the target location, the vehicle system will synchronously display the action process of the target object. For example, using the existing simulation display technology in vehicles, when the left door is opened, the vehicle's 360-degree imaging system will simulate and display the image of the left door opening, prompting the user with the real-time status of the device / hardware at the target location.

[0133] Similarly, when a user operates on a target device / hardware based on the target location of the transmitted signal, the vehicle system can display the corresponding target device / hardware's actions in real time.

[0134] After the operation is complete, the voice assistant image can also be projected back onto the car's infotainment screen or main control panel via holographic projection. The projection process can include voice, sound effects, and animations.

[0135] Finally, there's feedback information. This involves providing feedback on the operation results or the device status after completion. This can be displayed on the vehicle's infotainment screen or communicated to passengers via voice assistant.

[0136] If the user discovers an operational error during operation, the above steps can be repeated to prompt the user to perform the operation again. Specific steps are described above.

[0137] Therefore, this application combines the vehicle infotainment system with holographic projection equipment to show users, especially novices, the specific locations of in-vehicle devices / hardware. For novice or temporary car owners, this allows for quick understanding of vehicle functions and reduces distractions caused by unfamiliarity with the equipment, which can seriously affect driving safety. Furthermore, it provides more interactive gaming experiences for the in-vehicle entertainment system.

[0138] It should be noted that although the above simulation demonstration uses closing the rear left-side window as an example, those skilled in the art will understand that this disclosure is not limited to this. In fact, users can flexibly set up scenarios according to actual application scenarios, as long as the technical functions of this application can be achieved by following the above technical methods.

[0139] Example 2

[0140] Based on the implementation principle of Embodiment 1, this application, in another aspect, proposes an apparatus for implementing the above-described in-vehicle system human-machine interaction method, comprising:

[0141] The scene definition module is used to predefine scene triggering conditions and save them in the vehicle system;

[0142] An association module is used to deploy holographic projection equipment on the vehicle and establish an association with the vehicle system;

[0143] The trigger determination module is used to obtain scene trigger information and determine whether the scene trigger information triggers the scene trigger condition.

[0144] If the scene triggering condition is triggered, a vehicle target image to be projected is generated and projected onto the corresponding position on the vehicle through the holographic projection device to guide the user's operation;

[0145] The feedback module is used to wait for user feedback.

[0146] For the application principles and interaction methods of the above modules, please refer to the description in Example 1.

[0147] Obviously, those skilled in the art should understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the control methods described above. The modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, the present invention is not limited to any specific hardware and software combination.

[0148] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the control methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0149] Example 3

[0150] Furthermore, this application also proposes a control system, comprising:

[0151] processor;

[0152] Memory used to store processor-executable instructions;

[0153] The processor is configured to implement the above-described in-vehicle system human-machine interaction method when executing the executable instructions.

[0154] This disclosure discloses an embodiment of a control system including a processor and a memory for storing processor-executable instructions. The processor is configured to implement any of the preceding vehicle system human-machine interaction methods when executing the executable instructions.

[0155] It should be noted here that the number of processors can be one or more. Furthermore, the control system in this embodiment may also include input devices and output devices. The processors, memory, input devices, and output devices can be connected via a bus or other means, without specific limitations herein.

[0156] As a computer-readable storage medium, the memory can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the human-machine interaction method of an in-vehicle system according to an embodiment of this disclosure. The processor executes various functional applications and data processing of the control system by running the software program or module stored in the memory.

[0157] Input devices can be used to receive input digital numbers or signals. These signals can be key signals related to user settings and function control of the device / terminal / server. Output devices can include display devices such as screens.

[0158] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A human-machine interaction method for an in-vehicle system, characterized in that, Includes the following steps: Predefined scenario trigger conditions are stored in the vehicle system; Deploy holographic projection equipment in the vehicle and establish a connection with the vehicle system; Acquire scene trigger information and determine whether the scene trigger information triggers the scene trigger condition: if the scene trigger condition is triggered, generate a vehicle-mounted target image to be projected and project the vehicle-mounted target image onto the corresponding position on the vehicle through the holographic projection device to guide user operation; wherein, acquiring scene trigger information and determining whether the scene trigger information triggers the scene trigger condition includes: acquiring first scene trigger information sent from the vehicle system; determining whether the first scene trigger information triggers the first scene trigger condition: if the first scene trigger information triggers the first scene trigger condition, the vehicle autonomously issues a response to the first scene trigger condition. The system triggers a message prompting the user to take action; it waits for the user to execute the prompt; when the vehicle system detects a defined scenario, it issues a first scenario trigger message indicating that the rear left window has been open for more than 3 hours, and determines whether the vehicle system's custom first scenario trigger condition of "rear left window open for more than 3 hours" has been triggered. At this time, the vehicle system issues a "rear left window open for more than 3 hours" prompt message, reminding the user to close the rear left window; the user then chooses whether to close it based on the prompt; during the prompt phase, the holographic projection device projects the voice assistant onto the marked position of the rear left window based on the projection position coordinates of the rear left window, prompting the user. Waiting for user feedback: When a user operates a certain in-vehicle device / hardware, the information will be sent to the vehicle's infotainment platform. After the information is parsed, the target location of the in-vehicle device / hardware is obtained, and a corresponding projection image is generated. This projection image will be projected by a holographic projection device deployed in the vehicle onto the corresponding target location, prompting the user to find the operation location of the in-vehicle device and allowing the user to quickly find the accurate location of the device / control button.

2. The in-vehicle system human-machine interaction method according to claim 1, characterized in that, Predefined scenario trigger conditions are stored in the vehicle system, including: Based on various types of data from the vehicle system, scene triggering conditions that are autonomously triggered by the vehicle are set and stored in the vehicle system; wherein, the scene triggering conditions are activated by the vehicle. and / or Based on the user's needs, a second scenario trigger condition is set and stored in the vehicle system; wherein the second scenario trigger condition is activated by the user.

3. The in-vehicle system human-machine interaction method according to claim 2, characterized in that, Obtaining scene trigger information and determining whether the scene trigger information triggers the scene trigger condition includes: The system obtains the second scenario trigger information sent by the user through the in-vehicle system; Analyze and determine whether the trigger information of the second scene triggers the trigger condition of the second scene: If the second scenario trigger information triggers the second scenario trigger condition, the vehicle system will send a prompt message to the user regarding the second scenario trigger information, prompting the user to perform an operation. Waiting for the user to execute this prompt message.

4. The in-vehicle system human-machine interaction method according to claim 1, characterized in that, If the scene trigger condition is triggered, a vehicle-mounted target image to be projected is generated and projected onto the corresponding position on the vehicle via the holographic projection device to guide user operation, including: Determine whether the user has executed the triggered scenario conditions: If not executed, the operation ends; If executed, the system obtains the user's confirmation instruction for the triggered scenario conditions and responds to the vehicle system. The vehicle system receives the response and generates a vehicle target image to be projected based on the target information of the first scene trigger information or the second scene trigger information, and sends it to the holographic projection device. The holographic projection device receives and projects the vehicle-mounted target image onto the target location of the vehicle-mounted device, guiding the user to operate according to the projection information of the target location.

5. The in-vehicle system human-machine interaction method according to claim 4, characterized in that, Waiting for user feedback, including: Confirm whether the user is operating manually: If the user manually operates the on-board equipment at the target location, the control of the on-board equipment at the target location will be completed manually. If the user does not manually operate the on-board equipment at the target location, the on-board system will execute and complete the control of the on-board equipment at the target location. Execution complete. Awaiting feedback on the results.

6. The in-vehicle system human-machine interaction method according to claim 5, characterized in that, After the user finishes executing the command, the following is also included: Once the user has completed the task, the vehicle system receives the execution feedback signal and returns the execution feedback signal to the holographic projection device. The holographic projection device receives the execution feedback signal, generates a corresponding execution result projection image, and projects the execution result projection image onto the vehicle system; Display the execution results to the user.

7. The in-vehicle system human-machine interaction method according to claim 6, characterized in that, After displaying the execution results to the user, it also includes: The user determines whether the execution was correct based on the execution result; If correct, execution ends; If an error is found, the implementation steps of claims 5 to 6 shall be repeated.

8. An apparatus for implementing the human-machine interaction method of an in-vehicle system according to any one of claims 1-7, characterized in that, include: The scene definition module is used to predefine scene triggering conditions and save them in the vehicle system; An association module is used to deploy holographic projection equipment on the vehicle and establish an association with the vehicle system; The trigger determination module is used to obtain scene trigger information and determine whether the scene trigger information triggers the scene trigger condition. If the scene triggering condition is triggered, a vehicle target image to be projected is generated and projected onto the corresponding position on the vehicle through the holographic projection device to guide the user's operation; The feedback module is used to wait for user feedback.

9. A control system, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the in-vehicle system human-machine interaction method according to any one of claims 1-7 when executing the executable instructions.

Images