Method for in-vehicle voice interaction, and related device
By creating a system user account and an independent voice process for each voice zone, and combining this with the central adapter to handle requests, the interaction conflicts and resource consumption issues of multi-voice zone in-vehicle voice systems are resolved, achieving independent voice interaction and efficient resource utilization.
Patent Information
- Application Number
- PCT/CN2025/142107
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2025-12-12
- Publication Date
- 2026-11-19
AI Technical Summary
Existing in-vehicle voice systems cannot achieve independent voice interaction in multi-voice zone environments, resulting in interaction conflicts and excessive resource consumption, and failing to meet the needs of multiple passengers to perform voice operations simultaneously.
By creating system user accounts for each independent voice zone and starting an independent voice process for each user, the central adapter process handles cross-user collaboration requests and in-vehicle device control requests, enabling independent voice interaction for each voice zone and optimizing resource usage through user status detection.
It enables independent voice interaction capabilities for each voice region, reducing system complexity and resource consumption, and improving the stability and response speed of voice interaction.
Smart Images

Figure CN2025142107_19112026_PF_FP_ABST
Abstract
Description
In-vehicle voice interaction methods and related equipment
[0001] Related applications
[0002] This application claims priority to Chinese patent application filed on May 15, 2025, with application number 2025106231667, entitled "In-vehicle Voice Interaction Method and Related Device", the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to the field of intelligent driving technology, and more specifically, to an in-vehicle voice interaction method and related equipment. Background Technology
[0004] With the continuous development of smart cockpit technology, in-vehicle voice interaction systems, as an important component for enhancing user experience, have been widely applied in modern automobiles. In-vehicle voice systems are not only used for navigation and entertainment control, but also play a vital role in various fields such as in-vehicle smart homes and driver assistance. To improve the voice interaction experience for multiple passengers in the vehicle, especially in complex environments with multiple seats and voice zones, the independence and accuracy of in-vehicle voice interaction systems have become important technical requirements.
[0005] However, many current in-vehicle voice systems still suffer from the technical challenge of preventing independent voice interaction between users in different seating areas. Most systems employ a centralized voice recognition process, where all voice requests from different seating areas are processed by a single voice engine. This approach often leads to interaction conflicts or delays, failing to meet the needs of multiple passengers interacting simultaneously. Particularly in multi-zone scenarios, current systems fail to adequately address the independence of each zone, often preventing passengers from performing voice operations without disturbing others. Furthermore, these technologies typically require creating independent voice engines and resource instances for each zone, resulting in high system complexity, high resource consumption, and difficulty in maintaining efficient and stable operation in practical applications. In short, these technologies suffer from insufficient multi-zone independent voice interaction capabilities and high resource consumption. Summary of the Invention
[0006] The summary section of this application introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. The summary section of this application is not intended to limit the key features and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.
[0007] The in-vehicle voice interaction method and related equipment provided in this application can utilize system user accounts, independent voice processes and hardware device mapping to enable each independent voice zone to have independent voice interaction capabilities, while reducing the implementation complexity and resource consumption of the in-vehicle voice interaction method.
[0008] Firstly, this application provides an in-vehicle voice interaction method applied to a vehicle. The vehicle has multiple independent audio zones, each with a corresponding system user account, to manage the resources of the corresponding independent audio zone through the system user account, enabling independent voice interaction for the corresponding independent audio zone. The in-vehicle voice interaction method includes: obtaining the system user account corresponding to the independent audio zone; responding to the dedicated startup broadcast of the system user account, performing user status detection on the independent audio zone corresponding to the system user account; when the user status is active, starting the independent voice process corresponding to the system user account; and responding through the independent voice process when a hardware device interaction signal corresponding to the independent voice process is received.
[0009] In some implementations, responding through the independent voice process when a hardware device interaction signal corresponding to the independent voice process is received includes: when a hardware device interaction signal corresponding to the independent voice process is received, sending the hardware device interaction signal to the central adapter process through the independent voice process, wherein the central adapter process is simultaneously connected to the independent voice process and the vehicle device control module through a bidirectional communication link; and processing cross-user collaboration requests or vehicle device control requests attached to the hardware device interaction signal through the central adapter process.
[0010] In some implementations, the process of handling cross-user collaboration requests or in-vehicle device control requests attached to the hardware device interaction signals by the central adapter process includes: when the hardware device interaction signal is accompanied by the cross-user collaboration request, the central adapter process performs asynchronous task scheduling for the cross-user collaboration request through a message queue; when the hardware device interaction signal is accompanied by the in-vehicle device control request, the central adapter process forwards the in-vehicle device control request to the in-vehicle device control module.
[0011] In some embodiments, the in-vehicle voice interaction method further includes: performing user status detection on the independent voice zone corresponding to the system user account at preset time intervals; and terminating the independent voice process corresponding to the system user account when the user status is empty.
[0012] In some implementations, the user status detection of the independent audio zone corresponding to the system user account includes: when the vehicle screen, pressure sensor or biometric sensor in the independent audio zone corresponding to the system user account is in a passively activated state, determining that the user status is present; otherwise, determining that the user status is absent.
[0013] In some implementations, after starting an independent voice process corresponding to the system user account when the user state is present, the in-vehicle voice interaction method further includes: creating an independent wake-up engine, a speech recognition engine, a text-to-speech engine, and a visual graphical interface module corresponding to the independent voice process.
[0014] In some embodiments, the in-vehicle voice interaction method further includes: receiving a data packet sent by a first independent voice process and forwarding the data packet to a second independent voice process, so that the second independent voice process can display the information in the data packet based on the visual graphical interface module corresponding to the second independent voice process; wherein the first independent voice process and the second independent voice process are independent voice processes corresponding to different system user accounts.
[0015] In some embodiments, the in-vehicle voice interaction method further includes: switching the context mode of multiple independent voice zones according to the topics of conversation among users in the vehicle; wherein the context mode includes a shared mode and a non-shared mode.
[0016] In some embodiments, the in-vehicle voice interaction method further includes: when multiple independent voice processes corresponding to multiple system user accounts are started, detecting whether multiple independent voice processes receive hardware device interaction signals for the same in-vehicle device within a preset time period; if so, arbitrating the hardware device interaction signals corresponding to the multiple independent voice processes; and processing the hardware device interaction signals corresponding to the multiple independent voice processes based on the arbitration result.
[0017] In some implementations, processing the hardware device interaction signals corresponding to the multiple independent voice processes based on the arbitration result includes: determining the target independent voice process from the multiple independent voice processes based on the arbitration result; sending the corresponding hardware device interaction signal to the central adapter process through the target independent voice process; and processing cross-user collaboration requests or in-vehicle device control requests attached to the hardware device interaction signal through the central adapter process.
[0018] In some implementations, determining the target independent voice process from among the multiple independent voice processes based on the arbitration result includes: if the arbitration result indicates that the hardware device interaction signals corresponding to the multiple independent voice processes do not conflict, then all of the multiple independent voice processes are taken as the target independent voice process.
[0019] In some implementations, determining the target independent voice process from among the multiple independent voice processes based on the arbitration result includes: if the arbitration result indicates a conflict in the hardware device interaction signals corresponding to the multiple independent voice processes, then the independent voice process with the highest priority is selected as the target independent voice process.
[0020] Secondly, this application also provides an in-vehicle voice interaction device applied to a vehicle. The vehicle has multiple independent audio zones, each with a corresponding system user account, to manage the resources of the corresponding independent audio zone through the system user account, enabling independent voice interaction for the corresponding independent audio zone. The in-vehicle voice interaction device includes: an account acquisition unit for acquiring the system user account corresponding to the independent audio zone; a process control unit for responding to the dedicated startup broadcast of the system user account and performing user status detection on the independent audio zone corresponding to the system user account; when the user status is active, starting the independent voice process corresponding to the system user account; and a voice interaction unit for responding through the independent voice process when receiving a hardware device interaction signal corresponding to the independent voice process.
[0021] Thirdly, this application also provides an electronic device, including: a memory and a processor, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of the in-vehicle voice interaction method described in the first aspect.
[0022] Fourthly, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the in-vehicle voice interaction method described in the first aspect.
[0023] Fifthly, this application also provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implement the in-vehicle voice interaction method provided in the embodiments of this application.
[0024] In summary, this application creates multiple system user accounts and launches an independent voice process for each user, enabling passengers in each seat to use voice interaction independently, unaffected by other passengers. Furthermore, leveraging the independence of system user accounts ensures that the voice processes in each audio region are isolated, each handling its own voice requests, thereby simplifying system logic and improving the stability and maintainability of the in-vehicle voice interaction method. Additionally, user status detection is performed on the independent audio regions corresponding to system user accounts. When a user is present, the independent voice process corresponding to that user account is launched, avoiding the launch of independent voice processes when no user is present, thus reducing resource consumption. In conclusion, the in-vehicle voice interaction method provided by this application utilizes system user accounts, independent voice processes, and hardware device mapping, enabling each independent audio region to have independent voice interaction capabilities, while simultaneously reducing the implementation complexity and resource consumption of the in-vehicle voice interaction method. Attached Figure Description
[0025] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0026] Figure 1 is a flowchart illustrating an in-vehicle voice interaction method provided in an embodiment of this application;
[0027] Figure 2 is a schematic diagram of the composition structure of an in-vehicle voice interaction device provided in an embodiment of this application;
[0028] Figure 3 is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0029] The terms used in the specification, claims, and drawings of this application, such as "first," "second," "third," "fourth," etc. (if any), are used to distinguish similar objects and not to describe a specific order or sequence. Therefore, it is to be understood that these terms can be used interchangeably where appropriate, allowing the described embodiments to be used in different orders, unless specifically required by the illustrations or description. Furthermore, the terms "is" and "has," and any variations thereof, are intended to cover, non-exclusively, all possible constituent elements. For example, a process, method, system, product, or apparatus comprising several steps or units is not necessarily limited to the steps or units explicitly listed, but may also include other steps or units not explicitly listed, or steps or units inherent to the process, method, product, or apparatus.
[0030] In this application, a "module" or "unit" refers to a computer program or part of a computer program that has a specific function and works in conjunction with other related parts to achieve a predetermined goal. These modules or units can be implemented by software, hardware (e.g., processing circuitry or memory), or a combination of both. One or more processors or memories can implement one or more modules or units. Furthermore, each module or unit can also be part of a larger module or unit.
[0031] The technical solutions of this application will be described in detail below with reference to the accompanying drawings of the embodiments. It should be noted that the described embodiments are only a part of this application, and not all embodiments. In the following description, the "some embodiments" mentioned are only a subset of all possible embodiments, which may be the same or different subsets, and different embodiments can be combined with each other without conflict.
[0032] Figure 1 is a flowchart illustrating an in-vehicle voice interaction method provided in an embodiment of this application. For example, referring to Figure 1, the in-vehicle voice interaction method provided in this embodiment is applied to a vehicle with multiple independent voice zones. Each independent voice zone has a corresponding system user account, allowing the system user account to manage the resources of the corresponding independent voice zone, enabling independent voice interaction in the corresponding independent voice zone. The method may include the following steps 101 to 103:
[0033] Step 101: Obtain the system user account corresponding to the independent audio zone.
[0034] In some examples, independent voice zones are different areas or seating areas within the vehicle. Each area uses different hardware, sensors, or audio input devices for independent voice interaction, and there are at least two independent voice zones. For example, each seat or area, such as the driver's seat, front passenger seat, and rear seats, can be defined as an independent voice zone. Each independent voice zone has its own dedicated voice recognition, processing, and response mechanism. For instance, the driver's seat can be defined as voice zone 0, the front passenger seat as voice zone 1, and the rear seats as voice zone 2, etc. System user accounts are virtual user accounts created for each independent voice zone within the in-vehicle operating system (such as Android Automotive). These accounts manage all interactive settings, voice processes, and user personalization information for that voice zone. Each system user account operates independently and manages its corresponding voice interaction process based on the different seating areas or voice zones within the vehicle. For example, the system user account for the driver's area can be named "User_0," the front passenger's system user account can be named "User_1," and the rear seat system user account can be named "User_2," etc. Creating independent system user accounts for each independent voice zone isolates the voice processes, recognition engines, and hardware resources of each seat, ensuring that voice interaction in each independent voice zone can proceed independently. This effectively avoids voice conflicts between multiple seats in traditional in-vehicle voice systems, improving the user's voice interaction experience. The process of obtaining the system user account corresponding to each independent voice zone can proceed in a preset order, ensuring that each independent voice zone can wake up its corresponding independent voice process at the correct time and perform voice interaction according to the user's needs. For example, when the vehicle starts, the driver's system user account (User_0) is logged in first, followed by the front passenger's system user account (User_1), and finally the rear passenger's system user account (User_2).
[0035] By implementing step 101, the system user account corresponding to each independent voice zone can load the required voice process and resources separately for each voice zone, avoiding interference between multiple voice processes and ensuring that the voice system in different seats can be accurately started and meet their respective needs.
[0036] Step 102: In response to the dedicated startup broadcast of the system user account, perform user status detection on the independent audio zone corresponding to the system user account; when the user status is present, start the independent voice process corresponding to the system user account.
[0037] In some examples, an independent voice process is a dedicated voice recognition and interaction processing flow launched for each independent voice zone (each system user account). Each independent voice process can independently handle voice commands, recognition requests, speech synthesis, and other tasks within its independent voice zone, without sharing resources with processes in other independent voice zones, ensuring that interactions in each zone do not interfere with each other. For example, when the "User_0" voice zone in the driver's area receives the power-on broadcast, an independent voice process, such as process ID "Voice_Process_0," will be launched, performing voice recognition and response only in the driver's area; the passenger's area and the rear seat area will launch their respective independent processes.
[0038] For example, when the vehicle starts, it reads the preset audio zone configuration (e.g., 4-zone) and calls the AndroidAutomotive UserManager interface to create User0-User3 accounts. Each account is associated with the hardware ID of the corresponding audio zone, such as the driver's microphone hardware address audio_dev_001. After detecting the passenger seat pressure sensor signal, it logs into the User1 account and sends a boot broadcast carrying the audio zone identifier zone=front_passenger. The voice process under the User1 account starts, loads the audio stream from the passenger microphone (audio_dev_002), and displays the interactive interface on the passenger screen.
[0039] By implementing step 102, the voice process is bound to a specific user account, which enables precise allocation of resources to each seating area, avoiding unnecessary resource consumption and improving the response speed and resource utilization of voice interaction.
[0040] In some examples, a dedicated startup broadcast is a specific broadcast message sent to a system user account when the account is successfully acquired, triggering the activation of the voice process corresponding to that account. This broadcast is customized for that account, ensuring that the vehicle system can identify which account needs to activate its dedicated independent voice process. User status indicates whether the user of a specific voice zone is in an interactive state. This can involve detecting whether a passenger is seated and active; for example, by monitoring whether there is a passenger in the seat, whether there is pressure on the seat, or by detecting the presence of the driver or passenger through biosensors; for example, if a seat pressure sensor detects someone sitting in the driver's seat, and biosensors confirm the driver's face or biometric features, then the user of the driver's independent voice zone is considered to be "present".
[0041] For example, in response to the dedicated startup broadcast of the system user account, the user status is detected in the independent audio zone corresponding to the system user account; when a user is detected in the independent audio zone of the driver's seat through the seat pressure sensor or biometric technology (i.e. the user status is "present"), the independent voice process corresponding to the system user account will be started.
[0042] Through the implementation of the above embodiments, the dynamic detection of user status can start and terminate the voice process according to actual needs, ensuring that the voice process is started only when a valid user exists, thereby improving the system's resource utilization and avoiding accidental or invalid voice interactions; for example, if a seat is empty or the user is not present, the system can release resources in a timely manner to prevent invalid voice processes from occupying computing resources.
[0043] Step 103: When a hardware device interaction signal corresponding to the independent voice process is received, the independent voice process responds.
[0044] In some examples, hardware device interaction signals are commands or input signals issued by users through various hardware devices set by independent audio zones, such as in-vehicle microphones, touchscreens, buttons, and sensors. These hardware device interaction signals represent user voice commands, touch operations, button presses, and other interactive behaviors. For example, when the driver says "turn on navigation," the driver's independent audio zone microphone will capture the voice signal and convert it into a hardware device interaction signal. When the passenger touches the corresponding screen, the touch operation will generate a hardware interaction signal. The independent voice process can process and provide feedback on the received hardware device interaction signals. In particular, for voice signals, the independent voice process will call the voice recognition module for parsing and execute related operations according to the instructions, such as controlling in-vehicle navigation and adjusting volume. For example, when "User_0" issues the voice command "turn up the volume," the independent voice process will parse it and increase the in-vehicle volume through the volume control module.
[0045] By implementing step 103, it is ensured that each independent voice process responds to the hardware device interaction signal of the corresponding voice zone. Independent voice processes in different voice zones can process interaction signals simultaneously, supporting multi-user parallel operation, such as the passenger playing music while the driver is navigating.
[0046] In summary, this application embodiment creates multiple system user accounts and starts an independent voice process for each user, enabling passengers in each seat to use voice interaction independently, unaffected by other passengers. Furthermore, leveraging the independence of system user accounts ensures that the voice processes in each audio region are isolated, each handling its corresponding voice request, thereby simplifying system logic and improving the stability and maintainability of the in-vehicle voice interaction method. Additionally, user status detection is performed on the independent audio region corresponding to each system user account. When the user is present, the independent voice process corresponding to that user account is started, thus avoiding starting an independent voice process when no user is present, thereby reducing resource consumption. In conclusion, the in-vehicle voice interaction method provided by this application embodiment utilizes system user accounts, independent voice processes, and hardware device mapping, enabling each independent audio region to have independent voice interaction capabilities, while reducing the implementation complexity and resource consumption of the in-vehicle voice interaction method.
[0047] In some embodiments, step 103 may include: when a hardware device interaction signal corresponding to an independent voice process is received, the hardware device interaction signal is sent to the central adapter process through the independent voice process, wherein the central adapter process is connected to both the independent voice process and the vehicle device control module through a bidirectional communication link; and the cross-user collaboration request or vehicle device control request attached to the hardware device interaction signal is processed through the central adapter process.
[0048] In some examples, the central adapter process is a core process in the vehicle's infotainment system responsible for coordinating and processing communication requests from multiple independent voice processes and in-vehicle device modules. It acts as a bridge, ensuring data and commands can be exchanged between different independent voice processes. This central adapter process (adapter application) starts in single-user mode. The in-vehicle device control module is responsible for controlling various hardware devices within the vehicle, including but not limited to seat adjustment, air conditioning control, and audio system adjustment. For example, when a user uses the voice command "lower the air conditioning temperature," the central adapter process transmits this command to the air conditioning control module, which then performs the temperature adjustment. Cross-user collaboration requests involve collaborative operations from multiple users within the vehicle. For instance, if user A requests to activate a function but requires confirmation or input from user B to complete the operation, this is a cross-user collaboration request. For example, suppose the driver requests to activate the navigation function but requires the passenger to confirm the destination or receive prompts. In this case, the driver's independent voice process generates a cross-user collaboration request and transmits it to the central adapter process, which then collaborates with the passenger's independent voice process. In-vehicle device control requests are requests from users to control in-vehicle hardware devices via voice or other input methods. These requests are passed to the in-vehicle device control module to execute the corresponding operation. For example, when the driver says "adjust the seat," the voice process generates an in-vehicle device control request. After receiving the request, the central adapter process passes it to the seat adjustment module to execute the seat adjustment operation.
[0049] Through the implementation of the above embodiments, the intervention of the central adapter process enables the handling of cross-user collaborative requests from different user accounts / independent voice zones, realizing collaborative work among multiple users. For example, when users in two independent voice zones issue requests related to the same hardware, coordination can be carried out to ensure no conflict occurs, thereby decoupling complex business modules within the voice system and reducing system complexity. Through a bidirectional communication link, the central adapter can simultaneously connect to independent voice processes and vehicle equipment control modules, enabling more complex collaboration and hardware management.
[0050] In some embodiments, the aforementioned processing of cross-user collaboration requests or in-vehicle device control requests attached to hardware device interaction signals by the central adapter process may include: when the hardware device interaction signal is accompanied by a cross-user collaboration request, the central adapter process performs asynchronous task scheduling for the cross-user collaboration request through a message queue; when the hardware device interaction signal is accompanied by an in-vehicle device control request, the central adapter process forwards the in-vehicle device control request to the in-vehicle device control module.
[0051] In some examples, a message queue is a temporary buffer that stores requests, allowing the system to process multiple tasks sequentially without blocking other operations. Asynchronous task scheduling means that the system can continue processing other operations even when some requests are being processed. Message queues can be implemented through the system's inter-process communication mechanism. When the central adapter process receives a cross-user collaboration request, it places the request in the message queue and processes it asynchronously according to a predetermined task scheduling order. For example, suppose the driver requests to activate a function, but the passenger also needs to confirm some settings. The driver's request is placed in the message queue, and the central adapter process processes the request after the passenger confirms it, ensuring task completion through asynchronous scheduling. Vehicle device control requests received by the central adapter process are forwarded to the specific hardware modules responsible for device control, such as air conditioning, seats, and audio systems. These requests are usually triggered by the user's voice or touch operation. The central adapter process is responsible for handing the request over to the device control module for actual operation. For example, if a user requests to "raise the air conditioning temperature" via voice command, this request is forwarded to the air conditioning control module through the central adapter process, and the air conditioning module then performs temperature adjustment.
[0052] Through the implementation of the above embodiments, asynchronous task scheduling using message queues allows the central adapter process to efficiently handle concurrent requests, ensuring that cross-user collaborative requests can be scheduled according to priority and actual needs, thereby avoiding system overload or task blocking. Furthermore, by forwarding vehicle device control requests to the device control module, the central adapter process ensures that the control commands of the hardware devices are executed in a timely and accurate manner, improving the reliability and response speed of the vehicle system.
[0053] In some embodiments, the aforementioned in-vehicle voice interaction method may further include: performing user status detection on the independent voice zone corresponding to the system user account at preset time intervals; and terminating the independent voice process corresponding to the system user account when the user status is empty.
[0054] In some examples, the preset time interval refers to a pre-defined time period used by the system to periodically check the user's status. This time interval can be adjusted according to specific needs, ranging from a few seconds to several minutes. For example, if the preset time interval is 30 seconds, then every 30 seconds, the user status of all independent audio zones, such as the driver, front passenger, and rear seat users, will be checked. An empty status indicates that there is no passenger in the independent audio zone, or that the passenger is inactive and unable to interact with the in-vehicle voice system.
[0055] By implementing the above embodiments, the user status is checked regularly and unnecessary voice processes are terminated in a timely manner when there are no users, and the status of the voice processes is adjusted, thereby optimizing system response and resource allocation, further reducing resource consumption, improving system efficiency, and extending the service life of the device.
[0056] In some embodiments, the aforementioned user status detection for the independent audio zone corresponding to the system user account may include: determining that the user status is present when the vehicle screen, pressure sensor, or biometric sensor in the independent audio zone corresponding to the system user account is in a passively activated state; otherwise, determining that the user status is absent.
[0057] In some examples, an in-vehicle screen refers to a display device installed in any independent audio zone of the vehicle cabin for displaying information, controlling the entertainment system, navigation, and other functions. The in-vehicle screen can be a touchscreen, LCD screen, etc., used for passenger interaction. For example, if the in-vehicle screen in the front passenger seat is on and displaying navigation information, it indicates that the passenger in that independent audio zone is in an interactive state. A pressure sensor is a device installed under the seat in any independent audio zone to detect whether a passenger is sitting in the seat. It can determine the presence of a passenger by measuring changes in pressure on the seat. For example, when the pressure sensor in the driver's seat detects a certain pressure, it indicates that the driver is sitting in the seat, and the driver in the driver's independent audio zone can be considered "present." A biometric sensor is a device used to identify or verify biometric features, including technologies such as facial recognition, fingerprint recognition, or eye tracking, to determine the presence of a passenger. For example, if facial recognition technology confirms the driver's facial features, it indicates that the driver in the driver's independent audio zone is "present." Passive activation refers to a device that automatically activates without active operation; for example, an in-vehicle screen automatically lights up when it receives certain input or conditions, or a sensor automatically activates when it senses a human body or other signals.
[0058] By implementing the above embodiments, combining the status of multiple sensors such as the vehicle screen, pressure sensor, and biometric sensor to determine the presence of a user helps to achieve more accurate and intelligent user status recognition.
[0059] In some embodiments, after step 102, the aforementioned in-vehicle voice interaction method may further include: creating an independent wake-up engine, a speech recognition engine, a text-to-speech engine, and a visual graphical interface module corresponding to an independent voice process.
[0060] In some examples, a wake-up engine refers to a system module that detects and recognizes specific voice commands or wake words (such as "Hey, in-vehicle system"). Once a valid wake word is recognized, the wake-up engine activates the system and begins subsequent voice interaction. A speech recognition engine is the core module that converts user speech input into text. It understands speech content through sound waveform analysis and translates it into machine-understandable commands or requests. A text-to-speech engine converts generated text information into speech output. When the in-vehicle system needs to provide voice feedback to the user, the text-to-speech engine converts the required text into speech and plays it to the user through the speaker. A graphical user interface module is the graphical user interface module in the in-vehicle system. It provides images, animations, or other visual elements so that users can obtain information or interact visually.
[0061] Through the implementation of the above embodiments, a dedicated wake-up engine, speech recognition engine, text-to-speech engine, and visual graphical interface module are created for each independent voice process. This ensures that the independent voice process in each independent voice zone has complete customization and independence, avoiding performance bottlenecks caused by excessive system centralization. Furthermore, each independent voice zone has complete voice interaction functions and resources, which not only improves the flexibility of the system but also ensures that each user's voice interaction experience receives exclusive and personalized support, further enhancing the user-friendliness and response speed of the voice interaction method.
[0062] In some embodiments, the in-vehicle voice interaction method further includes: receiving a data packet sent by a first independent voice process and forwarding the data packet to a second independent voice process, so that the second independent voice process can display the information in the data packet based on the visualization graphical interface module corresponding to the second independent voice process; wherein, the first independent voice process and the second independent voice process are independent voice processes corresponding to different system user accounts.
[0063] The data packet includes at least one type of data, such as files, text, and images. The first independent voice process and the second independent voice process are independent voice processes corresponding to different system user accounts; that is, the first independent voice process and the second independent voice process are independent voice processes corresponding to different independent voice zones.
[0064] In one example, a first independent voice process sends data packets to a central adapter process, which then sends them to a second independent voice process. The second independent voice process then displays the information in the data packets using its corresponding graphical user interface module. In other words, this establishes communication connections between multiple independent voice processes when they are all running.
[0065] For example, if a vehicle has a driver and passengers in the back seats, the vehicle will detect that both the driver's voice zone and the rear voice zone are active, and then start the first independent voice process corresponding to the driver's voice zone and the second independent voice process corresponding to the rear voice zone. If the passenger in the rear seat is deaf or mute, the driver can communicate with the passenger in the rear seat by sending text messages through the first and second independent voice processes.
[0066] In this embodiment, when multiple independent voice processes are started, they can communicate with each other, realizing the diversity of communication methods between different independent voice zones, which facilitates communication between the driver and special passengers (such as deaf-mute people).
[0067] In some embodiments, the in-vehicle voice interaction method further includes: switching the context mode of multiple independent voice zones according to the topics of conversation among users in the vehicle; wherein the context mode includes a shared mode and a non-shared mode.
[0068] In this context, the shared mode means that the context of each independent vocal range can be shared, while the non-shared mode means that the context of each independent vocal range is independent.
[0069] In one example, when users in a vehicle are discussing the same topic, they can switch the context mode of multiple independent voice zones to a shared mode. In shared mode, the electronic devices in each independent voice zone will perform corresponding operations based on the voice information corresponding to multiple independent voice processes. For instance, if users in the vehicle are discussing what to eat for lunch, the driver in the driver's voice zone says, "Find a highly rated restaurant," and the passenger in the passenger's voice zone says, "We want Sichuan cuisine." The search results displayed on both the driver's and passenger's in-vehicle screens are based on the voice information "Find a highly rated restaurant" and "We want Sichuan cuisine."
[0070] When users in the vehicle are not discussing the same topic, the context mode of multiple independent audio zones can be switched to non-shared mode. In non-shared mode, the electronic devices in each independent audio zone will only perform corresponding operations based on the voice information corresponding to their own independent voice process. For example, if the driver in the driver's audio zone says "play pop music," and the passenger in the passenger's audio zone says "search for a puzzle game," the electronic device in the driver's audio zone will play pop music and display the currently playing pop music, while the electronic device in the passenger's audio zone will display the searched puzzle game.
[0071] In this embodiment, the intelligence of the vehicle is further enhanced by switching the context mode of multiple independent audio zones according to the topics of conversation among users in the vehicle.
[0072] In some embodiments, the in-vehicle voice interaction method further includes: when multiple independent voice processes corresponding to multiple system user accounts are started, detecting whether multiple independent voice processes receive hardware device interaction signals for the same in-vehicle device within a preset time period; if so, arbitrating the hardware device interaction signals corresponding to the multiple independent voice processes; and processing the hardware device interaction signals corresponding to the multiple independent voice processes based on the arbitration result.
[0073] For example, the preset duration is 5 seconds; within 3 seconds, the driver in the driver's seat area and the passengers in the rear seat area both issue commands to the air conditioning. As a result, the vehicle will detect that within 3 seconds, multiple independent voice processes have received hardware device interaction signals for the same in-vehicle device. The vehicle will then arbitrate the hardware device interaction signals corresponding to the multiple independent voice processes, obtain the arbitration result, and then process the hardware device interaction signals corresponding to the multiple independent voice processes based on the arbitration result.
[0074] In this embodiment, when multiple independent voice processes receive hardware device interaction signals for the same in-vehicle device within a preset time period, the hardware device interaction signals corresponding to the multiple independent voice processes will be arbitrated, thus avoiding the disorder of the in-vehicle device state.
[0075] In some embodiments, based on the arbitration result, the hardware device interaction signals corresponding to multiple independent voice processes are processed, including: determining a target independent voice process from multiple independent voice processes based on the arbitration result; sending the corresponding hardware device interaction signal to the central adapter process through the target independent voice process; and processing cross-user collaboration requests or in-vehicle device control requests attached to the hardware device interaction signals through the central adapter process.
[0076] In one possible implementation, if the arbitration result indicates that the hardware device interaction signals corresponding to multiple independent voice processes do not conflict, then all multiple independent voice processes are taken as the target independent voice process; if the arbitration result indicates that the hardware device interaction signals corresponding to multiple independent voice processes conflict, then the independent voice process with the highest priority is taken as the target independent voice process.
[0077] In one example, if the driver says to turn up the temperature and the rear passenger says to turn it to 26 degrees, then it is determined that the hardware device interaction signal corresponding to the independent voice process in the driver's area does not conflict with the hardware device interaction signal corresponding to the independent voice process in the rear area. Therefore, both the independent voice process in the driver's area and the independent voice process in the rear area are used as target independent voice processes. At this time, the corresponding hardware device interaction signals are sent to the central adapter process through the independent voice processes in the driver's area and the rear area.
[0078] In one example, the priorities of multiple independent voice processes are as follows: the independent voice process in the driver's area has higher priority than the independent voice process in the passenger's area, and the independent voice process in the passenger's area has higher priority than the independent voice process in the rear area. When the driver says to lower the temperature and the rear passenger says to raise the temperature, it is determined that the hardware device interaction signal corresponding to the independent voice process in the driver's area conflicts with the hardware device interaction signal corresponding to the independent voice process in the rear area. In this case, the independent voice process in the driver's area is selected as the target independent voice process, and the corresponding hardware device interaction signal is sent to the central adapter process through the independent voice process in the driver's area.
[0079] The specific implementation of cross-user collaboration requests or onboard device control requests that are attached to hardware device interaction signals through the central adapter process can be seen above, and will not be repeated here.
[0080] In some embodiments, when multiple independent voice processes corresponding to multiple system user accounts are started, it is detected whether multiple independent voice processes receive hardware device interaction signals for the same in-vehicle device within a preset time period; if so, the hardware device interaction signals corresponding to the multiple independent voice processes are arbitrated; if the arbitration result indicates that the hardware device interaction signals corresponding to the multiple independent voice processes conflict, the hardware device interaction signals corresponding to the multiple independent voice processes are discarded.
[0081] Furthermore, as an implementation of the aforementioned method embodiments, this application also provides an in-vehicle voice interaction device for implementing the aforementioned method embodiments. This device embodiment corresponds to the aforementioned method embodiments. For ease of reading, this in-vehicle voice interaction device embodiment will not repeat the details of the aforementioned method embodiments one by one, but it should be understood that the device in this application embodiment can correspondingly implement all the contents of the aforementioned method embodiments.
[0082] As shown in Figure 2, the in-vehicle voice interaction device 20 includes: an account acquisition unit 201, a process control unit 202, and a voice interaction unit 203. The account acquisition unit 201 is used to acquire the system user account corresponding to the independent voice zone. The process control unit 202 is used to detect the user status of the independent voice zone corresponding to the system user account in response to the dedicated power-on broadcast of the system user account. When the user status is active, the independent voice process corresponding to the system user account is started. The voice interaction unit 203 is used to respond through the independent voice process when it receives the hardware device interaction signal corresponding to the independent voice process.
[0083] In some embodiments, the voice interaction unit 203 is further configured to, when receiving a hardware device interaction signal corresponding to an independent voice process, send the hardware device interaction signal to the central adapter process through the independent voice process, wherein the central adapter process is connected to both the independent voice process and the vehicle device control module through a bidirectional communication link; and process cross-user collaboration requests or vehicle device control requests attached to the hardware device interaction signal through the central adapter process.
[0084] In some embodiments, the voice interaction unit 203 is further configured to, when the hardware device interaction signal is accompanied by a cross-user collaboration request, implement asynchronous task scheduling for the cross-user collaboration request through a message queue; and when the hardware device interaction signal is accompanied by an in-vehicle device control request, the central adapter process forwards the in-vehicle device control request to the in-vehicle device control module.
[0085] In some embodiments, the process control unit 202 is further configured to perform user status detection on the independent audio region corresponding to the system user account at preset time intervals; when the user status is empty, the independent voice process corresponding to the system user account is terminated.
[0086] In some embodiments, the process control unit 202 is further configured to determine that the user status is present when the in-vehicle screen, pressure sensor or biometric sensor in the independent audio zone corresponding to the system user account is in a passively activated state; otherwise, it determines that the user status is absent.
[0087] In some embodiments, the process control unit 202 is also used to create an independent wake-up engine, a speech recognition engine, a text-to-speech engine, and a visual graphical interface module corresponding to the independent voice process.
[0088] In some embodiments, the voice interaction unit 203 is further configured to receive data packets sent by the first independent voice process and forward the data packets to the second independent voice process, so that the second independent voice process can display the information in the data packets based on the visualization graphical interface module corresponding to the second independent voice process; wherein, the first independent voice process and the second independent voice process are independent voice processes corresponding to different system user accounts.
[0089] In some embodiments, the voice interaction unit 203 is further configured to switch the context mode of multiple independent voice zones according to the topics of conversation among users in the vehicle; wherein the context mode includes a shared mode and a non-shared mode.
[0090] In some embodiments, the voice interaction unit 203 is further configured to detect, within a preset time period, whether multiple independent voice processes receive hardware device interaction signals for the same in-vehicle device when multiple independent voice processes corresponding to multiple system user accounts are started; if so, to arbitrate the hardware device interaction signals corresponding to the multiple independent voice processes; and to process the hardware device interaction signals corresponding to the multiple independent voice processes based on the arbitration result.
[0091] In some embodiments, the voice interaction unit 203 is specifically used to determine a target independent voice process from multiple independent voice processes based on the arbitration result; send the corresponding hardware device interaction signal to the central adapter process through the target independent voice process; and process the cross-user collaboration request or vehicle device control request attached to the hardware device interaction signal through the central adapter process.
[0092] In some embodiments, the voice interaction unit 203 is specifically used to treat all the multiple independent voice processes as the target independent voice process if the arbitration result indicates that the hardware device interaction signals corresponding to the multiple independent voice processes do not conflict.
[0093] In some embodiments, the voice interaction unit 203 is specifically used to select the highest priority independent voice process as the target independent voice process if the arbitration result indicates that there is a conflict in the hardware device interaction signals corresponding to multiple independent voice processes.
[0094] This application also provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, will cause the processor to perform any step of the in-vehicle voice interaction method provided in this application.
[0095] In some embodiments, the computer-readable storage medium may be a random access memory (RAM), a read-only memory (ROM), flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); or it may be a variety of devices that include one or any combination of the above-mentioned memories.
[0096] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.
[0097] In some embodiments, computer-executable instructions may, but do not necessarily, correspond to files in a file system, and may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).
[0098] In some embodiments, computer-executable instructions may be deployed to execute on an electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.
[0099] As shown in Figure 3, this application also provides an electronic device 30, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements any step of the above-mentioned vehicle voice interaction method.
[0100] This application also provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium and executes the computer program or computer-executable instructions, causing the electronic device to perform any step of the in-vehicle voice interaction method described above.
[0101] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A vehicle-mounted voice interaction method, characterized in that, Applied to vehicles, the vehicle has multiple independent audio zones, each with a corresponding system user account, to manage the resources of the corresponding independent audio zone through the system user account, enabling independent voice interaction for the corresponding independent audio zone; the in-vehicle voice interaction method includes: Obtain the system user account corresponding to the independent audio region; In response to the dedicated startup broadcast of the system user account, user status detection is performed on the independent audio zone corresponding to the system user account; When the user status is present, start an independent voice process corresponding to the system user account; When a hardware device interaction signal corresponding to the independent voice process is received, the independent voice process responds.
2. The in-vehicle voice interaction method of claim 1, wherein, The step of responding through the independent voice process when receiving an interaction signal from a hardware device corresponding to the independent voice process includes: When a hardware device interaction signal corresponding to the independent voice process is received, the hardware device interaction signal is sent to the central adapter process through the independent voice process. The central adapter process is connected to both the independent voice process and the vehicle device control module through a bidirectional communication link. The central adapter process handles cross-user collaboration requests or in-vehicle device control requests attached to the hardware device interaction signals.
3. The in-vehicle voice interaction method of claim 2, wherein, The process of handling cross-user collaboration requests or in-vehicle device control requests attached to the hardware device interaction signals through the central adapter process includes: When the hardware device interaction signal is accompanied by the cross-user collaboration request, the central adapter process performs asynchronous task scheduling for the cross-user collaboration request through a message queue. When the hardware device interaction signal carries the vehicle device control request, the central adapter process forwards the vehicle device control request to the vehicle device control module.
4. The in-vehicle voice interaction method of claim 1, wherein, The in-vehicle voice interaction method also includes: User status detection is performed on the independent audio region corresponding to the system user account at preset time intervals. When the user status is vacant, the independent voice process corresponding to the system user account is terminated.
5. The in-vehicle voice interaction method of claim 1 or 4, characterized in that, The step of detecting the user status of the independent audio region corresponding to the system user account includes: When the in-vehicle screen, pressure sensor, or bio-detection sensor in the independent audio zone corresponding to the system user account is in a passively activated state, the user status is determined to be present; otherwise, the user status is determined to be absent.
6. The in-vehicle voice interaction method of claim 1, wherein, After initiating an independent voice process corresponding to the system user account when the user state is active, the in-vehicle voice interaction method further includes: Create independent wake-up engine, speech recognition engine, text-to-speech engine, and visual graphical interface module corresponding to the independent speech process.
7. The in-vehicle voice interaction method of claim 6, wherein, The in-vehicle voice interaction method also includes: Receive data packets sent by the first independent voice process and forward the data packets to the second independent voice process, so that the second independent voice process can display the information in the data packets based on the visualization graphical interface module corresponding to the second independent voice process; The first independent voice process and the second independent voice process are independent voice processes corresponding to different system user accounts.
8. The in-vehicle voice interaction method of claim 1, wherein, The in-vehicle voice interaction method also includes: The context mode of multiple independent audio zones is switched according to the topics of communication among users in the vehicle; The context modes include shared mode and non-shared mode.
9. The in-vehicle voice interaction method of claim 1, wherein, The in-vehicle voice interaction method also includes: When multiple independent voice processes corresponding to multiple system user accounts are started, it is detected whether multiple independent voice processes receive hardware device interaction signals for the same in-vehicle device within a preset time period. If so, the hardware device interaction signals corresponding to the multiple independent voice processes are arbitrated; based on the arbitration result, the hardware device interaction signals corresponding to the multiple independent voice processes are processed.
10. The in-vehicle voice interaction method of claim 9, wherein, The processing of hardware device interaction signals corresponding to multiple independent voice processes based on the arbitration result includes: Based on the arbitration result, a target independent speech process is determined from the plurality of independent speech processes; The target independent voice process sends the corresponding hardware device interaction signal to the central adapter process; The central adapter process handles cross-user collaboration requests or in-vehicle device control requests attached to the hardware device interaction signals.
11. The in-vehicle voice interaction method of claim 10, wherein, The step of determining the target independent speech process from among the multiple independent speech processes based on the arbitration result includes: If the arbitration result indicates that the hardware device interaction signals corresponding to the multiple independent voice processes do not conflict, then all of the multiple independent voice processes will be regarded as the target independent voice process.
12. The in-vehicle voice interaction method of claim 10, wherein, The step of determining the target independent speech process from among the multiple independent speech processes based on the arbitration result includes: If the arbitration result indicates that the hardware device interaction signals corresponding to multiple independent voice processes conflict, then the independent voice process with the highest priority will be taken as the target independent voice process.
13. A vehicle-mounted voice interaction device, characterized in that, Applied to vehicles, the vehicle has multiple independent audio zones, each with a corresponding system user account to manage resources of the corresponding independent audio zone, enabling independent voice interaction for each independent audio zone; the in-vehicle voice interaction device includes: An account acquisition unit is used to acquire the system user account corresponding to the independent audio zone; The process control unit is used to respond to the dedicated startup broadcast of the system user account, and to perform user status detection on the independent audio zone corresponding to the system user account; when the user status is present, the independent voice process corresponding to the system user account is started. The voice interaction unit is used to respond through the independent voice process when it receives an interaction signal from a hardware device corresponding to the independent voice process.
14. An electronic device comprising: The memory and processor are characterized in that the processor, when executing a computer program stored in the memory, implements the steps of the in-vehicle voice interaction method as described in any one of claims 1-12.
15. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the in-vehicle voice interaction method as described in any one of claims 1-12.
