Voice interaction method and device, vehicle

Abstract

The present disclosure relates to a voice interaction method and device and vehicle, and relates to the technical field of information processing. The voice interaction is performed in a first sound area. Based on an operation of switching the voice interaction function to a target sound area, the voice interaction is performed in the target sound area. Because the voice interaction function can be switched between sound areas, the novelty and flexibility of the voice interaction function are improved, and the user experience is improved.

Description

[0001] This application claims priority to Chinese Patent Application No. 202510697492.2, filed on May 27, 2025, entitled "Voice Interaction Method and Apparatus, Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of information processing technology, and in particular to a voice interaction method and device, and a vehicle. Background Technology

[0003] In-vehicle voice interaction functions are becoming increasingly popular. After a user speaks, the vehicle recognizes the voice and then responds to the content of the voice.

[0004] In scenarios where voice interaction is based on a specific voice zone, even if the voice interaction function has been activated in one voice zone, users in another voice zone need to activate the voice interaction function again if they want to interact with the user. This repeated activation method is rather rigid and monotonous. How to provide a more novel and flexible voice interaction method to improve the user experience is a problem that needs to be solved. Summary of the Invention

[0005] To address the aforementioned technical issues, this disclosure provides a voice interaction method, apparatus, and vehicle, offering a more novel and flexible voice interaction approach to improve user experience.

[0006] A first aspect of this disclosure provides a voice interaction method, including:

[0007] Voice interaction is performed in the first audio region;

[0008] Based on the operation of switching the voice interaction function to the target voice region, voice interaction is performed in the target voice region.

[0009] A second aspect of this disclosure provides a voice interaction method, including:

[0010] Based on the target operation, a voice interaction interface is displayed, and the first virtual image of the first character is displayed on the voice interaction interface.

[0011] A third aspect of this disclosure provides a voice interaction device, including:

[0012] A module for implementing the voice interaction method described in the first or second aspect of the embodiments of this disclosure.

[0013] A fourth aspect of this disclosure provides a vehicle including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the voice interaction method as described in the first or second aspect above.

[0014] A fifth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the voice interaction method as described in the first or second aspect above.

[0015] The voice interaction method, device, and vehicle provided in this application perform voice interaction in the first voice zone. Based on the operation of switching the voice interaction function to the target voice zone, voice interaction is performed in the target voice zone. That is, the voice interaction function can be switched between voice zones. The switching is equivalent to the flow of the voice interaction function without repeated wake-up. Therefore, it can improve the novelty and flexibility of the voice interaction function and is conducive to improving the user experience. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a voice interaction method provided in an embodiment of this disclosure;

[0019] Figure 2 This is an example diagram of the voice interaction interface provided in the embodiments of this disclosure;

[0020] Figure 3 This is an example diagram of switching voice zones based on a voice interaction interface provided in this embodiment of the disclosure;

[0021] Figure 4 This is an example diagram of switching voice assistants based on a voice interaction interface provided in this embodiment of the disclosure;

[0022] Figure 5 This is an example diagram of a minimized voice interaction interface provided in an embodiment of this disclosure;

[0023] Figure 6 This is an example diagram of controlling a voice interaction interface on a non-central control screen based on a remote control interface on a central control screen, according to an embodiment of this disclosure.

[0024] Figure 7 This is a flowchart of yet another voice interaction method provided in this disclosure embodiment;

[0025] Figure 8 This is a schematic diagram of the structure of a vehicle provided in an embodiment of this disclosure. Detailed Implementation

[0026] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0027] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0030] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] During their research, the inventors discovered that current voice interaction functions still have room for improvement. Some exemplary issues are as follows:

[0032] 1. Task-based voice interaction: When a user initiates a voice interaction, they need to wake up the voice assistant with a wake word to start a task. After the task starts, the voice assistant responds to the user's voice. The task ends when the response ends or when the user does not interact with the voice for a period of time. After a task ends, if the user needs to interact with the voice assistant again, they need to wake up the voice assistant with the wake word again. This task-based voice interaction process is temporary and has a short interaction time. Therefore, it cannot provide users with an immersive and more human-like dialogue experience.

[0033] 2. The voice interaction function still needs improvement in terms of flexibility, versatility, convenience, and user experience.

[0034] 3. Models that implement voice interaction functions, such as large language models, require that both input and output data be in text form. Based on this, an example of the technical chain for obtaining the interaction result for voice interaction is: first, convert the voice into text, then call the large language model to understand the text and generate the response text, and finally convert the response text into voice output. It can be seen that the implementation chain of voice interaction functions is relatively long, resulting in a large latency.

[0035] 4. The output speech used to respond to the user's voice is usually converted into speech by text-to-speech (TTS). TTS still needs improvement in expressing emotions in a targeted manner, such as not being able to provide personalized voice expressions for different users.

[0036] Therefore, there is still room for improvement in current voice interaction methods.

[0037] To improve current voice interaction functions, i.e., to at least solve the aforementioned exemplary problems, embodiments of this application disclose a voice interaction method. This method can be applied to a processor. For example, a processor is provided in a smart device, and the processor can run the voice interaction method disclosed in the embodiments of this application. For example, the smart device can be a vehicle, and the vehicle is equipped with an intelligent system, which includes a processor.

[0038] This application discloses a voice interaction method, such as... Figure 1 As shown, it includes the following steps:

[0039] S11, Use voice interaction in the first vocal register.

[0040] A voice zone is a set of areas designated for voice interaction functions. One voice zone can correspond to one or more seating areas. A voice zone is an area that not only has a seat and microphone, but can also have a screen. The screen set within a certain voice zone is called the screen corresponding to that voice zone. Each voice zone is independent of each other and can interact with the voice of at least one voice zone without responding to the voice of other voice zones.

[0041] The first voice register can be any voice register. For example, in voice interaction, the user makes a voice statement, and the smart device, such as a vehicle, responds to the user's voice, such as answering the user's questions, chatting with the user, or executing the user's instructions.

[0042] The voice interaction described in this step refers to a voice interaction function based on something like "making a phone call" (also known as the first voice interaction function). This voice interaction function can also be implemented in at least one voice zone, that is, the first voice zone can be at least one voice zone.

[0043] S12. Based on the operation of switching the voice interaction function to the target voice region, perform voice interaction in the target voice region.

[0044] Switching the voice interaction function can be understood as switching the voice interaction function between different audio ranges. Based on S11, where voice interaction occurs in the first audio range, the voice interaction function is implemented in the first audio range. Switching the voice interaction function means changing the audio range from the first audio range to another audio range. In S12, this other audio range specifically refers to the target audio range. That is, after the switch, the audio range implementing the voice interaction function is the target audio range, and therefore, voice interaction is performed in the target audio range. The target audio range can be at least one audio range.

[0045] It is understandable that switching voice interaction functions is different from waking up voice interaction functions. The following will use two different switching methods to illustrate the difference between switching voice interaction functions and waking up voice interaction functions.

[0046] For example, the voice interaction described in this embodiment is a voice interaction function similar to "making a phone call". That is, after the voice interaction function is activated, it remains in a state similar to "on a call". The user can "talk" to the vehicle in a way similar to talking to a person. Even if the user does not output voice for a period of time, as long as the voice interaction function is not turned off, the user does not need to say the activation voice again when he / she needs to perform voice interaction again.

[0047] For example, in order to enhance the experience of "making a phone call", prior to S12, in addition to voice interaction in the first voice zone, a voice interaction interface was also displayed on the first screen corresponding to the first voice zone. When voice interaction functions are implemented in multiple voice zones, the voice interaction interface can be displayed on the screens corresponding to multiple voice zones.

[0048] For example, taking the driver's side audio zone as the first audio zone, the screen corresponding to the driver's side audio zone is the vehicle's central control screen. Figure 2 This is an example of a voice interaction interface 22 displayed on the vehicle's central control screen 21. The voice interaction interface 22 displays a first virtual avatar 221 of a target character, which is one of the pre-configured characters. After the user utters the voice command "How's the weather today?", voice interaction occurs in the first voice zone. In the voice interaction interface 22, the first virtual avatar 221 outputs a response to "How's the weather today?" using specific actions, timbre, and tone. Figure 2 (Not shown in the image) After the response is completed, the voice interaction interface 22 remains displayed. Even after a period of time, if the user does not speak (i.e., no voice is detected), the voice interaction interface 22 continues to be displayed. If voice in the first voice range is detected again, the first virtual avatar 221 will also respond to the voice. It is understandable that even when the voice interaction interface or virtual avatar is not displayed, a voice response can still be output, or the voice response can be output with the timbre and tone corresponding to the target character.

[0049] For example, the voice interaction interface 22 also displays time information 225, which indicates the duration of the voice interaction function. Assume... Figure 2 This is the voice interaction interface from the start of the voice interaction function to the time before switching. The time information 225 indicates the duration from the start of the voice interaction function to the time before switching, i.e., the first duration. In other words, the voice interaction interface displayed on the first screen shows the first duration, which represents the duration that the voice interaction function has lasted from the start of the function to the time before switching.

[0050] Displaying the duration of voice interaction helps to enhance the non-temporary nature of the interaction and the ability to engage with users for longer periods, thereby strengthening the immersive and human-like voice interaction experience.

[0051] Understandable Figure 2 The voice interaction interface shown enhances the voice call experience, similar to "making a phone call." Compared with the existing task-based voice interaction methods, it does not require repeated wake-up by the user, is non-temporary, and can interact with the user for a longer period of time. Furthermore, this form allows for continuous dialogue with the user after activating the function only once (i.e., the first voice interaction function). These features make the voice interaction closer to human-to-human conversation, thus providing an immersive and more human-like voice interaction function.

[0052] As can be seen from the above, before S12, the voice interaction interface is displayed on the first screen corresponding to the first voice zone. Based on this, an exemplary operation for switching the voice interaction function to the target voice zone in S12 is: selecting the target voice zone in the voice interaction interface.

[0053] For example, the voice interaction interface 22 also displays a voice zone selection control 223.

[0054] Combination Figure 3 As shown, when the user clicks the audio zone selection control 223, the audio zone selection area 31 is displayed. This area displays information about the audio zones already set in the vehicle, such as driver's seat, front passenger seat, second row left, second row right, third row left, and third row right. The user can select at least one audio zone in the audio zone selection area 31 by clicking. Figure 3 In the example, assuming the user selects the driver's seat (highlighted), after selecting the audio region, the user clicks the "Confirm" control 311 to complete the selection of the target audio region. If the user clicks the "Cancel" control 312, the selection of the audio region is canceled, and the user can exit the audio region selection area 31.

[0055] like Figure 3 As shown, the audio zone selection area 31 is displayed over the voice interaction interface 22, in which case the voice interaction interface 22 is not visible. Alternatively, the audio zone selection area 31 can also be displayed within or outside the voice interaction interface 22; that is, the audio zone selection area 31 can be displayed anywhere on the first screen (such as the central control screen).

[0056] Alternatively, the voice zone selection area 31 can also be displayed as a virtual layout map of each voice zone. The virtual layout map is displayed outside the voice interaction interface 22. Users can drag the voice interaction interface 22 to the target voice zone in the virtual layout map to select the target voice zone.

[0057] After selecting the target audio region and capturing speech within that region, voice interaction is initiated. For example, the voice interaction interface is displayed on the second screen corresponding to the target audio region. Figure 3 As shown, the voice interaction interface 22 is displayed on the screen corresponding to the target audio region. For example, if the target audio region is the second row left, then the voice interaction interface 22 is displayed on the screen corresponding to the second row left (such as the screen located behind the driver's seat). For example, cross-screen display of the voice interaction interface does not change the content displayed by the voice interaction interface 22.

[0058] In summary, an example of a switching scenario for a voice interaction function similar to "making a phone call" is as follows: After the driver activates the voice interaction interface 22, it is displayed on the central control screen. To facilitate the use of the voice interaction function by the child on the left side of the second row, the driver can switch the voice interaction interface 22 from the central control screen to the screen corresponding to the left side of the second row using the voice zone selection control 223. After the voice interaction interface 22 is displayed on the screen corresponding to the left side of the second row, it responds to the voice in the left side of the second row and no longer responds to the voice in the driver's voice zone. Therefore, the child can perform real-time voice interaction.

[0059] Furthermore, the voice response function targeting the desired audio range can clearly identify the recipient when multiple people are speaking in the car, reducing the possibility of false responses.

[0060] It is understandable that, in addition to switching voice interaction functions based on the voice interaction interface, it is also possible to implement it based on voice. That is, the user can use voice to instruct the user to switch the voice interaction function from the first voice zone to the target voice zone. For example, if the user says "to perform voice interaction in the second row left" or "to switch the voice interaction function to the second row left", the voice interaction function will switch to the second row left voice zone and respond to the voice in the second row left voice zone. Furthermore, the voice interaction interface will be displayed on the screen corresponding to the second row left.

[0061] Regardless of whether the voice interaction function is switched based on the interface or voice, when the voice interaction interface is displayed, the duration of the voice interaction interface before and after the switch is continuous. That is, the first screen displays the first duration of the voice interaction function, which represents the duration of the voice interaction function from its inception to the switch. The second screen displays the second duration, which is obtained by summing the first and third durations. The third duration is the duration of the voice interaction function in the target voice region. For example, as... Figure 3 As shown, the first duration is "00:26", or 26 seconds. When the voice interaction function switches from the first voice zone corresponding to the central control screen to the second row left voice zone, the duration in the second row left voice zone has already been 0 seconds. Therefore, the second duration in the voice interaction interface displayed on the screen corresponding to the second row left is also "00:26". Here, we assume that the switching process time is not considered. Of course, the switching action time can also be considered. In either case, the second duration is a continuous countdown based on the first duration (and the switching process), indicating that the voice interaction function is continuously running.

[0062] As can be seen from the examples above, there is a fundamental difference between switching voice interaction functions and waking up voice interaction functions: switching voice interaction functions is equivalent to "making a phone call" by moving from one voice zone to another, which is a switching of voice interaction functions between voice zones, while waking up voice interaction functions is to start the voice interaction function a new time.

[0063] In the case of voice-based voice interaction switching, the voice needs to indicate the switching and target voice zone. In the case of voice-activated voice interaction, a specific wake-up word is required. Moreover, the essence of wake-up is to restart the voice interaction function. Suppose the voice interaction function is activated in the first voice zone the first time, and then activated in the second voice zone the second time. Even if the voice interaction is performed immediately after the first wake-up, and then activated in the second voice zone the second time and performed again, it seems that the voice interaction function is not interrupted but continuous. However, in reality, it is only because the time interval between the second wake-up and the voice interaction after the first wake-up is short. The second wake-up is still a restart of the voice interaction function.

[0064] As mentioned earlier, regarding the voice-activated interaction function, the duration of the instructions on the voice interaction interface is continuous before and after the function switch. That is, the voice interaction function is not restarted, but rather the switching occurs within a specific pitch range. Even if the voice interaction interface is not displayed, the voice interaction function does not end and then restart, but rather maintains a continuous state while switching within the pitch range.

[0065] Combination Figures 1-3 As can be seen, the voice interaction method provided in this embodiment can realize the switching of voice interaction functions between voice zones, improve the convenience of voice interaction functions, and facilitate the provision of more novel and flexible voice interaction functions.

[0066] In addition to the aforementioned voice zone selection control and duration of voice interaction, the voice interaction interface displays at least one of the following: a pause control, an exit control for exiting the voice interaction function, a minimize control for minimizing the voice interaction interface, a role selection control, a style selection control, and text for voice interaction.

[0067] For example, Figure 2 In the voice interaction interface 22, a close control 222 (an example of an exit control for exiting the voice interaction function) is also displayed. After the user operates the close control 222, the voice interaction interface 22 closes and the voice interaction function exits. It can be understood that, in addition to this exit method, the voice interaction function can also be exited based on a specific voice command.

[0068] For example, text-based voice interaction Figure 2 (Not shown in the image) This includes: text from voice interaction, i.e., text of the user's voice and / or text of responses to the user's voice; specifically, voice is transcribed into text and displayed. Displaying text from voice in the voice interaction interface improves the user experience.

[0069] The functions and related content of other controls on the voice interaction interface will be explained in detail below with reference to the accompanying drawings.

[0070] To improve user experience, multiple roles are pre-configured for the virtual avatars displayed on the voice interaction interface. These roles include multiple roles belonging to different categories. For example, the different categories include children and adults, meaning the multiple roles include both children and adults. Alternatively, the different categories may only include children or only adults. It is understood that the above classification is merely an example, and other classification criteria can also be used.

[0071] Based on the operation of selecting a target character through the character selection control (also known as the second character selection control) in the voice interaction interface, the first virtual image of the target character is displayed in the voice interaction interface. The target character is one of multiple characters.

[0072] For example, the voice interaction interface 22 also displays a role selection control 224.

[0073] like Figure 4 As shown, the user clicks on the character selection control 224, triggering the display of the character display area 41. The character display area 41 displays the virtual images and / or names of multiple pre-configured characters. The user selects a character by clicking on any virtual image or name. After selecting a character's virtual image or name, the user clicks the "Confirm" control 411 to complete the character selection. If the user clicks the "Cancel" control 412, the character selection is canceled, and the user exits the character display area 41.

[0074] The following example will demonstrate a scenario where a character is changed based on a voice interaction interface. For ease of explanation, the "target character" can also be referred to as the "first character".

[0075] Assuming that before the user selects a role, the voice interaction interface displays the virtual image of the first role, "Ideal Classmate," and the user... Figure 4 The operation selected the virtual avatar of the second character, "Courage of Light." Since the first and second characters are different, the virtual avatar 221 of the first character in the voice interaction interface 22 needs to be updated; that is, the virtual avatar 221 of the first character needs to be switched to the virtual avatar of the second character. Figure 4 For example, the virtual avatar in the voice interaction interface 22 has changed.

[0076] The voice interaction interface also displays a style selection control corresponding to the target character. Assuming the voice interaction interface currently displays the first virtual image of the target character, based on the operation of selecting a target style, such as a sweet style, through the style selection control, a second virtual image of the target character is displayed on the voice interaction interface. The second virtual image corresponds to the target style. For example, any character may also correspond to different styles, which may include at least one of: childlike, sweet, and standard styles. The voice interaction interface 22 also displays a style selection control (such as...). Figure 4 (Not shown in the image), each style is represented by a control: the "Childlike" control represents the childlike style, the "Sweet" control represents the sweet style, and the "Standard" control represents the standard style. Each style not only corresponds to a virtual avatar different from other styles, but also to at least one of the following: timbre, tone of voice, and action. Timbre refers to the timbre used in the voice interaction function's output speech; tone of voice refers to the tone of voice used in the voice interaction function's output speech; and action refers to the action used by the virtual avatar. Voice interaction is performed using at least one of the timbre, tone of voice, and action corresponding to the target style.

[0077] At least one of the timbre, tone, and movement corresponding to each style can be pre-configured.

[0078] After selecting a character, users can further choose the character's style by clicking the playful, sweet, or standard controls in the voice interaction interface 22. If a user only selects a character without choosing a style, the standard style will be used by default. Users can also select a character's style using voice commands, gestures, etc., which are not limited here.

[0079] In this embodiment, each character corresponds to one of three different styles: childlike, sweet, and standard. Of course, different styles can be configured for different characters, which is not limited here.

[0080] Setting and switching between different roles and / or styles can enhance the fun of voice interaction.

[0081] In addition to setting styles for each character, styles can also exist independently of characters; this example does not impose such limitations.

[0082] The virtual images and names of the characters shown in the accompanying drawings of this embodiment are examples and not intended to be limiting.

[0083] In addition to users actively selecting roles, smart devices such as vehicles can also automatically select roles based on target information. For example, multiple roles and their corresponding styles, virtual images, voice timbres, and actions are pre-configured in the processor. In this case, a target role is selected from multiple roles based on target information, which includes at least one of account information, user biometric information, voice region information, and voice interaction data.

[0084] For example, the system can distinguish between a child's account and a non-child's account based on the logged-in account. If it's a child's account, a child-related role, such as the target role, can be selected. This account can be the account for the voice interaction function or a vehicle account. When determining the role, users can choose between adult and child-related roles based on account attributes, or they can match the account to the corresponding role based on a pre-set account-role mapping. This account-role mapping can be user-defined or a system default. For example, if account A is pre-defined as role 'a' and account B is pre-defined as role 'b', when account A logs in, role 'a' will be selected to interact with the user.

[0085] The user's biometric information includes, but is not limited to: the voiceprint of the voice interaction function activation command, and facial recognition within the vehicle. For example, if the voiceprint used to activate the voice interaction function is identified as a child's, or if the user's voice content includes information about a child, such as the voice saying "tell a story to Potato," and "Potato" is recognized as a child's nickname, then the system automatically selects the child-related role. Another example is recognizing a child's face in the seat corresponding to the voice interaction function's designated area, which then selects the child-related role.

[0086] The voice interaction zone is the zone that enables voice interaction functions, such as the driver's voice zone.

[0087] Voice interaction data includes historical voice interaction data. For example, historical voice interaction data is more relevant to children, such as nursery rhymes; in this case, a role related to children is selected.

[0088] The above target information can also be used in combination. For example, if the system receives the user's voice message "tell a story to the potato in the back row", and the back row is the sound zone for voice interaction, and a child's face is detected in the back row, then when the voice interaction function is switched to the back row, the system can select a child-related character to interact with the potato in the back row.

[0089] The style of the child-related character can be pre-configured, as well as the virtual image, voice, and actions corresponding to each style, making them more appealing to children. Therefore, compared with non-child-related characters, the child-related characters are closer to children's habits in terms of actions and voice, which is conducive to realizing child-friendly voice interaction functions.

[0090] Characterizing virtual avatars can enhance the ability of voice interaction functions to express emotions and allow for more personalized emotional expression for different users.

[0091] Understandably, the methods described above for children are just examples and can also be applied to adults.

[0092] When the voice interaction interface is first launched, the virtual avatar displayed can be the virtual avatar displayed in the last voice interaction interface, the default virtual avatar, or a virtual avatar selected based on the target information.

[0093] The embodiments of this application also provide a voice interaction method, focusing on the role-related memory function. In this embodiment, the technical feature embodying the role-related memory function is: sharing user information and / or historical interaction information in continuous or discontinuous voice interactions performed by the same or different users using the same or different roles.

[0094] The specific implementation of the above-mentioned technical features will be described in detail below in the voice interaction process. In this embodiment, the selection of different roles is involved. For ease of explanation, the "target role" is also referred to as the "first role". The method includes the following steps:

[0095] A1. Perform voice interaction in the first voice zone and display the voice interaction interface on the first screen corresponding to the first voice zone. The voice interaction interface displays the virtual image of the first character.

[0096] For details regarding the voice interaction interface, please refer to [link / reference]. Figure 2 The corresponding content will not be repeated here.

[0097] In this embodiment, it is assumed that the voice interaction interface displayed on the first screen shows a virtual image corresponding to the standard style of the first character. The method for selecting the first character is as described above.

[0098] B1. Record the interaction logs of voice interaction using the first character.

[0099] Interaction logs include: interaction content and basic user information.

[0100] The interactive content includes the user's voice content, the voice content based on the user's voice feedback, and information related to the operation performed by the voice interaction function in response to the user's voice commands, such as the name of the song being played.

[0101] A user's basic information consists of relatively fixed information about the user, including but not limited to: biometric information such as voiceprint, name, gender, age, and height.

[0102] C1. Based on the operation of selecting the second character in the voice interaction interface, change the virtual image of the first character in the voice interaction interface to the virtual image of the second character.

[0103] As mentioned earlier, you can also optionally choose a style under the second character.

[0104] D1. When the second role is selected, voice interaction is conducted in the first voice zone based on the user's basic information.

[0105] For example, if the primary driver is female, and B1 records her basic information such as age, height, and weight, in this step, if the first voice register recognizes the primary driver's voice asking "What should I eat tonight?", the vehicle will ask "Do you need a weight-loss diet plan?" This shows that although the role changes, the user's basic information can be shared between different roles, thus improving the alignment between voice interaction and user needs.

[0106] A1-D1 is an example of the first scenario, which is: when the same user uses different roles, the user's basic information is shared between the different roles, and voice interaction is carried out based on the basic information.

[0107] That is, sharing the user's basic information in continuous or discontinuous voice interactions performed by the same user using different roles.

[0108] Understandably, continuous or discontinuous voice interaction refers to continuous or discontinuous voice interaction rounds. This can mean using different roles within the same voice interaction round, i.e., using different roles in continuous voice interaction. For example, a user activates the voice interaction function, uses the first role, and then selects the second role to continue voice interaction. Alternatively, it can mean using different roles in different voice interaction rounds, i.e., using different roles in discontinuous voice interaction. For example, a user activates the voice interaction function, uses the first role, then closes the voice interaction function, and after a period of time, activates the voice interaction function again and selects the second role. In this case, the first role was used in the first round of voice interaction, and the second role was used in the second round.

[0109] The above examples use the activation of the voice interaction function as the start of a voice interaction round. In addition, the switching of the voice interaction function can also be used as the start of a voice interaction round; a certain time interval can also be used to distinguish different voice interaction rounds. For example, after using the first role for voice interaction, after a certain time interval, the second role is used for voice interaction. In this case, the first role is used in the first round of voice interaction, and the second role is used in the second round of voice interaction.

[0110] E1. Based on the operation of switching the voice interaction function to the target voice zone, voice interaction is performed in the target voice zone, and the virtual image of the second character is displayed on the screen corresponding to the target voice zone.

[0111] For example, if the driver needs to switch the voice interaction function to the left-hand audio zone in the second row where the child is located, the voice interaction function will be switched to the left-hand screen in the second row. In this case, the vehicle can interact with the child via voice. F1: Invoke the interactive content and, based on the interactive content, perform voice interaction in the target audio zone.

[0112] Although it remains the second role, the user performing the voice interaction has changed from the driver to a child in the second-row left voice zone. Therefore, for example, the relevant information is queried from the interaction content recorded in B1, and voice interaction is performed in the target voice zone based on the retrieved relevant information. For example, when using the second role for voice interaction in the driver's voice zone, animation A was played based on the driver's voice "play animation A". In this step, after receiving the child's voice "animation", the system asks "Do you want to continue playing animation A?"

[0113] This step is an example of the second scenario, which is: the ability to share associated information of voice interaction between different roles and / or different users, thereby providing a function for continuous interaction of associated information.

[0114] That is, in continuous or discontinuous voice interactions between different users using different roles, the related information in the historical interaction information is shared.

[0115] Among them, the associated information in the historical interaction information can be understood as the information in the historical voice interaction record that is related to the current voice interaction. For example, in the previous example, the associated information of "cartoon" in the current voice interaction is "cartoon A" in the historical interaction information.

[0116] Continuing interaction can be understood as the re-emphasis of the first piece of information (such as animation A) in a previous interaction.

[0117] For an explanation of continuous or discontinuous voice interaction, please refer to the first case; it will not be repeated here.

[0118] G1. Based on the operation of selecting the first character and the first voice zone in the voice interaction interface, switch the voice interaction function back to the first voice zone, and change the virtual image of the second character in the voice interaction interface to the virtual image of the first character.

[0119] In other words, in this step, the first role is selected again. Continuing with the previous example, the driver can switch the voice interaction function from the target voice zone back to the driver voice zone and then select the first role, or select the first role first and then switch back to the driver voice zone, or select the first role and / or driver voice zone again via remote control (see the example below).

[0120] H1. If the first role is selected again, voice interaction will be conducted based on the interaction record corresponding to the first role.

[0121] In some implementations, voice interaction based on interaction records means that when responding to the user's voice output, the voice interaction is based on the content of the previous voice interaction when using the first role. For example, if the interaction content includes recommending restaurant A to the user, then when the user says "find a good restaurant", restaurant A is found in the interaction content, and the voice interaction message "how was that restaurant A last time?" is issued.

[0122] G1-H1 is an example of the third scenario in this step, which is: when the same user uses the same role in discontinuous voice interactions, the voice interaction is based on the historical interaction records of that role.

[0123] That is, sharing historical interaction information in discontinuous voice interactions performed by the same user using the same role.

[0124] For a detailed explanation of discontinuous voice interaction, please refer to the explanation of the first case. In this step, the voice interaction function switches back from the target voice region to the first voice region, and a new voice interaction round begins. Alternatively, the voice interaction function may be turned off and then restarted, in which case a new voice interaction round begins.

[0125] In some implementations, voice interaction records of different users and / or roles are stored in different storage spaces; that is, voice interaction records are stored according to at least one dimension of user and role.

[0126] In this embodiment, historical voice interaction content and user basic information are remembered based on role classification. For the same user, historical information can be transmitted based on the same role, making the voice interaction based on the same role increasingly "understand" the user. At the same time, basic user information can be transmitted between different roles, making the voice interaction more suitable for a particular user (such as a car owner) to meet the user's basic needs. For different roles and / or different users, relevant information can also be continuously interacted, thereby providing a richer interactive experience.

[0127] It is understood that in this embodiment, the execution order of switching between voice interaction functions and switching between roles is only an example and not a limitation. That is, the switching of roles does not depend on the switching of voice interaction functions, and the switching of voice interaction functions has no causal relationship with the switching of roles. Furthermore, the information recorded in different voice regions can be shared, or it can be recorded separately according to the voice regions, and the recorded information is not shared between different voice regions. All of these situations are included in the protection scope of this application.

[0128] The embodiments of this application provide another voice interaction method, mainly describing the pause function. The voice interaction process based on the pause control (not shown in the accompanying drawings) includes the following steps:

[0129] A2. Display the voice interaction interface on the screen corresponding to the voice zone that implements the first voice interaction function.

[0130] The first voice interaction function refers to the voice interaction function similar to "making a phone call" proposed in this application, namely the "voice interaction function" described in the foregoing embodiments.

[0131] The voice interaction interface displays a pause control.

[0132] In this embodiment, the voice region that implements the first voice interaction function may be the first voice region before the first voice interaction function is switched, or it may be the target voice region after the first voice interaction function is switched.

[0133] B2. Based on the first operation of the pause control in the voice interaction interface, pause the first voice interaction function.

[0134] Pause refers to temporarily stopping the first voice interaction function. After the first voice interaction function is temporarily stopped, the paused first voice interaction function will not respond to voice, but the voice interaction interface will remain displayed. In the content displayed on the voice interaction interface, the time information used for timing will stop timing and instead display the time before the pause to indicate the duration of the first voice interaction function before the pause. The pause control displays the paused status to indicate the pause of the first voice interaction function.

[0135] Other content displayed on the voice interaction interface can remain unchanged compared to before the pause.

[0136] Understandably, because this first voice interaction function, similar to "making a phone call," can be activated without requiring the user to reactivate it, the voice of the user within the voice zone implementing the first voice interaction function will be responded to by the first voice interaction function. In cases where the user needs to talk to other users or make a phone call, rather than to engage in voice interaction, pausing the first voice interaction function can meet the user's other needs, such as other control needs of the smart terminal or non-voice interaction needs like making a phone call.

[0137] C2. When the first voice interaction function is paused, after receiving the first voice control command, the second voice interaction function is started based on the wake word in the first voice control command, and the first voice control command is responded to based on the second voice interaction function.

[0138] The second voice interaction function refers to a voice interaction function that requires a wake word for each response. For example, if a user says "Ideal classmate, open the car window," the vehicle will open the car window based on the wake word "Ideal classmate." If the user wants to play music, they need to say "Ideal classmate, play music" again, which also requires a wake word for the vehicle to play music.

[0139] The first voice interaction function focuses on providing richer chat capabilities, emphasizing voice interaction and dialogue between users and smart devices such as vehicles. For example, vehicles can use different tones and timbres based on user roles and styles to converse with users. The second voice interaction function focuses on responding to users' voice control commands to control smart devices such as vehicles.

[0140] For example, the second voice function can respond to voice commands in the vocal range of the captured wake word.

[0141] D2. Based on the second operation on the pause control, resume the first voice interaction function.

[0142] For example, after the first operation, the pause control is displayed as a paused state (the pressed state). In the paused state, after the second operation, it becomes a non-paused state (the released state), and the first voice interaction function is restored, that is, the voice interaction function similar to "making a phone call" is restored.

[0143] After the first voice interaction function is restored, it can respond to the voice of the voice zone that implemented the first voice interaction function before it was paused. That is, the voice zone that implemented the first voice interaction function remains unchanged before and after the pause. For example, if the first voice interaction function could respond to the voice of the driver's voice zone before it was paused, it will also respond to the voice of the driver's voice zone after it is restored, but will not respond to the voice of other voice zones outside the driver's voice zone.

[0144] After the first voice interaction function is restored, the time information continues to count down on the voice interaction interface.

[0145] The voice interaction method provided in this embodiment allows voice interaction functions such as "making a phone call" to be paused so as not to affect the user's needs for conversation, voice command control of smart devices, or making phone calls. It can also be resumed after being paused and can also wake up a second voice interaction function, thereby enabling more flexible voice interaction.

[0146] The “voice interaction function” mentioned in the following embodiments refers to the first voice interaction function.

[0147] In the embodiments of this application, the minimize control in the voice interaction interface is used to minimize the voice interaction interface. For example, it is minimized to the status bar. For example, the voice interaction interface 22 displays a minimize control 226, and the user minimizes the voice interaction interface 22 by clicking the minimize control 226. Figure 5 For example, minimizing the display means minimizing it to the status bar, which means displaying the minimized icon in the status bar, called status bar icon 51. Here, we take displaying capsule icon 51 in the status bar as an example. Capsule icon 51 displays time information to indicate that it is still in the voice interaction hold state (i.e., the voice interaction function remains on). It can be understood that the time information displayed in capsule icon 51 indicates the duration of the voice interaction function. Figure 5 The time information in the capsule icon 51 is for illustrative purposes only and does not constitute a limitation.

[0148] The capsule icon 51 also displays a virtual avatar. Assuming that the virtual avatar of the target character was displayed in the voice interaction interface 22 before minimization, the virtual avatar of the target character is also displayed in the capsule icon 51.

[0149] Because the capsule icon 51 is small, for example, the capsule icon 51 displays the outline of the virtual image of the target character, without displaying other information such as color.

[0150] The capsule icon 51 is for illustrative purposes only and is not intended to limit the shape, size, color, or other information of the voice interaction interface icon displayed in the status bar. The minimized voice interaction interface icon is not limited to the status bar but can be placed anywhere on the screen.

[0151] In addition to minimizing the voice interaction interface 22, the display position of the voice interaction interface 22 can also be changed on the screen, such as by dragging the voice interaction interface 22 to move it from a first position to a second position on the screen.

[0152] Minimizing or dragging the voice interaction interface can reduce the possibility of it obscuring other content displayed on the screen, thus improving convenience.

[0153] The embodiments of this application also provide a voice interaction method. Compared with the foregoing embodiments, the method focuses on the remote control function of the voice interaction function, including the following steps:

[0154] A3. Use voice interaction in the first audio range.

[0155] In this embodiment, it is assumed that the first voice zone is the passenger voice zone. For example, a voice interaction interface is displayed on the passenger screen corresponding to the passenger voice zone. Figure 2 The voice interaction interface shown.

[0156] B3. The remote control interface is displayed on the third screen.

[0157] The remote control interface is used to control the voice interaction function of at least one audio zone to be controlled.

[0158] Since the control is for the voice interaction function, at least one voice zone to be controlled is at least one voice zone that implements the voice interaction function. In this embodiment, as described in A3, the voice interaction is performed in the first voice zone. Therefore, the voice zone to be controlled is the first voice zone, such as the passenger voice zone. That is, the remote control interface controls the voice interaction function of the first voice zone.

[0159] For example, the third screen is the central control screen.

[0160] For example, the remote control interface includes a remote control object, which is an object used to implement specific control over voice interaction functions, such as a control.

[0161] For example, the controls in the remote control interface include at least one control from the voice interaction interface to reflect a similar style to the voice interaction interface, serving as a prompt that it is a remote control interface with a voice interaction interface.

[0162] To distinguish them from controls in the voice interaction interface, controls in the remote control interface are designated as "first," while controls in the voice interaction interface can be designated as "second." Specifically, the remote control interface includes: a remote control object, which includes at least one of the following: a first pause control for pausing the voice interaction function, a first exit control for exiting the voice interaction function, a first minimize control, a first voice zone selection control, a first role selection control, a first style selection control, a first duration of the voice interaction function, and information about the voice zone to be controlled.

[0163] The various controls in the remote control interface are used to control the voice interaction functions of the controlled audio region. Specifically:

[0164] The first pause control has the same function as the pause control in the aforementioned voice interaction interface (which can be referred to as the second pause control). That is, based on the first operation of the first pause control, the voice interaction function of the voice area to be controlled is paused. In this case, based on the second operation of the first pause control, the voice interaction function of the voice area to be controlled is restored.

[0165] The first exit control has the same function as the exit control in the aforementioned voice interaction interface (which can be referred to as the second exit control). That is, based on the operation of the first exit control, the voice interaction function of the controlled audio area is exited (the voice interaction interface is closed and no longer responds to voice). In addition, based on the operation of the first exit control, the remote control interface is also exited.

[0166] The first minimize control has the same function as the minimize control in the aforementioned voice interaction interface (which can be called the second minimize control). That is, based on the operation of the first minimize control, the voice interaction interface of the voice area to be controlled is minimized. In addition, based on the operation of the first minimize control, the remote control interface is also minimized.

[0167] The first voice zone selection control has the same function as the voice zone selection control in the aforementioned voice interaction interface (which can be called the second voice zone selection control), that is, based on the operation of the first voice zone selection control, the voice zone that implements the voice interaction function is selected. In another implementation, the first voice zone selection control can select the voice zone to be controlled. Assuming that multiple voice zones can implement the voice interaction function, the first voice zone selection control can select the voice zone to be controlled as the control object of the remote control interface. For example, both the passenger voice zone and the rear voice zone can implement the voice interaction function. If the user selects the passenger voice zone based on the first voice zone selection control, then the subsequent remote control interface can control the voice interaction function of the passenger voice zone.

[0168] The first role selection control has the same function as the role selection control in the aforementioned voice interaction interface (which can be called the second role selection control), that is, to select a role based on the operation of the first role selection control.

[0169] The first style selection control functions identically to the style selection control in the aforementioned voice interaction interface (which can be referred to as the second style selection control), i.e., based on the operation of the first style selection control, a certain style is selected. The virtual image corresponding to the selected character and / or style is displayed in the voice interaction interface on the screen corresponding to the voice area to be controlled. At least one of the timbre, tone, and action corresponding to the selected character and / or style is used for the voice interaction of the voice area to be controlled.

[0170] The first duration of the voice interaction function is consistent with the time information displayed in the voice interaction interface, that is, the duration during which the voice interaction function has been running.

[0171] The information of the voice region to be controlled represents the information of the voice region that implements the voice interaction function.

[0172] Example of a remote control interface Figure 6 As shown, the remote control interface 61 is displayed on the central control screen. The remote control interface 61 includes a first exit control 611 (also known as a first close control), a first voice zone selection control 612, a first role selection control 613, a first minimize control 614, and a first duration 615.

[0173] For example, when the user clicks the minimize control, such as Figure 6 As shown, the status bar icon of the remote control interface 61 is displayed on the central control screen. The status bar icon is as described above and will not be repeated here. For example, the remote control interface 61 can also display text for voice interaction, such as the text for voice interaction displayed in the aforementioned voice interaction interface.

[0174] Compared to the voice interaction interface 22, the remote control interface 61 no longer displays a virtual avatar, but instead displays controls with operational functions. Figure 6 In this system, the remote control interface 61 retains some of the controls found in the voice interaction interface 22. Alternatively, the remote control 61 can display even fewer controls than those in the voice interaction interface 22. It can be understood that the remote control interface 61 can be viewed as a thumbnail of the voice interaction interface 22. This thumbnail display method simplifies the content displayed on the remote control interface, reduces the impact on other content displayed on the third screen, and ensures control over the voice interaction function.

[0175] Understandably, the remote control interface displayed on the third screen can also be moved.

[0176] Figure 6 In this context, the display position of the remote control interface, as well as the display positions of the remote control interface and the voice interaction interface on the screen, do not limit each other.

[0177] For example, the remote control interface can be displayed on multiple screens so that users in multiple audio zones can control the voice interaction function.

[0178] C3. Based on the operation of selecting the target audio region on the remote control interface, voice interaction is performed in the target audio region.

[0179] In this embodiment, it is assumed that the target audio zone is the second row right. Continuing the previous example, based on the operation of the first audio zone selection control 612 in the remote control interface 61, after selecting the second row right audio zone, the voice interaction interface switches from the passenger screen to the screen corresponding to the second row right audio zone. At this time, the example of the information of the audio zone to be controlled displayed in the remote control interface is "Chatting in the second row right". It can be understood that before the audio zone switching, the information of the audio zone to be controlled displayed in the remote control interface could be "Chatting in the passenger seat" (not shown in the figure).

[0180] Compared to the method of switching audio zones based on the voice interaction interface in the previous embodiments, this step is based on switching audio zones based on the remote control interface.

[0181] D3. Keep the remote control interface displayed on the third screen.

[0182] As can be seen from B3, the remote control interface is displayed on the third screen before and after the voice interaction function is switched. That is, if the remote control interface is displayed on the third screen before the voice zone is switched, it will still be displayed on the third screen after the voice zone is switched. In other words, the voice zone switching for the voice interaction function does not affect the display of the remote control interface. The purpose is to enable continuous remote control of the voice interaction function.

[0183] In this embodiment, a remote control interface is displayed on the third screen. Based on the remote control interface, the voice interaction function can be controlled. When the user is far away from the screen corresponding to the voice zone that enables the voice interaction function, it is convenient for the user to control the voice interaction function. For example, if the voice interaction function is currently enabled in the passenger voice zone and the voice interaction interface is displayed on the passenger screen, while the remote control interface is displayed on the central control screen, it is beneficial for the driver to control the voice interaction function. Furthermore, when the voice zone that enables the voice interaction function is switched to the second row left rear, the remote control interface is still displayed on the central control screen, which is convenient for the driver to control the voice interaction function.

[0184] It is understandable that the remote control interface may not be displayed before the audio zone switch, i.e., B3 above is not executed. Instead, the remote control interface may be displayed on the third screen after the audio zone switch. The specific screen of the third screen can be pre-configured. For example, if the third screen is the central control screen, it is also possible that, regardless of whether the audio zone implementing the voice interaction function is the audio zone corresponding to the third screen, the remote control interface will not be displayed. For example, if the audio zone implementing the voice interaction function is the driver's audio zone and the third screen is the central control screen, then when a voice interaction function similar to "making a phone call" is implemented in the driver's audio zone, the remote control interface will not be displayed on the central control screen.

[0185] It is understood that the functions implemented by the controls in the above embodiments can all be implemented in other ways, such as through at least one operation of voice commands and gestures.

[0186] The examples of implementing various functions based on voice or interface involved in the above embodiments can be used in different embodiments and are not limited to one embodiment.

[0187] Figure 7 This application also provides another voice interaction method in its embodiments, which, compared with the above embodiments, focuses more on the backend implementation logic. Figure 7 The process includes the following steps:

[0188] S21. Based on the startup operation, the voice interaction interface is displayed on the vehicle's first screen.

[0189] The activation operation is used to trigger the display of the voice interaction interface. For example, the activation operation can be at least one of voice and touch operations. For instance, a virtual button is displayed on the vehicle's central control screen; after the user clicks the virtual button, the voice interaction interface is displayed. The voice interaction interface can be found in the above embodiments and will not be described again here.

[0190] It is understood that the voice interaction method provided in the embodiments of this application can coexist with existing voice interaction functions (such as the aforementioned second voice interaction function). In this case, the activation operation is different from the operation of triggering existing voice interaction functions.

[0191] When the voice interaction interface is launched, a default voice interaction function similar to "making a phone call" is activated. Alternatively, based on the launch operation, the voice interaction function similar to "making a phone call" can be enabled without displaying the voice interaction interface, and then the voice interaction interface can be launched based on other operations.

[0192] S22. The vehicle-side connects to the cloud service based on the startup operation.

[0193] The cloud service is used to detect speech from the sound signal and obtain the response speech by calling the target large model. After obtaining the response speech, the response speech is sent to the vehicle.

[0194] For example, connecting to cloud services is based on SDK technology.

[0195] S23. After successfully connecting to the cloud service, the vehicle starts the timer on the voice interaction interface.

[0196] The timing is used to count the duration of the voice interaction function. The timing result can be found in the time information 225 of the voice interaction function in the above embodiment.

[0197] Understandably, after successfully connecting to the cloud service, the voice interaction function based on the voice interaction interface can be used normally.

[0198] S24. The vehicle-mounted system collects the in-vehicle audio signals in real time and uploads the collected audio signals to the cloud service.

[0199] S25. The cloud service detects speech in the received audio signal and obtains the response speech by calling the target large model.

[0200] The difference between the target large model and existing large language models is that the target large model can process speech rather than text, that is, the input data is speech and the output data is also speech.

[0201] That is, the input data of the target large model is the speech detected from the sound inside the car, and the output data of the target large model is the speech response to the speech.

[0202] For example, cloud services use Voice Activity Detection (VAD) technology to detect speech from audio signals.

[0203] S26. The cloud service sends a response voice message to the vehicle.

[0204] S27, Vehicle-side outputs response voice.

[0205] It is possible that after connecting to the cloud service, the connection may be lost due to reasons such as weak network signal. In this case, the vehicle will reconnect to the cloud service. For details, please refer to S28-S29.

[0206] Outputting voice input means engaging in voice interaction. For example, the voice input response can be based on the character displayed on the voice interaction interface.

[0207] Understandably, if the connection to the cloud service fails, proceed with the following steps:

[0208] S28. If the vehicle fails to connect to the cloud service, it will reconnect to the cloud service and maintain the display of the voice interaction interface.

[0209] For example, after failing to connect to a cloud service, wait for a certain period of time, such as 20 seconds, before reconnecting to the cloud service to prevent unnecessary occupation of network resources.

[0210] During the reconnection process, the voice interaction interface remains on display so that the user is unaware of the reconnection, thereby improving the user experience.

[0211] S29. If the vehicle fails to reconnect to the cloud service, a connection failure message will be displayed on the voice interaction interface.

[0212] For example, the connection failure message includes a text prompt and a control for reconnecting. The user can trigger the vehicle to reconnect to the cloud service by interacting with the reconnection control.

[0213] Understandably, connection failure can also be indicated through voice interaction interfaces or other means.

[0214] For example, the voice interaction interface can also display or provide a voice prompt indicating that the connection failure was caused by a network error or excessive cloud service load (i.e., too many users).

[0215] The voice interaction process provided in this embodiment uses only voice as input and output data for the target large model, thus saving the steps of voice-to-text conversion and shortening the technical link, resulting in lower latency. Furthermore, the vehicle directly transmits all collected sound signals to the cloud service, and voice detection is no longer performed by the vehicle but by the cloud service, which further improves timeliness and reduces latency, thereby enhancing the response time of voice interaction.

[0216] In addition, considering the specific features of the aforementioned voice interaction interface, the method provided in this embodiment can offer a more timely, immersive, personalized, and human-like voice interaction experience.

[0217] It should be noted that the duration of the voice interaction function refers to the display duration of the voice interaction interface, or the duration of the connection with the server that implements the voice interaction function, but not the duration of the voice interaction with the user.

[0218] Understandable Figure 7 The process shown is the process of starting the voice interaction interface. After starting, the voice interaction interface can be switched between different voice ranges, and the operations involved in the above embodiments can also be performed based on the voice interaction interface.

[0219] Based on the above embodiments, embodiments of this application also disclose a voice interaction method, including: displaying a voice interaction interface based on a target operation, and displaying a first virtual image of a first character on the voice interaction interface.

[0220] The target operation can be a "start operation" as described above, including at least one of voice, gesture, or operation on a physical or virtual switch. Examples and descriptions of the voice interaction interface can be found in the above embodiments and... Figures 2-6 As shown, it will not be elaborated further here.

[0221] For example, the method further includes performing voice interaction based on a first style of the first character. For example, the first style corresponds to at least one of the following: the first virtual image, a first tone of voice, a first action, and a first timbre. Style-related content can be found in the above embodiments.

[0222] For example, before displaying the voice interaction interface based on the target operation, the method further includes: selecting the first role based on at least one of account information, user biometric information, voice region information, and voice interaction data. For details, please refer to the aforementioned content on selecting the first role based on target information.

[0223] In addition to selecting the first role based on target information, another approach is for the user to actively select the first role, such as selecting the first role based on the role selection control in the voice interaction interface. For details, please refer to the above embodiments.

[0224] For example, selecting the first role includes selecting the first role from a plurality of roles, wherein the plurality of roles includes child roles and / or adult roles. In the above embodiments, the example mainly focuses on child roles; adult roles are similar and will not be described in detail here.

[0225] For example, the voice interaction interface also displays at least one of the following: a second pause control to pause the voice interaction function, a second exit control to exit the voice interaction function, a second minimize control, a second voice zone selection control, a second role selection control, a second style selection control, a second duration of the voice interaction function, and the text of the voice interaction. See the above embodiments for details.

[0226] For example, the method further includes: receiving an operation to select a second character; and switching the first virtual avatar of the first character to a second virtual avatar having the second character. For more information on character selection, please refer to [link / reference]. Figure 4 Corresponding implementation examples.

[0227] For example, selecting a second role includes: selecting the second role based on the voice interaction interface, or selecting the second role based on the remote control interface of the voice interaction interface. The operation can be a touchscreen operation or a voice operation, etc., as detailed in the above embodiments.

[0228] For example, after displaying the voice interaction interface, the method further includes: based on a minimize operation, displaying a minimized version of the voice interaction interface, wherein the minimized version displays information about the first character. See details... Figure 5 and Figure 6 Corresponding implementation examples.

[0229] The above embodiments focus on different technical features for description. It is understood that different embodiments can be referred to each other, that is, the steps in each embodiment can be freely combined to form a new solution.

[0230] The example diagrams of various interfaces in this application and some icons that are unrelated to the technical solutions of this application are all well-known icons that may be displayed on the vehicle's display screen, and will not be described again in the embodiments of this application.

[0231] The embodiments of this application also disclose a voice interaction device, including: a module that implements the voice interaction method provided in the foregoing method embodiments.

[0232] In some implementations, the voice interaction device includes an interaction module and a switching module.

[0233] The interaction module is used for voice interaction in the first audio region, and the switching module is used for voice interaction in the target audio region based on the operation of switching the voice interaction function to the target audio region.

[0234] For example, the interaction module is further configured to display a voice interaction interface on a first screen corresponding to the first voice region before performing voice interaction in the target voice region, based on the operation of switching the voice interaction function to the target voice region. Therefore, the operation of the switching module to switch the voice interaction function to the target voice region includes: selecting the target voice region in the voice interaction interface.

[0235] For example, the switching module is also used to display the voice interaction interface on the second screen corresponding to the target voice region after the operation of selecting the target voice region is performed in the voice interaction interface.

[0236] For example, a first duration is displayed in the voice interaction interface displayed on the first screen. The first duration represents the duration that the voice interaction function has lasted from the start of the function until the function is switched. A second duration is displayed in the voice interaction interface displayed on the second screen. The second duration is obtained based on the sum of the first duration and the third duration, where the third duration is the duration that the voice interaction function has lasted in the target voice region.

[0237] For example, the device further includes: a remote control module for displaying a remote control interface on a third screen, the remote control interface being used to control the voice interaction function of at least one controllable audio zone.

[0238] For example, the remote control interface includes: a remote control object, which includes at least one of the following: a first pause control to pause the voice interaction function, a first exit control to exit the voice interaction function, a first minimize control, a first voice zone selection control to select the voice zone to implement the voice interaction function, a first role selection control, a first style selection control, a first duration of the voice interaction function, and information about the voice zone to be controlled in the voice interaction interface.

[0239] For example, the remote control module is also configured to maintain the display of the remote control interface on the third screen after the voice interaction function is switched from the first voice zone to the target voice zone. The remote control interface is the remote control interface displayed on the third screen before the voice interaction function is switched.

[0240] For example, the device further includes an interface display module for displaying a voice interaction interface on the screen corresponding to the voice zone where the voice interaction function is implemented. The voice interaction interface displays at least one of the following: a second pause control for pausing the voice interaction function, a second exit control for exiting the voice interaction function, a second minimize control for the voice interaction interface, a second voice zone selection control, a second character selection control in the voice interaction interface, a second style selection control, a second duration of the voice interaction function, and the text of the voice interaction.

[0241] For example, the device further includes a pause module, configured to pause the voice interaction function based on a first operation on the first pause control and / or the second pause control; receive a first voice control command; respond to the first voice control command; and resume the voice interaction function based on a second operation on the first pause control and / or the second pause control.

[0242] For example, the device further includes a first character selection module, used to display a first virtual image of the first character on the voice interaction interface based on the operation of selecting the first character through the first character selection control and / or the second character selection control.

[0243] For example, the device further includes a style selection module, used to display a second virtual image of the first character on the voice interaction interface based on the operation of selecting a first style through the first style selection control and / or the second style selection control, the second virtual image corresponding to the first style.

[0244] For example, the interaction module or the switching module performs voice interaction by using at least one of the timbre, tone, and action corresponding to the first style.

[0245] For example, the device further includes a second role selection module, used to select a first role for voice interaction from multiple roles based on target information before the voice interaction is performed. The target information includes at least one of account information, user biometric information, voice region information, and voice interaction data.

[0246] For example, the device further includes a first memory module for performing voice interaction based on the interaction record when the first role is selected again, the interaction record including: historical interaction data of voice interaction using the first role.

[0247] For example, the device further includes a second memory module for performing voice interaction based on basic information when a second role is selected, the basic information including: basic information of the user who uses the first role for voice interaction.

[0248] For example, the interaction module or the switching module performs voice interaction, including: collecting sound signals and uploading them to a cloud service, wherein the cloud service is used to detect the voice from the sound signals and obtain the response voice of the voice by calling a target large model; and obtaining the response voice from the cloud service.

[0249] In some other implementations, the voice interaction device includes a display module for displaying a voice interaction interface based on a target operation, wherein a first virtual image of a first character is displayed on the voice interaction interface.

[0250] For example, the device further includes an interaction module for performing voice interaction based on a first style of the first character.

[0251] For example, the device further includes a role selection module, used to select the first role based on at least one of account information, user biometric information, voice region information, and voice interaction data before displaying the voice interaction interface based on the target operation.

[0252] For example, the role selection module is specifically used to select the first role from a plurality of roles, the plurality of roles including child roles and / or adult roles.

[0253] For example, the role selection module is further configured to receive an operation to select a second role; and switch the first virtual image of the first role to a second virtual image with the second role.

[0254] For example, the operation of selecting the second role includes: selecting the second role based on the voice interaction interface, or selecting the second role based on the remote control interface of the voice interaction interface.

[0255] For example, the display device is further configured to, after displaying the voice interaction interface, display a minimized interface of the voice interaction interface based on a minimize operation, wherein the minimized interface displays information about the first character.

[0256] The specific implementation methods of the functions of each of the above modules can be found in the above method implementation examples, and will not be repeated here.

[0257] The voice interaction device provided in this embodiment improves the convenience of voice interaction functions and provides a more flexible voice interaction method, which is conducive to obtaining a better user experience.

[0258] Figure 8 A schematic diagram of the structure of a vehicle provided in an embodiment of this disclosure is shown.

[0259] like Figure 8 As shown, the vehicle may include a processor 71 and a memory 72 storing computer program instructions.

[0260] Specifically, the processor 71 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0261] Memory 72 may include mass storage for information or instructions. For example, and not limitingly, memory 72 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 72 may include removable or non-removable (or fixed) media. Where appropriate, memory 72 may be internal or external to the integrated gateway device. In a particular embodiment, memory 72 is non-volatile solid-state memory. In a particular embodiment, memory 72 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0262] The processor 71 performs the steps of the voice interaction method provided in the embodiments of this disclosure by reading and executing computer program instructions stored in the memory 72.

[0263] In one example, the vehicle may also include a transceiver 73 and a bus 74. Wherein, as... Figure 8 As shown, the processor 71, memory 72 and transceiver 73 are connected via bus 74 and communicate with each other.

[0264] Bus 74 may include hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 74 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0265] Furthermore, the vehicle may also include human-machine interaction devices, such as in-vehicle screens, which can communicate with the processor via a bus. These devices can display voice control commands and their corresponding execution results to the user, making voice interaction visual and improving the user's voice interaction experience.

[0266] This disclosure also provides a computer-readable storage medium that can store a computer program, which, when executed by a processor, enables the processor to implement the voice interaction method provided in this disclosure.

[0267] The aforementioned storage medium may, for example, include a memory 72 containing computer program instructions, which can be executed by the processor 71 of the voice control device to complete the voice interaction method provided in the embodiments of this disclosure. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), compact disc ROM (CD-ROM), magnetic tape, floppy disk, and optical data storage device. The aforementioned computer program may be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code may be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0268] The contents of the above embodiments can be referred to in relation to each other.

[0269] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A voice interaction method, characterized in that, include: Voice interaction is performed in the first audio region; Based on the operation of switching the voice interaction function to the target voice region, voice interaction is performed in the target voice region.

2. The method according to claim 1, characterized in that, Before performing voice interaction in the target voice region, the method based on switching the voice interaction function to the target voice region includes: The voice interaction interface is displayed on the first screen corresponding to the first voice zone; The operation of switching the voice interaction function to the target audio region includes: The operation of selecting the target audio region in the voice interaction interface.

3. The method according to claim 2, characterized in that, After selecting the target voice region in the voice interaction interface, the process further includes: The voice interaction interface is displayed on the second screen corresponding to the target audio region.

4. The method according to claim 3, characterized in that, The first duration is displayed in the voice interaction interface on the first screen. The first duration represents the duration that the voice interaction function has lasted from the start of the function until the function is switched. The second duration is displayed in the voice interaction interface on the second screen. The second duration is obtained based on the sum of the first duration and the third duration. The third duration is the duration that the voice interaction function has lasted in the target voice region.

5. The method according to any one of claims 1-4, characterized in that, Also includes: The remote control interface is displayed on the third screen, which is used to control the voice interaction function of at least one audio zone to be controlled.

6. The method according to claim 5, characterized in that, The remote control interface includes: The remote control object includes at least one of the following: a first pause control for pausing the voice interaction function, a first exit control for exiting the voice interaction function, a first minimize control, a first voice zone selection control, a first role selection control, a first style selection control, a first duration of the voice interaction function, and information about the voice zone to be controlled.

7. The method according to claim 5 or 6, characterized in that, Before the voice interaction function is switched, the remote control interface is displayed on the third screen. After the voice interaction function is switched from the first voice zone to the target voice zone, the remote control interface is displayed on the third screen.

8. The method according to any one of claims 1-7, characterized in that, Also includes: On the screen corresponding to the voice zone where the voice interaction function is implemented, a voice interaction interface is displayed. The voice interaction interface displays at least one of the following: a second pause control to pause the voice interaction function, a second exit control to exit the voice interaction function, a second minimize control, a second voice zone selection control, a second role selection control, a second style selection control, a second duration of the voice interaction function, and the text of the voice interaction.

9. The method according to any one of claims 6-8, characterized in that, Also includes: Based on the first operation on the first pause control and / or the second pause control, the voice interaction function is paused; Receive the first voice control command; Respond to the first voice control command; The voice interaction function is restored based on the second operation performed on the first pause control and / or the second pause control.

10. The method according to any one of claims 6-9, characterized in that, Also includes: Based on the operation of selecting a target character through the first character selection control and / or the second character selection control, the first virtual image of the target character is displayed on the voice interaction interface.

11. The method according to claim 10, characterized in that, Also includes: Based on the operation of selecting a target style through the first style selection control and / or the second style selection control, a second virtual image of the target character is displayed on the voice interaction interface, and the second virtual image corresponds to the target style.

12. The method according to claim 11, characterized in that, The voice interaction includes: Voice interaction is performed using at least one of the timbre, tone, and gestures corresponding to the target style.

13. The method according to any one of claims 1-12, characterized in that, Before the voice interaction, it also includes: Based on target information, a target role for voice interaction is selected from multiple roles. The target information includes at least one of account information, user biometric information, voice region information, and voice interaction data.

14. The method according to any one of claims 10-13, characterized in that, Also includes: If the target role is selected again, voice interaction is performed based on the interaction record, which includes: historical interaction data of voice interaction using the target role.

15. The method according to any one of claims 10-14, characterized in that, Also includes: When a second role is selected, voice interaction is performed based on basic information, including the basic information of the user who uses the target role for voice interaction.

16. The method according to any one of claims 1-15, characterized in that, The voice interaction includes: The system collects sound signals and uploads them to a cloud service. The cloud service is used to detect speech from the sound signals and obtain the response speech of the speech by calling a target large model. The response voice is obtained from the cloud service.

17. A voice interaction method, characterized in that, include: Based on the target operation, a voice interaction interface is displayed, and the first virtual image of the first character is displayed on the voice interaction interface.

18. The method according to claim 17, characterized in that, Also includes: Based on the first character's first style, voice interaction is performed.

19. The method according to claim 17 or 18, characterized in that, Before displaying the voice interaction interface based on the target operation, the method further includes: The first role is selected based on at least one of the following: account information, user biometric information, voice region information, and voice interaction data.

20. The method according to claim 19, characterized in that, The selection of the first role includes: Select the first character from a plurality of characters, the plurality of characters including child characters and / or adult characters.

21. The method according to any one of claims 17-20, characterized in that, Also includes: Receive the option to select a second role; Switch the first virtual image of the first character to a second virtual image that has the second character.

22. The method according to claim 21, characterized in that, The operation of selecting the second role includes: The second character's operation can be selected based on the voice interaction interface, or the second character's operation can be selected based on the remote control interface of the voice interaction interface.

23. The method according to any one of claims 17-22, characterized in that, Following the display of the voice interaction interface, the system also includes: Based on the minimize operation, the minimized interface of the voice interaction interface is displayed, and the minimized interface displays the information of the first character.

24. A voice interaction device, characterized in that, include: A module for implementing the voice interaction method according to any one of claims 1-23.

25. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the voice interaction method as described in any one of claims 1-23.

26. A vehicle, characterized in that, include: A memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the voice interaction method as described in any one of claims 1-23.

Images