Voice interaction method and server

Abstract

The application provides a voice interaction method and a server. The voice interaction method comprises the following steps: receiving voice requests of users in at least two different sound areas in a cabin forwarded by a vehicle; determining whether each voice request belongs to a single-turn voice dialogue or a multi-turn voice dialogue according to a predetermined rule; determining at least two different feedback modes including display of a user interface control in a vehicle system graphical user interface, display of a text message in the vehicle system graphical user interface, and text-to-speech (TTS) reply, according to a result of the determination; and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request. The scheme provided by the application can provide timely feedback for users in different sound areas, so that users in different sound areas can perceive the result feedback in time, and the use experience of the users is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice interaction, and in particular to a voice interaction method and a server. BACKGROUND

[0002] Voice interaction is a new generation of interaction mode based on voice input. With the continuous development of the automobile industry and human-computer interaction technology, intelligent vehicles also provide voice interaction functions for users.

[0003] In related technologies, intelligent vehicles can carry voice assistants to provide natural human-computer interaction. Users in the vehicle can not only control vehicle software such as navigation and music through voice, but also control vehicle hardware such as vehicle windows and air conditioners. In the vehicle space, there may be a situation in which multiple users simultaneously perform voice interaction with the vehicle voice assistant.

[0004] In related technologies, although the vehicle voice assistant can recognize the voice prints of different users, it cannot simultaneously process voice requests initiated by multiple users and reply. Generally, it can only interact in a single round serving only one user, which cannot provide timely feedback for multiple users and cannot guarantee that all users can perceive the result feedback. SUMMARY

[0005] To solve or partially solve the problems in related technologies, the present application provides a voice interaction method and a server, which can provide timely feedback for users in different sound zones, so that users in different sound zones can perceive the result feedback in time and improve the user experience.

[0006] The first aspect of the present application provides a voice interaction method, comprising:

[0007] receiving voice requests of users in at least two different sound zones in the vehicle cabin forwarded by the vehicle;

[0008] determining whether each voice request belongs to a single-turn voice dialogue or a multi-turn voice dialogue according to a predetermined rule;

[0009] determining at least two different feedback modes including vehicle system graphical user interface user interface control display, vehicle system graphical user interface text message display, and text-to-speech (TTS) reply according to the determination result, and issuing the feedback modes to the vehicle to respond to each voice request and complete voice interaction.

[0010] The determination of whether each voice request belongs to a single-turn voice dialogue or a multi-turn voice dialogue according to a predetermined rule comprises:

[0011] determining that it belongs to a single-turn voice dialogue according to that the voice request can be completed in a single turn;

[0012] determining that it belongs to a multi-turn voice dialogue according to that the voice request cannot be completed in a single turn.

[0013] determining at least two different feedback modes including display of user interface controls in the graphical user interface of the in-vehicle system, display of text messages in the graphical user interface of the in-vehicle system, and TTS reply from text to speech, according to the judgment result, and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request, comprising:

[0014] determining three different feedback modes including display of user interface controls in the graphical user interface of the in-vehicle system, display of text messages in the graphical user interface of the in-vehicle system, and TTS reply from text to speech, according to the judgment result, and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request.

[0015] determining three different feedback modes including display of user interface controls in the graphical user interface of the in-vehicle system, display of text messages in the graphical user interface of the in-vehicle system, and TTS reply from text to speech, according to the judgment result, and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request.

[0016] determining three different feedback modes including display of user interface controls in the graphical user interface of the in-vehicle system, display of text messages in the graphical user interface of the in-vehicle system, and TTS reply from text to speech, according to the judgment result, and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request.

[0017] if the first sentence of the voice reply to the voice request of the user corresponding to the at least two different sound zones is a conclusion, determining that the reply to each voice request is the feedback mode of TTS reply, and sequentially feeding back.

[0018] determining three different feedback modes including display of user interface controls in the graphical user interface of the in-vehicle system, display of text messages in the graphical user interface of the in-vehicle system, and TTS reply from text to speech, according to the judgment result, and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request.

[0019] determining three different feedback modes including display of user interface controls in the graphical user interface of the in-vehicle system, display of text messages in the graphical user interface of the in-vehicle system, and TTS reply from text to speech, according to the judgment result, and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request.

[0020] if the first sentence of the voice reply to the voice request of the user corresponding to the at least two different sound zones is not a conclusion, and the reply content to the voice request of the driver sound zone user belongs to long text, determining that the reply to the voice request of the driver sound zone user is the feedback mode of display of user interface controls in the graphical user interface of the in-vehicle system, and the reply to the voice request of the non-driver sound zone user is the feedback mode of voice broadcast.

[0021] determining three different feedback modes including display of user interface controls in the graphical user interface of the in-vehicle system, display of text messages in the graphical user interface of the in-vehicle system, and TTS reply from text to speech, according to the judgment result, and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request.

[0022] determining three different feedback modes including display of user interface controls in the graphical user interface of the in-vehicle system, display of text messages in the graphical user interface of the in-vehicle system, and TTS reply from text to speech, according to the judgment result, and issuing the feedback modes to the vehicle to complete voice interaction in response to each voice request.

[0023] If the first sentence of the voice reply to the voice request of the user corresponding to the at least two different sound areas is not a conclusion, and the reply content of the voice request of the main driver sound area user belongs to short text, it is determined that the reply to the voice request of the main driver sound area user is the feedback mode of the vehicle-mounted system graphical user interface text message display, and the reply to the voice request of the non-main driver sound area user is the feedback mode of voice broadcast.

[0024] The at least two different feedback modes including vehicle-mounted system graphical user interface user interface control display, vehicle-mounted system graphical user interface text message display, and text-to-speech (TTS) reply are determined according to the judgment result, and the feedback modes are sent to the vehicle to complete voice interaction in response to each voice request.

[0025] After it is judged that each voice request does not belong to a single-turn voice dialogue, two different feedback modes including vehicle-mounted system graphical user interface user interface control display and vehicle-mounted system graphical user interface text message display are determined, and the feedback modes are sent to the vehicle to complete voice interaction in response to each voice request.

[0026] The two different feedback modes including vehicle-mounted system graphical user interface user interface control display and vehicle-mounted system graphical user interface text message display are determined after it is judged that each voice request does not belong to a single-turn voice dialogue, and the feedback modes are sent to the vehicle to complete voice interaction in response to each voice request.

[0027] After it is judged that each voice request does not belong to a single-turn voice dialogue,

[0028] For a voice request belonging to a single-turn voice dialogue, if the reply content of the voice request belongs to long text, it is determined that the reply to the voice request is the feedback mode of vehicle-mounted system graphical user interface user interface control display,

[0029] For a voice request belonging to a multi-turn voice dialogue, it is determined that the reply to the voice request is the feedback mode of voice broadcast.

[0030] The two different feedback modes including vehicle-mounted system graphical user interface user interface control display and vehicle-mounted system graphical user interface text message display are determined after it is judged that each voice request does not belong to a single-turn voice dialogue, and the feedback modes are sent to the vehicle to complete voice interaction in response to each voice request.

[0031] After it is judged that each voice request does not belong to a single-turn voice dialogue,

[0032] For a voice request belonging to a single-turn voice dialogue, if the reply content of the voice request belongs to short text, it is determined that the reply to the voice request is the feedback mode of vehicle-mounted system graphical user interface text message display.

[0033] For a voice request belonging to a multi-turn voice dialogue, it is determined that the reply to the voice request is a feedback mode of voice broadcast.

[0034] The method further includes:

[0035] Among the two different feedback modes of the user interface control display in the vehicle-mounted system graphical user interface and the vehicle-mounted system graphical user interface text message display, the sound effect prompt mode is matched.

[0036] The second aspect of the present application provides a server, comprising:

[0037] The receiving module is configured to receive voice requests of users in at least two different sound zones in the cabin forwarded by the vehicle;

[0038] The determining module is configured to determine, according to a predetermined rule, whether each voice request received by the receiving module belongs to a single-turn voice dialogue or a multi-turn voice dialogue;

[0039] The processing module is configured to determine, according to the determination result of the determining module, at least two different feedback modes including the user interface control display in the vehicle-mounted system graphical user interface, the vehicle-mounted system graphical user interface text message display, and the TTS reply from text to voice, and to issue the feedback modes to the vehicle to complete voice interaction in response to each voice request.

[0040] The third aspect of the present application provides a server, comprising:

[0041] A processor; and

[0042] A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described above.

[0043] The fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method as described above.

[0044] The technical solution provided by the present application can include the following beneficial effects:

[0045] The application scheme receives the voice requests of the users in at least two different sound areas in the cabin forwarded by the vehicle, judges whether each voice request belongs to a single round voice dialogue or a multi-round voice dialogue according to predetermined rules, and then determines at least two different feedback modes including the display of the user interface control in the vehicle-mounted system graphical user interface, the display of the text message in the vehicle-mounted system graphical user interface, and the TTS (Text To Speech) reply according to the judgment result. Through the above processing, the problem that the instructions of other users need to wait for a long time for a reply caused by single round service is avoided, so that multiple users in different sound areas can obtain timely feedback through different feedback modes, and the result feedback can be perceived in time, thereby improving the user experience.

[0046] Further, the application scheme can determine three different feedback modes including the display of the user interface control in the vehicle-mounted system graphical user interface, the display of the text message in the vehicle-mounted system graphical user interface, and the TTS (Text To Speech) reply after judging that each voice request belongs to a single round voice dialogue, and determine two different feedback modes including the display of the user interface control in the vehicle-mounted system graphical user interface and the display of the text message in the vehicle-mounted system graphical user interface after judging that each voice request does not belong to a single round voice dialogue, and then the feedback mode is sent to the vehicle to complete the voice interaction in response to each voice request. Through the distinction between the single round voice dialogue and the multi-round voice dialogue, different feedback modes can be flexibly set for different voice requests to timely feedback the users in different sound areas.

[0047] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0048] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures, and wherein:

[0049] Figure 1 is a flowchart of the voice interaction method shown in the application;

[0050] Figure 2 is a flowchart of another voice interaction method shown in the application;

[0051] Figure 3 is a flowchart of another voice interaction method shown in the application;

[0052] Figure 4 is a flowchart of another voice interaction method shown in the application; Figure 3 is a schematic diagram of the judgment sub-flow of the toast display sound effect prompt in the application;

[0053] Figure 5 is a schematic diagram of GUI card display in the method of the present application;

[0054] Figure 6 is a schematic diagram of GUI toast display in the method of the present application;

[0055] Figure 7 is a schematic diagram of the structure of the server shown in the present application;

[0056] Figure 8 is a schematic diagram of the structure of another server shown in the present application;

[0057] Figure 9 is a schematic diagram of the structure of the server shown in the present application. DETAILED DESCRIPTION

[0058] Embodiments of the present application will be described in more detail with reference to the drawings. Although the embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0059] The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the present application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0060] It should be understood that although the terms "first," "second," "third," etc. can be employed in this application to describe various information, such information should not be limited by these terms. These terms are only used to distinguish one piece of information from another. For example, a first information can also be termed a second information, and, similarly, a second information can also be termed a first information, without departing from the scope of the present application. Therefore, the characteristics defined with "first," "second" can explicitly or implicitly include one or more of the characteristics. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0061] The technical solutions of the present application are described in detail below with reference to the accompanying drawings.

[0062] Figure 1 is a schematic diagram of the flow of the voice interaction method shown in the present application.

[0063] Referring to Figure 1 , the method comprises:

[0064] S101, receiving voice requests of users in at least two different sound zones in the vehicle cabin forwarded by the vehicle.

[0065] The voice requests can come from different sound zones in the vehicle, i.e., the users belong to different seats in the vehicle cabin. Different sound zones include, for example, the driver sound zone, the front passenger sound zone, the rear sound zone, etc. In S101, the server can receive voice requests of users in at least two different sound zones in the vehicle cabin forwarded by the vehicle, for example, receive voice requests of users in the driver sound zone, the front passenger sound zone, and the rear sound zone in the vehicle cabin forwarded by the vehicle.

[0066] S102, determining whether each voice request belongs to a single-turn voice dialogue or a multi-turn voice dialogue according to a predetermined rule.

[0067] If the voice request can be completed in a single turn, it is determined to belong to a single-turn voice dialogue; if the voice request cannot be completed in a single turn, it is determined to belong to a multi-turn voice dialogue.

[0068] S103, determining at least two different feedback modes including display of user interface controls in the vehicle system graphical user interface, display of text messages in the vehicle system graphical user interface, and TTS reply from text to voice according to the determination result, and issuing the feedback modes to the vehicle to respond to each voice request to complete voice interaction.

[0069] For example, after determining that each voice request belongs to a single-turn voice dialogue, three different feedback modes including display of user interface controls in the vehicle system graphical user interface, display of text messages in the vehicle system graphical user interface, and TTS reply from text to voice are determined, and the feedback modes are issued to the vehicle to respond to each voice request to complete voice interaction.

[0070] For example, after determining that each voice request does not belong to a single-turn voice dialogue, two different feedback modes including display of user interface controls in the vehicle system graphical user interface and display of text messages in the vehicle system graphical user interface are determined, and the feedback modes are issued to the vehicle to respond to each voice request to complete voice interaction.

[0071] The application scheme receives the voice requests of the users in at least two different sound areas in the cabin forwarded by the vehicle, judges whether each voice request belongs to a single-turn voice dialogue or a multi-turn voice dialogue according to a predetermined rule, and then determines at least two different feedback modes including display of a user interface control in a vehicle-mounted system graphical user interface, display of a text message in the vehicle-mounted system graphical user interface, and TTS reply from text to voice according to the judgment result. Through the above processing, the problem that the instructions of other users need to be waited for a long time for reply caused by single-turn service is avoided, so that the users in different sound areas can obtain timely feedback through different feedback modes, can timely perceive the result feedback, and the user experience is improved.

[0072] Figure 2 is another flowchart of a voice interaction method shown in the application.

[0073] Referring to Figure 2 The method comprises the following steps.

[0074] S201, receiving voice requests of users in at least two different sound areas in the cabin forwarded by the vehicle.

[0075] This step can refer to the description in S101.

[0076] S202, judging whether each voice request belongs to a single-turn voice dialogue or a multi-turn voice dialogue, if each voice request belongs to a single-turn voice dialogue, entering S203, if each voice request does not belong to a single-turn voice dialogue, entering S204.

[0077] Among them, if the voice request can be completed in a single turn, it is judged to belong to a single-turn voice dialogue; if the voice request cannot be completed in a single turn, it is judged to belong to a multi-turn voice dialogue.

[0078] S203, after judging that each voice request belongs to a single-turn voice dialogue, determining three different feedback modes including display of a user interface control in a vehicle-mounted system graphical user interface, display of a text message in the vehicle-mounted system graphical user interface, and TTS reply, and issuing the feedback mode to the vehicle to respond to each voice request to complete voice interaction.

[0079] For example, after judging that each voice request belongs to a single-turn voice dialogue, if the first sentence of the voice reply to the voice requests of the users in at least two different sound areas is a conclusion, it is determined that the reply to each voice request is the feedback mode of TTS reply, and the feedback is sequentially fed back.

[0080] For example, after judging that each voice request belongs to a single-turn voice dialogue, if the first sentence of the voice reply to the voice request of the user in at least two different sound zones is not a conclusion, and the reply content to the voice request of the user in the driver sound zone belongs to a long text, it is determined that the reply to the voice request of the user in the driver sound zone is a feedback mode of displaying a user interface control in a vehicle system graphical user interface, and the reply to the voice request of the user in a non-driver sound zone is a feedback mode of voice broadcast.

[0081] For example, after judging that each voice request belongs to a single-turn voice dialogue, if the first sentence of the voice reply to the voice request of the user in at least two different sound zones is not a conclusion, and the reply content to the voice request of the user in the driver sound zone belongs to a short text, it is determined that the reply to the voice request of the user in the driver sound zone is a feedback mode of displaying a text message in a vehicle system graphical user interface, and the reply to the voice request of the user in a non-driver sound zone is a feedback mode of voice broadcast.

[0082] S204, after judging that each voice request does not belong to a single-turn voice dialogue, determining two different feedback modes including displaying a user interface control in a vehicle system graphical user interface and displaying a text message in a vehicle system graphical user interface, and issuing the feedback modes to the vehicle to respond to each voice request to complete voice interaction.

[0083] For example, after judging that each voice request does not belong to a single-turn voice dialogue, for the voice request belonging to a single-turn voice dialogue, if the reply content of the voice request belongs to a long text, it is determined that the reply to the voice request is a feedback mode of displaying a user interface control in a vehicle system graphical user interface (for example, a card display mode), and for the voice request belonging to a multi-turn voice dialogue, it is determined that the reply to the voice request is a feedback mode of voice broadcast.

[0084] For example, after judging that each voice request does not belong to a single-turn voice dialogue, for the voice request belonging to a single-turn voice dialogue, if the reply content of the voice request belongs to a short text, it is determined that the reply to the voice request is a feedback mode of displaying a text message in a vehicle system graphical user interface (for example, a toast display mode), and for the voice request belonging to a multi-turn voice dialogue, it is determined that the reply to the voice request is a feedback mode of voice broadcast.

[0085] It should be noted that in the above steps, in the two different feedback modes of displaying a user interface control in a vehicle system graphical user interface and displaying a text message in a vehicle system graphical user interface, an audio prompt mode can also be matched.

[0086] The application can flexibly set different feedback modes for different voice requests and timely feedback users in different sound zones by distinguishing whether the voice request belongs to a single-turn voice dialogue or a multi-turn voice dialogue, and further distinguishing whether the reply content of the voice request belongs to short text or long text.

[0087] Figure 3 is a flowchart of another illustrated voice interaction method of the application.

[0088] The application provides a voice interaction processing method, which can adopt different feedback forms such as TTS (Text To Speech, from text to speech) reply, GUI display, and sound effect prompt to timely provide feedback to multiple users. The so-called reply degradation can refer to that when a certain instruction needs TTS audio broadcast but the audio output resource is tight due to too many conflicting instructions, different reply forms can be used for feedback to assist information transmission.

[0089] The application can be applied to the case of multi-person voice interaction, that is, it can be directed to multi-person voice requests, which can come from different sound zones in the vehicle cabin, such as the driver sound zone, the copilot sound zone, the rear sound zone, etc., i.e., belonging to different seats. The application can be applied to different dialogue scenarios, including a multi-turn voice dialogue scenario (which can also be referred to as a multi-turn task scenario) and a single-turn voice dialogue scenario (which can also be referred to as a single-turn task scenario). The so-called multi-turn voice dialogue scenario can refer to that any user is in a task-driven scenario that needs to complete a multi-turn continuous dialogue, such as navigation. The so-called single-turn voice dialogue scenario can refer to that all users who issue voice requests are in a task scenario that can be completed in a single turn, such as opening the window. Among them, according to whether the voice request can be completed in a single turn, it can be judged to belong to a single-turn voice dialogue or a multi-turn voice dialogue.

[0090] The feedback of the application to the user includes TTS reply mode and GUI (Graphical User Interface, graphical user interface) display mode. Among them, the GUI display can include card display mode (referred to as card mode) and toast (notification) display mode (referred to as toast mode) two types. The card display mode is the display mode of the user interface control in the graphical user interface, and the toast display mode is the text message display mode of the graphical user interface. The toast prompt box is a non-modal pop-up window, which pops up a small information as a reminder or message feedback, which is generally used to display the operation result or the change of the application state. The most common toast prompt box is generally a short descriptive sentence. Considering that the toast prompt box is displayed for a short time (generally only a few seconds) and occupies a small area, it is easy to be ignored by the user, so the toast display generally does not carry too much text and important information.

[0091] Among them, the equal-length text reply such as the question and answer type can adopt the card mode to carry the reply content and display on the screen of the vehicle terminal, for example, the car machine screen, and the short text reply such as the vehicle control type can adopt the toast mode to carry the reply content. The card display can refer to Figure 5 The toast display can refer to Figure 6 .

[0092] Referring to Figure 3 , the method comprises:

[0093] S301, receiving the voice requests of the users in multiple different sound zones in the vehicle cabin forwarded by the vehicle.

[0094] In this step, the voice requests of multiple users located in different sound zone positions in the vehicle cabin are received, which are forwarded by the vehicle. These voice requests can be voice requests issued simultaneously, or voice requests issued successively within a set time period, for example, within 5 seconds. For example, the voice request A issued by the user in the main driver sound zone position, the voice request B issued by the user in the copilot sound zone position, the voice request C issued by the user in the left rear position of the rear sound zone, and the voice request D issued by the user in the right rear position of the rear sound zone.

[0095] S302, judging whether each voice request belongs to a single-turn voice dialogue, if yes, entering S303, if not, entering S305.

[0096] S303, judging whether the first sentence of the reply content corresponding to each voice request is a final conclusion, if yes, entering S304, if not, entering S308.

[0097] Taking the TTS reply as an example, if the final conclusion or the central idea of the voice reply content corresponding to the voice request exists in the first sentence, at this time, it can be degenerated to only broadcast the first sentence, and no longer broadcast the complete voice reply content. For example, the question and answer type voice request, such as asking the weather, stock, translation, and characters, can be degenerated to only broadcast the first sentence. For example, the following table 1 shows:

[0098]

[0099] Table 1

[0100] It should be noted that the final conclusion or the central idea of the voice reply content is taken as an example of the first sentence, but is not limited thereto, and can also be located at the tail sentence position. No matter which position it is located at, as long as the description of the final conclusion or the central idea can be extracted for broadcasting.

[0101] S304, performing a TTS reply to each voice request.

[0102] At this time, if the final conclusion or central idea of the voice reply content corresponding to each voice request is all in the first sentence, it is determined that the reply of each voice request is a feedback mode of TTS reply, and is a feedback in turn. Therefore, the TTS reply is performed for each voice request in turn, that is, it is degraded to only broadcast the first sentence, and the complete voice reply content is no longer broadcast. For example, the voice request is "What is the weather tomorrow?", and the complete voice reply content corresponding to the voice request is "It will be light rain and overcast tomorrow, the highest temperature is 17 degrees, the lowest temperature is 10 degrees, the north wind is three to four levels, and the air quality is excellent". The final conclusion of the complete voice reply content is the first sentence "It will be light rain and overcast tomorrow", so the TTS reply is performed, that is, it is degraded to only broadcast the first sentence "It will be light rain and overcast tomorrow".

[0103] For example, the voice request of the main driver sound area user received is "open the window", the voice request of the rear sound area user is "close the seat heating", and the voice request of the co-driver sound area user is "What is the weather tomorrow?". Since each voice request belongs to a single round voice dialogue, and the first sentence of the reply content corresponding to each voice request is the final conclusion, the TTS reply finally performed for the voice requests of different sound area users is "The main driver's window is opened, the rear seat heating is closed. It will be sunny to cloudy tomorrow".

[0104] S305, judging whether the reply content of the voice request of the single round voice dialogue is a short text reply. If it is a short text reply, go to S306, and if it is not a short text reply, go to S307.

[0105] S306, performing a voice broadcast reply for a multi-round voice dialogue, and performing a toast display reply + sound effect prompt for a single round voice dialogue with a short text reply.

[0106] Because the multi-round voice dialogue scene needs to complete multi-round continuous dialogue, a multi-round voice broadcast reply is performed for the multi-round voice dialogue.

[0107] The single round voice dialogue scene is a single round that can end the task. Because it is a short text reply at this time, the toast method can be used to display the reply content on the screen of the vehicle terminal, such as the car machine screen, and sound effect prompt can also be added.

[0108] In this way, in different situations, for the voice request demand initiated by multiple users in different sound areas in the vehicle at the same time, the application can use different reply modes to feed back to the users in different sound areas at the same time, for example, a multi-round voice broadcast reply is performed for a multi-round voice dialogue, and a toast method is used for a single round voice dialogue and a short text reply, so that the feedback of any user is not lost through different feedback forms.

[0109] S307, performing voice broadcast reply for multi-turn voice dialogue, performing card display reply + sound effect prompt for single-turn voice dialogue of long text reply.

[0110] Because the multi-turn voice dialogue scene needs to complete multi-turn continuous dialogue, multi-turn voice broadcast reply is performed for multi-turn voice dialogue.

[0111] The single-turn voice dialogue scene is a single-turn task that can be completed, because it is not a short text reply at this time, that is, it belongs to a long text reply, so the reply content can be displayed on the screen of the vehicle terminal such as the car screen in the form of a card, and a sound effect prompt can also be added.

[0112] In this way, for voice request demands initiated by multiple users in different sound areas in the vehicle at the same time in different situations, the application can use different reply modes to feed back to users in different sound areas at the same time, for example, performing multi-turn voice broadcast reply for multi-turn voice dialogue, and using card mode reply for single-turn voice dialogue and long text reply, so that feedback loss of any user is avoided through different feedback forms.

[0113] S308, judging whether the voice broadcast reply corresponding to the voice request can be degraded to GUI display, if yes, entering S310, if not, entering S309.

[0114] S309, queuing to perform voice broadcast reply.

[0115] Among them, for voice requests of users in different sound areas, voice broadcast reply is queued to be performed.

[0116] S310, judging whether the reply content of the voice request of the main driver sound area user is a short text reply, if yes, entering S312, if not, entering S311.

[0117] S311, partially performing voice broadcast reply, and performing card display reply + sound effect prompt for the other part.

[0118] For voice requests of the main driver sound area user, card display reply + sound effect prompt is performed. For voice requests of the main driver sound area user, long text replies such as question and answer can use card mode to carry the reply content displayed on the screen of the vehicle terminal such as the car screen, and a sound effect prompt can also be added.

[0119] For voice requests of non-main driver sound area users, voice broadcast reply can be performed at the same time.

[0120] In this way, for the voice request demand of multiple users in different sound areas in the vehicle initiated at the same time in different situations, the application can simultaneously feed back to the users in different sound areas in different reply ways, for example, for the voice request of the user in the main driver sound area, the card display reply + sound effect prompt is executed, and for the voice request of the user in the non-main driver sound area, the voice broadcast reply is executed, so that the feedback loss of any user is prevented through different feedback forms.

[0121] S312, part of the voice broadcast reply is executed, and the other part of the toast display reply + sound effect prompt is executed.

[0122] For the voice request of the user in the main driver sound area, the toast display reply + sound effect prompt is executed. The vehicle control type and other short text replies can adopt the toast method to carry the reply content to display on the screen of the vehicle terminal, for example, the car screen, and in addition, the sound effect prompt can be added.

[0123] For the voice request of the user in the non-main driver sound area, the voice broadcast reply can be executed at the same time.

[0124] In this way, for the voice request demand of multiple users in different sound areas in the vehicle initiated at the same time in different situations, the application can simultaneously feed back to the users in different sound areas in different reply ways, for example, for the voice request of the user in the main driver sound area, the card display reply + sound effect prompt is executed, and for the voice request of the user in the non-main driver sound area, the voice broadcast reply is executed, so that the feedback loss of any user is prevented through different feedback forms.

[0125] The following is an example in combination with the above steps. The application can distinguish different types for reply degradation, as shown in the following table 2:

[0126]

[0127]

[0128] Table 2

[0129] Figure 4 is the schematic diagram of the judgment sub-process of the application Figure 3 for the toast display to execute the sound effect prompt.

[0130] The application can match the sound effect prompt with the toast method to assist information transmission. The sound effect prompt rules are shown in the following table 3:

[0131]

[0132] Table 3

[0133] Referring to Figure 4 , the method comprises:

[0134] S401, determine whether it is toast display reply, if it is toast display reply, enter S403, if it is not toast display reply, enter S402.

[0135] S402, no sound effect prompt.

[0136] S403, determine whether it is multiple toast display reply, if it is multiple toast display reply, enter S405, if it is not multiple toast display reply, enter S404.

[0137] S404, play a sound effect prompt corresponding to the execution attribute.

[0138] S405, determine whether the instruction execution attribute is consistent, if the instruction execution attribute is consistent, enter S407, if the instruction execution attribute is not consistent, enter S406.

[0139] S406, play a sound effect prompt of failed execution.

[0140] S407, play a sound effect prompt corresponding to the execution attribute.

[0141] In summary, in the related art, although multiple users and multiple sound zones can be recognized and voice interaction can be provided, due to the limitation of the audio output channel, generally only one user is served in a single round, that is, only one or more voice requests of one user are responded to by voice at a time. When multiple users in different sound zones in the vehicle simultaneously issue voice requests, generally only the subsequent voice request of the user who wakes up the voice assistant is heard, and the other users inserted in between cannot obtain voice feedback and need to use the wake-up word to seize the focus of the voice assistant to themselves. This processing method of competing for control with each other not only causes low system service efficiency, but also easily causes chaos in the interaction scene. The scheme provided in the present application can meet the voice request demand of multiple users in different sound zones in the vehicle simultaneously in different scenarios, can simultaneously feed back to users in different sound zones in different reply ways, and can ensure that the feedback of any user is not lost through different feedback forms. For example, when the user at the position of the co-driver sound zone requires the voice system to tell a joke, the user at the position of the driver sound zone requires the atmosphere lamp to be purple. In order to ensure that the requirements of all users in different sound zones are executed and good perceptual experience is obtained, using the scheme of the present application, the vehicle-mounted voice assistant will respond to the request of the user at the position of the co-driver sound zone through audio broadcast, at the same time, will execute the operation of setting the atmosphere lamp to purple issued by the user at the position of the driver sound zone, and can also emit a sound effect indicating successful execution through the car audio, or can display a prompt that the atmosphere lamp has been set to purple on the large screen of the vehicle-mounted terminal.

[0142] It should be noted that if the voice request of the previous instruction is received in the middle of the voice broadcast, the voice request of the other sound area can be processed according to different situations.

[0143] For example, when the remaining time of the broadcast reply of the previous voice request is less than 2 seconds, the new voice request can be broadcasted after the broadcast is completed:

[0144] Instruction A: The air volume is adjusted to the maximum

[0145] Instruction B: Who is Steve Jobs

[0146] Voice broadcast: The air volume of the air conditioner is adjusted. Steve Jobs was born in San Francisco, California, USA, and is an American inventor, entrepreneur and co-founder of Apple...

[0147] For example, when the remaining time of the broadcast reply of the previous voice request is greater than 2 seconds, if TTS reply can be used, the degraded voice content is broadcasted by TTS reply, and if TTS reply cannot be used, GUI display is used at this time:

[0148] Instruction A: What is the stock price of Apple

[0149] Instruction B: What is the English name of avocado

[0150] Voice broadcast: Apple stock is now $166.230 per share, down 0.20%, with a total market value of $271 billion. After trading, the English name of avocado is avocado.

[0151] For example, when the remaining time of the broadcast reply of the previous voice request is greater than 2 seconds, that is, the voice request cannot be replied by TTS and cannot be displayed by GUI, a waiting reminder is provided at this time:

[0152] Instruction A: What is the weather tomorrow

[0153] Instruction B: Tell me a joke

[0154] Voice broadcast: Tomorrow will be rainy and cloudy... The copilot will tell you a joke in a moment... The highest temperature is 17 degrees, and the lowest temperature is 10 degrees.

[0155] Corresponding to the foregoing application function implementation method, the application also provides a server.

[0156] Figure 7 It is a structural schematic diagram of the server shown in the application.

[0157] Referring to Figure 7 The server 70 provided by the application comprises a receiving module 71, a judging module 72 and a processing module 73.

[0158] The receiving module 71 is configured to receive the voice requests of the users in at least two different sound zones in the vehicle cabin forwarded by the vehicle. The receiving module 71 can receive the voice requests of the users in the main driver sound zone, the copilot sound zone, and the rear sound zone in the vehicle cabin forwarded by the vehicle.

[0159] The determining module 72 is configured to determine, according to a predetermined rule, whether each voice request received by the receiving module 71 belongs to a single-turn voice dialogue or a multi-turn voice dialogue. The determining module 72 can determine that each voice request belongs to a single-turn voice dialogue according to that the voice request can be answered in a single turn, and determine that each voice request belongs to a multi-turn voice dialogue according to that the voice request cannot be answered in a single turn.

[0160] The processing module 73 is configured to determine, according to the determination result of the determining module 72, at least two different feedback modes including display of a user interface control in a graphical user interface of the vehicle system, display of a text message in the graphical user interface of the vehicle system, and TTS reply from text, and send the feedback modes to the vehicle to respond to each voice request and complete voice interaction.

[0161] Figure 8 FIG. 4 is a structural schematic diagram of another server according to an embodiment of the present application.

[0162] Referring to FIG. 4, Figure 8 The server 70 provided by the present application includes a receiving module 71, a determining module 72, and a processing module 73. The processing module 73 includes a first processing submodule 731 and a second processing submodule 732.

[0163] The first processing submodule 731 is configured to, after determining that each voice request belongs to a single-turn voice dialogue, determine three different feedback modes including display of a user interface control in a graphical user interface of the vehicle system, display of a text message in the graphical user interface of the vehicle system, and TTS reply, and send the feedback modes to the vehicle to respond to each voice request and complete voice interaction.

[0164] After the determining module 72 determines that each voice request belongs to a single-turn voice dialogue, if the first processing submodule 731 determines that the first sentence of the voice reply to the voice request of the user in at least two different sound zones is a conclusion, and determines that the reply to each voice request is a feedback mode of TTS reply and is sequentially fed back.

[0165] If the judgment module 72 judges that each voice request belongs to a single-turn voice dialogue, if the first processing submodule 731 determines that the first sentence of the voice reply to the voice request of the user in at least two different sound areas is not a conclusion, and the reply content to the voice request of the user in the driver sound area belongs to a long text, it is determined that the reply to the voice request of the user in the driver sound area is the feedback mode of the user interface control display in the vehicle-mounted system graphical user interface, and the reply to the voice request of the user in the non-driver sound area is the feedback mode of voice broadcast.

[0166] If the judgment module 72 judges that each voice request belongs to a single-turn voice dialogue, if the first processing submodule 731 determines that the first sentence of the voice reply to the voice request of the user in at least two different sound areas is not a conclusion, and the reply content to the voice request of the user in the driver sound area belongs to a short text, it is determined that the reply to the voice request of the user in the driver sound area is the feedback mode of the text message display in the vehicle-mounted system graphical user interface, and the reply to the voice request of the user in the non-driver sound area is the feedback mode of voice broadcast.

[0167] The second processing submodule 732 is configured to, after judging that each voice request does not belong to a single-turn voice dialogue, determine two different feedback modes including the user interface control display in the vehicle-mounted system graphical user interface and the text message display in the vehicle-mounted system graphical user interface, and send the feedback modes to the vehicle to respond to each voice request and complete voice interaction.

[0168] After the judgment module 72 judges that each voice request does not belong to a single-turn voice dialogue, for the voice request belonging to a single-turn voice dialogue, if the second processing submodule 732 determines that the reply content of the voice request belongs to a long text, it is determined that the reply to the voice request is the feedback mode of the user interface control display in the vehicle-mounted system graphical user interface; for the voice request belonging to a multi-turn voice dialogue, the second processing submodule 732 determines that the reply to the voice request is the feedback mode of voice broadcast.

[0169] After the judgment module 72 judges that each voice request does not belong to a single-turn voice dialogue, for the voice request belonging to a single-turn voice dialogue, if the second processing submodule 732 determines that the reply content of the voice request belongs to a short text, it is determined that the reply to the voice request is the feedback mode of the text message display in the vehicle-mounted system graphical user interface; for the voice request belonging to a multi-turn voice dialogue, the second processing submodule 732 determines that the reply to the voice request is the feedback mode of voice broadcast.

[0170] The application scheme can avoid the problem that the instructions of other users need to wait for a long time for a reply caused by single round service, so that users in different sound areas can obtain timely feedback through different feedback modes, and can timely perceive the result feedback, and the user experience is improved. The application can flexibly set different feedback modes for different voice requests to timely feedback the users in different sound areas by distinguishing whether the voice request belongs to single round voice dialogue or multi-round voice dialogue.

[0171] As to the device in the above embodiments, the specific manner in which the various modules perform operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0172] Figure 9 is a structural schematic diagram of a server shown in the application.

[0173] Referring to Figure 9 The server 1000 includes a memory 1010 and a processor 1020.

[0174] The processor 1020 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0175] The memory 1010 can include various types of storage units such as a system memory, a read-only memory (ROM), and a permanent storage device. Among them, the ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device can be a rewritable storage device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device employs a mass storage device (e.g., a magnetic or optical disk, a flash memory) as a permanent storage device. In some other embodiments, the permanent storage device can be a removable storage device (e.g., a floppy disk, an optical drive). The system memory can be a readable and writable storage device or a volatile readable and writable storage device such as a dynamic random access memory. The system memory can store some or all of the instructions and data required by the processor during runtime. In addition, the memory 1010 can include a combination of any computer readable storage media, including various types of semiconductor storage chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory 1010 can include a readable and / or writable removable storage device such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., an SD card, a min SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer readable storage medium does not include a carrier wave and an instantaneous electronic signal transmitted by wireless or wired transmission.

[0176] The memory 1010 stores executable code, which, when processed by the processor 1020, can cause the processor 1020 to perform part or all of the above-mentioned methods.

[0177] In addition, the method according to the present application can also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing part or all of the steps of the above-mentioned methods of the present application.

[0178] Alternatively, the present application can also be implemented as a computer readable storage medium (or non-transitory machine readable storage medium or machine readable storage medium) having executable code (or computer program or computer instruction code) stored thereon, which, when executed by a processor of an electronic device (or a server, etc.), causes the processor to perform part or all of the steps of the above-mentioned methods according to the present application.

[0179] Having described various embodiments of the application, it is to be understood that the above description is meant not to limit and not to encompass all of the possible embodiments. Many modifications and variations of this application can be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. It is intended that the scope of the application be defined by the scope of the patent and by the claims as allowed by the patent office, which can include adaptations based on the description, equivalents, and / or substitutions of elements individually or collectively to the entire disclosure.

Claims

1. A voice interaction method, characterized in that, include: Receive voice requests forwarded by the vehicle from at least two users in different audio zones within the cabin. Determine whether each voice request belongs to a single-turn or multi-turn voice dialogue based on predetermined rules; Based on the judgment results, at least two different feedback methods are determined, including the display of user interface controls in the vehicle system's graphical user interface, the display of text messages in the vehicle system's graphical user interface, and text-to-speech (TTS) responses. These feedback methods are then sent to the vehicle to respond to each voice request and complete the voice interaction. The judgment result is that each voice request belongs to a single-turn voice dialogue. The feedback method includes, based on the fact that the first sentence of the reply content corresponding to each voice request is the final conclusion, performing the text-to-speech (TTS) reply for each voice request and degrading it to only playing the first sentence. When the determination result is that each voice request does not belong to a single-turn voice dialogue, the feedback method includes performing a voice broadcast reply for voice requests that belong to a multi-turn voice dialogue.

2. The method according to claim 1, characterized in that, The step of determining whether each voice request belongs to a single-turn or multi-turn voice dialogue according to predetermined rules includes: The fact that the answer can be completed in a single round based on the voice request indicates that it is a single-round voice dialogue. Since the voice request could not be answered in a single turn, it was determined to be a multi-turn voice dialogue.

3. The method according to claim 1, characterized in that, The step of determining at least two different feedback methods based on the judgment result, including displaying user interface controls in the vehicle system's graphical user interface, displaying text messages in the vehicle system's graphical user interface, and providing text-to-speech (TTS) responses, and then sending these feedback methods to the vehicle to respond to each voice request and complete the voice interaction, includes: After determining that each voice request belongs to a single-turn voice dialogue, three different feedback methods are identified, including the display of user interface controls in the vehicle system's graphical user interface, the display of text messages in the vehicle system's graphical user interface, and the TTS response. The feedback methods are then sent to the vehicle to respond to each voice request and complete the voice interaction.

4. The method according to claim 3, characterized in that, After determining that each voice request belongs to a single-turn voice dialogue, the system identifies three different feedback methods: display of user interface controls in the vehicle system's graphical user interface, display of text messages in the vehicle system's graphical user interface, and TTS (Text-to-Speech) responses. After determining that each voice request belongs to a single-turn voice dialogue If the first sentence of the voice response to the voice requests of users in at least two different voice regions is a conclusion, then the response to each voice request is determined to be a TTS (Text-to-Speech) response, and the feedback is sequential.

5. The method according to claim 3, characterized in that, After determining that each voice request belongs to a single-turn voice dialogue, the system identifies three different feedback methods: display of user interface controls in the vehicle system's graphical user interface, display of text messages in the vehicle system's graphical user interface, and TTS (Text-to-Speech) responses. After determining that each voice request belongs to a single-turn voice dialogue If the first sentence of the voice response to the voice request of at least two users in different voice zones is not always a conclusion, and the response to the voice request of the user in the driver's voice zone is a long text, then the response to the voice request of the user in the driver's voice zone is determined to be the feedback method displayed by the user interface control in the graphical user interface of the vehicle system, while the response to the voice request of the user in the non-driver's voice zone is the feedback method of voice broadcast.

6. The method according to claim 3, characterized in that, After determining that each voice request belongs to a single-turn voice dialogue, the system identifies three different feedback methods: display of user interface controls in the vehicle system's graphical user interface, display of text messages in the vehicle system's graphical user interface, and TTS (Text-to-Speech) responses. After determining that each voice request belongs to a single-turn voice dialogue If the first sentence of the voice response to the voice request of at least two users in different voice zones is not always a conclusion, and the response to the voice request of the user in the driver's voice zone is short text, then the response to the voice request of the user in the driver's voice zone is determined to be a text message display feedback method of the vehicle system's graphical user interface, while the response to the voice request of users in non-driver's voice zones is a voice broadcast feedback method.

7. The method according to claim 1, characterized in that, The step of determining at least two different feedback methods based on the judgment result, including displaying user interface controls in the vehicle system's graphical user interface, displaying text messages in the vehicle system's graphical user interface, and providing text-to-speech (TTS) responses, and then sending these feedback methods to the vehicle to respond to each voice request and complete the voice interaction, includes: After determining that each voice request does not belong to a single-turn voice dialogue, two different feedback methods are identified, including the display of user interface controls in the vehicle system's graphical user interface and the display of text messages in the vehicle system's graphical user interface. The feedback methods are then sent to the vehicle to respond to each voice request and complete the voice interaction.

8. The method according to claim 7, characterized in that, After determining that each voice request does not belong to a single-turn voice dialogue, the system identifies two different feedback methods: displaying user interface controls in the vehicle's graphical user interface and displaying text messages in the vehicle's graphical user interface. After determining that each of the voice requests does not belong to a single-turn voice dialogue... For voice requests that are part of a single-turn voice dialogue, if the response to the voice request is a long text, then the response to the voice request is determined to be the feedback method displayed by the user interface controls in the vehicle system's graphical user interface. For voice requests that are part of a multi-turn voice dialogue, the response to the voice request will be a voice broadcast.

9. The method according to claim 7, characterized in that, After determining that each voice request does not belong to a single-turn voice dialogue, the system identifies two different feedback methods: displaying user interface controls in the vehicle's graphical user interface and displaying text messages in the vehicle's graphical user interface. After determining that each of the voice requests does not belong to a single-turn voice dialogue... For voice requests that are part of a single-turn voice dialogue, if the response to the voice request is a short text, then the response to the voice request is determined to be a text message display method of the in-vehicle system's graphical user interface. For voice requests that are part of a multi-turn voice dialogue, the response to the voice request will be a voice broadcast.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: In the in-vehicle system graphical user interface, a combination of sound effect prompts is used to provide two different feedback methods: displaying user interface controls and displaying text messages.

11. A server, characterized in that, include: The receiving module is used to receive voice requests from at least two users in different audio zones within the cabin, which are relayed by the vehicle. The judgment module is used to determine whether each voice request received by the receiving module belongs to a single-turn voice dialogue or a multi-turn voice dialogue according to a predetermined rule. The processing module is used to determine at least two different feedback methods based on the judgment result of the judgment module, including the display of user interface controls in the vehicle system's graphical user interface, the display of text messages in the vehicle system's graphical user interface, and a text-to-speech (TTS) response, and to send the feedback methods to the vehicle to respond to each voice request and complete the voice interaction. The judgment result is that each voice request belongs to a single-turn voice dialogue. The feedback method includes, based on the fact that the first sentence of the reply content corresponding to each voice request is the final conclusion, performing the text-to-speech (TTS) reply for each voice request and degrading it to only playing the first sentence. When the determination result is that each voice request does not belong to a single-turn voice dialogue, the feedback method includes performing a voice broadcast reply for voice requests that belong to a multi-turn voice dialogue.

12. A server, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-10.

13. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1-10.

Images