A method, apparatus and device for switching voice interaction dialogue style

Abstract

The application provides a method, device and equipment for switching voice interaction dialogue style, which comprises the following steps: receiving voice information sent by a user equipment and identifying the voice information to obtain text information corresponding to the voice information; determining a user intention and a corresponding service type according to the text information; obtaining corresponding to-be-played content from a corresponding content playing library according to the service type; determining structured data corresponding to the to-be-played content; determining a preset stylized template corresponding to the user intention under a dialogue style selected by the user, and replacing parameters in the structured data with content corresponding to the preset stylized template to obtain to-be-played text; performing voice synthesis on the to-be-played text and playing the to-be-played text through a loudspeaker of the user equipment. Through the method, feedback of dynamic content stylization is realized, and more humanized to-be-played text is generated.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a method, apparatus, and device for switching speech interaction dialogue styles. Background Technology

[0002] In existing voice interaction systems, responses to user interactions often use a fixed tone and intonation, resulting in a somewhat rigid and unnatural delivery. One solution addresses this issue by using a template that matches the user's chosen style. This approach is suitable for scenarios with relatively fixed user interaction content but cannot semantically understand changing content such as "weather" or "news," thus failing to provide stylized feedback for such content. Another solution uses keyword replacement, replacing certain words in the broadcast content with a pre-defined regional vocabulary. This allows for some application of dynamic content, as different regions may use different sentence structures for different intentions. However, word replacement does not alter the sentence structure, and indiscriminate word replacement can damage the language's sound and semantic information. Summary of the Invention

[0003] To address the problems of existing voice interaction systems being unable to achieve semantic understanding of changing content and having incoherent sentences, this application provides a method, apparatus, and device for switching voice interaction dialogue styles.

[0004] Firstly, this application provides a method for switching voice interaction dialogue styles, applied on a server side, the method comprising:

[0005] Receive and recognize voice information sent by a user equipment to obtain text information corresponding to the voice information;

[0006] The user intent is determined based on the text information, and the corresponding service type is determined based on the user intent.

[0007] According to the service type, retrieve the content to be played from the corresponding content playback library that corresponds to the user's intent;

[0008] Determine the structured data corresponding to the content to be broadcast, wherein the structured data consists of fields and parameters corresponding to key information in the broadcast content;

[0009] Determine the preset stylized template corresponding to the user's intent under the user's selected dialogue style, and replace the corresponding content of the preset stylized template with the parameters in the structured data to obtain the text to be broadcast;

[0010] The text to be broadcast is synthesized into speech and then broadcast through the speaker of the user's device.

[0011] In one possible implementation, determining the structured data corresponding to the content to be broadcast includes:

[0012] When it is determined that the type of the content to be broadcast corresponding to the user's intent is dynamic structured data, the structured data corresponding to the content to be broadcast is obtained.

[0013] When it is determined that the type of the content to be broadcast corresponding to the user intent is dynamic unstructured data, a semantic understanding model is used to extract key information related to the user intent from the dynamic unstructured data, and the fields and parameters corresponding to the key information are determined to obtain the structured data corresponding to the content to be broadcast.

[0014] In one possible implementation, when it is determined that the type of the content to be broadcast corresponding to the user intent is static data, a static preset stylized template corresponding to the user intent under the style selected by the user is obtained, and the content of the static preset stylized template is used as the text to be broadcast.

[0015] In one possible implementation, the semantic understanding model is trained in the following manner:

[0016] Obtain training samples, which include dynamic unstructured data and corresponding user intents, as well as fields and parameters of key information in dynamic structured data;

[0017] The dynamic unstructured data in the training samples and the corresponding user intent are input into the semantic understanding model, and the model is trained with the goal of outputting the fields and parameters of the corresponding key information in the training samples.

[0018] In one possible implementation, a preset stylized template corresponding to the user's intent under the user's selected dialogue style is determined, and the parameters in the structured data are used to replace the corresponding content of the preset stylized template, including:

[0019] Determine the dynamic preset stylized template corresponding to the user intent under the user's selected dialogue style, and the fields in the dynamic preset stylized template;

[0020] Identify the target field in the dynamically preset stylized template that matches the structured data, and replace the parameter corresponding to the target field with the parameter corresponding to the structured data.

[0021] In one possible implementation, retrieving the content to be played corresponding to the user's intent from the corresponding content playback library based on the service type includes:

[0022] When the service type is determined to be a local preset service type, the content to be played corresponding to the user intent is obtained from the local preset content playback library;

[0023] When the service type is determined to be a non-local service type, the content to be played corresponding to the user's intent is obtained from the content playback library of other servers;

[0024] In this process, the service type is matched with the content playback library in advance.

[0025] In one possible implementation, the parameters in the structured data are replaced with the corresponding content of the preset stylized template to obtain the text to be broadcast, including:

[0026] When it is determined that the semantic expression style of the parameters in the structured data is consistent with the dialogue style selected by the user, the parameters in the structured data are used to replace the corresponding content of the preset stylized template to obtain the text to be broadcast.

[0027] When it is determined that the semantic expression style of the parameters in the structured data is inconsistent with the dialogue style selected by the user, the parameters are converted into parameters with the same semantics and consistent with the dialogue style selected by the user, and then the corresponding content of the preset stylized template is replaced to obtain the text to be broadcast.

[0028] Secondly, this application provides a device for switching voice interaction dialogue styles, the device comprising:

[0029] The recognition module is used to receive and recognize voice information sent by the user equipment to obtain the text information corresponding to the voice information;

[0030] The service type determination module is used to determine the user intent based on the text information and determine the corresponding service type based on the user intent.

[0031] The acquisition module is used to acquire the content to be played from the corresponding content playback library according to the service type, which corresponds to the user's intent;

[0032] The structured data determination module is used to determine the structured data corresponding to the content to be broadcast, wherein the structured data is the fields and parameters corresponding to the key information in the broadcast content;

[0033] The replacement module is used to determine the preset stylized template corresponding to the user's intent under the user's selected dialogue style, and replace the corresponding content of the preset stylized template with the parameters in the structured data to obtain the text to be broadcast;

[0034] The synthesis module is used to synthesize the text to be broadcast into speech and broadcast it through the speaker of the user equipment.

[0035] Thirdly, this application provides a device for changing the style of voice interaction dialogue, the device comprising:

[0036] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the first aspect above.

[0037] Fourthly, this application provides a computer storage medium storing a computer program for causing a computer to perform the method described in the first aspect above.

[0038] This application provides a method, apparatus, and device for switching voice interaction dialogue styles. By processing different types of content to be broadcast according to a preset stylized template, it achieves stylized processing of fixed content, structured data, and unstructured data. It can stylize changing content such as "weather" and "news" without causing semantic damage. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of a system structure for switching voice interaction dialogue styles, as exemplified by an exemplary embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of another system structure for switching voice interaction dialogue styles, as exemplified by an exemplary embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram of another system structure for switching voice interaction dialogue styles, as exemplified by an exemplary embodiment of the present invention.

[0042] Figure 4 This is a flowchart illustrating a method for switching voice interaction dialogue styles according to an exemplary embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram illustrating a replacement process according to an exemplary embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram illustrating the process of determining the text to be broadcast, as exemplified by an exemplary embodiment of the present invention.

[0045] Figure 7 This is a schematic diagram illustrating a dynamic unstructured data processing flow according to an exemplary embodiment of the present invention;

[0046] Figure 8 This is a schematic diagram illustrating the training process of a semantic understanding model according to an exemplary embodiment of the present invention;

[0047] Figure 9 This is a schematic diagram illustrating a method for switching voice interaction dialogue styles according to an exemplary embodiment of the present invention.

[0048] Figure 10 This is a simplified flowchart illustrating a system for switching voice interaction dialogue styles, as exemplified by an exemplary embodiment of the present invention.

[0049] Figure 11 A simplified flowchart illustrating another system for switching voice interaction dialogue styles, as exemplified by an exemplary embodiment of the present invention.

[0050] Figure 12 A schematic diagram illustrating a specific process of another method for switching voice interaction dialogue styles according to an exemplary embodiment of the present invention;

[0051] Figure 13 A schematic diagram illustrating a specific process of another method for switching voice interaction dialogue styles according to an exemplary embodiment of the present invention;

[0052] Figure 14 A simplified flowchart illustrating another system for switching voice interaction dialogue styles, as exemplified by an exemplary embodiment of the present invention.

[0053] Figure 15 A schematic diagram illustrating a specific process of another method for switching voice interaction dialogue styles according to an exemplary embodiment of the present invention;

[0054] Figure 16 A schematic diagram illustrating a specific process of another method for switching voice interaction dialogue styles according to an exemplary embodiment of the present invention;

[0055] Figure 17 A schematic diagram of a device for switching voice interaction dialogue styles, as exemplified by an exemplary embodiment of the present invention;

[0056] Figure 18 This is a schematic diagram of a device for switching voice interaction dialogue styles, as exemplified by an exemplary embodiment of the present invention.

[0057] Figure 19 A schematic diagram of a user equipment according to an exemplary embodiment of the present invention;

[0058] Figure 20 This is a schematic diagram of a storage medium as an example of an exemplary embodiment of the present invention. Detailed Implementation

[0059] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The acquisition, storage, use, and processing of data in the embodiments of this application all comply with the relevant provisions of national laws and regulations.

[0060] like Figure 1 The diagram shown illustrates a system architecture for switching voice interaction dialogue styles according to this application. The system includes a user device 101, a language capability platform 102, a functional business platform 103, and a third-party skill service platform 104. The user device 101 sends user-inputted voice information to the language capability platform 102. The user device 101 includes, but is not limited to, devices such as smart speakers, televisions, refrigerators, washing machines, air conditioners, tablets, and mobile phones. The language platform 102 performs speech recognition and speech synthesis operations and sends the speech recognition results to the functional business platform 103. The functional business platform 103 understands the user's intent and generates text to be played. The third-party skill service platform 104 provides content such as news, weather, classical poems, and financial information. The language capability platform 102 and the functional business platform 103 are located on the first server, and the third-party skill service platform 104 is located on the second server, or the language capability platform 102, the functional business platform 103, and the third-party skill service platform 104 are located on the same server.

[0061] Apart from Figure 1 In addition to the system structure shown for switching voice interaction dialogue styles, the structure can also be modified according to the functions of the user device, such as... Figure 2 As shown, for some user devices, there is no need to perform too many business operations. For example, when a user instructs the air conditioner to adjust the temperature to a specific temperature, the air conditioner may remind the user "not to set the temperature too low to avoid catching a cold" based on the current outdoor temperature. In this case, it is only necessary to migrate the weather database of the third-party skill service platform to the language ability platform. There is no need to add other third-party skill service platforms to the system.

[0062] like Figure 3 The image shows another system architecture for switching voice interaction dialogue styles, and... Figure 1 Unlike other systems, this system architecture places the function of generating the text to be broadcast on the user device. This structure requires the user device to have a single function, and the user's intentions do not require a large amount of algorithmic resources to understand.

[0063] Existing voice interaction systems are widely used in both B2B (business-to-business) and B2C (business-to-consumer) sectors. In the B2C sector, home appliances such as mobile phones, air conditioners, washing machines, refrigerators, and televisions, and even mobile applications, have developed their own voice assistants to enhance service levels and help users better utilize the apps through voice commands. In the B2B sector, voice is gradually becoming an assistant to help business users with their work, such as intelligent customer service, voice systems in intelligent diagnostic assistance, and intelligent voice assistants in smart cities.

[0064] Existing voice systems have undergone extensive research and engineering practice in correctly recognizing dialects from different regions and understanding different ways of expressing the same intent. However, current methods for generating user-friendly response corpora with varying styles rely on fixed pre-set sentence structures or directly replacing certain words in the content to be broadcast. This results in inconsistent content style and stiff, unnatural speech. This application provides a method for switching voice interaction dialogue styles, applied to the server side, such as... Figure 4 As shown, the method includes:

[0065] S401: Receive and recognize voice information sent by the user equipment to obtain text information corresponding to the voice information.

[0066] When users interact with their devices via voice, they can activate the device's microphone function by speaking or by pressing a button on the device.

[0067] The method for switching voice interaction dialogue styles provided in this application includes two parts on the server side: a voice capability platform and a functional business platform. Depending on the user's different needs for the voice interaction system, the server side may also include three parts: a voice capability platform, a functional business platform, and a third-party skill service platform. For example, third-party skill services may also be available on some parts of the language capability platform, and the function of generating a script to be played may also be deployed on the user's device.

[0068] The user inputs voice information into the user device, which picks up the user's audio stream through a microphone, microphone, or microphone array. The user device then sends the audio stream to the server's language capability platform for speech recognition, obtaining text information corresponding to the voice information. The content of the text information is the same as the content of the voice information input by the user.

[0069] S402: Determine the user intent based on the text information, and determine the corresponding service type based on the user intent.

[0070] Text information can be fed back to the user device and displayed on the user device's display interface, allowing the user to judge whether the speech recognition result is accurate. If it is inaccurate, the user can re-enter the speech information, and the process of determining the user's intent will terminate and speech pickup will be performed again. In this way, the user can judge whether the user's intent is correct in a timely manner, which improves the accuracy of the feedback text to be played.

[0071] Specifically, parsing the text information and determining the user intent corresponding to the text information can be achieved by inputting the text information into a corresponding semantic understanding model for processing to obtain the corresponding user intent. The semantic understanding model is trained by taking the text information in the sample as input and the user intent of the output sample as the target. Other semantic understanding algorithms can also be used, which are not specifically limited here.

[0072] S403: Obtain the content to be played that corresponds to the user's intent from the corresponding content playback library according to the service type.

[0073] In one possible implementation, retrieving the content to be played corresponding to the user's intent from the corresponding content playback library based on the service type includes:

[0074] When the service type is determined to be a local preset service type, the content to be played corresponding to the user intent is obtained from the local preset content playback library;

[0075] When the service type is determined to be a non-local service type, the content to be played corresponding to the user's intent is obtained from the content playback library of other servers;

[0076] In this process, the service type is matched with the content playback library in advance.

[0077] After the functional business platform recognizes the voice information and obtains the corresponding text information, it determines the user's intent and the voice backend service decision. The business process within the system will differ depending on the voice backend service decision. If the voice backend service decision itself allows the functional business platform to meet the user's needs—for example, if the user needs to control the air conditioner temperature—the voice backend service generates the air conditioner control command, calls the text to be played service to generate the text to be played, and sends it back to the user's device. If the voice backend service decision requires calling other servers to play content from the database, then data is requested from other third-party skill service platforms.

[0078] S404: Determine the structured data corresponding to the content to be broadcast, wherein the structured data is the fields and parameters corresponding to the key information in the broadcast content.

[0079] The types of content to be broadcast include static data, dynamic structured data, and dynamic unstructured data.

[0080] Structured data, also known as row data, is logically expressed and implemented using a two-dimensional table structure, strictly adhering to data format and length specifications. Unstructured data, on the other hand, is data with irregular or incomplete structures, lacking a predefined data model, and is inconvenient to represent using a two-dimensional logical table in a database. To facilitate stylization of unstructured data, it is first necessary to convert it into structured data before proceeding with further processing.

[0081] The structured and unstructured data proposed in this solution are both dynamic, meaning they change in real time.

[0082] S405: Determine the preset stylized template corresponding to the user's intent under the dialogue style selected by the user, and replace the corresponding content of the preset stylized template with the parameters in the structured data to obtain the text to be broadcast.

[0083] Before inputting voice information into their device, users can pre-select a dialogue style. Specifically, for example, on mobile phones and tablets, this can be selected via the user interface; for devices like air conditioners, it can be selected via a remote control. Besides selecting a style before inputting voice information, users can also select a dialogue style during voice interaction. With the first method, all subsequent voice interactions adhere to that dialogue style until a different style is selected; with the second method, the dialogue style can be changed in real time.

[0084] The text to be broadcast provided in this application includes, but is not limited to, different sentence structures with the same semantic meaning, different regional dialect representations, different representations under different themes (competitive, calm, cute, science fiction), and different representations under different literary themes (classical Chinese, Analects style, Journey to the West style, Water Margin style).

[0085] Furthermore, the stylized templates mentioned above are pre-installed in the functional business platform (if the function of generating the text to be broadcast is deployed on the user device, then the stylized templates are pre-installed on the user device). Different user intentions under different dialogue styles may correspond to multiple stylized templates, and they can be dynamically updated to give users a feeling that they are not all the same, thus improving the user experience. At this time, one of the multiple stylized templates can be randomly selected, and the selection rule can be in a round-robin manner, which is not specifically limited here.

[0086] S406: The text to be broadcast is synthesized into speech and broadcast through the speaker of the user equipment.

[0087] When speech synthesis is performed, the text to be spoken is synthesized using the pre-selected voice tone according to the dialogue style chosen by the user, and then spoken through the user's device speaker. The speech synthesis function is deployed on a speech capability platform.

[0088] This application provides a method for switching voice interaction dialogue styles. By acquiring structured data of the content to be broadcast and replacing the parameters in the structured data according to a preset stylized template, dynamic content stylization feedback is achieved, while generating more humanized text to be broadcast.

[0089] When the functional service platform receives content to be broadcast, it adopts different processing schemes based on the type of content. The types of content to be broadcast include the following three categories:

[0090] (1) Static data.

[0091] In one possible implementation, when it is determined that the type of the content to be broadcast corresponding to the user intent is static data, a static preset stylized template corresponding to the user intent under the style selected by the user is obtained, and the content of the static preset stylized template is used as the text to be broadcast.

[0092] This approach may obtain multiple pre-set stylized templates corresponding to the user's intent under the user's selected dialogue style. In this case, the content of one of the stylized templates can be randomly selected as the text to be read. For example, if a user needs to control the air conditioner temperature, and the user selects an English reading style with the text to be read being "OK," the stylized templates corresponding to the "English reading" style under this user intent include "OK," "All Right," and "Well." Users can choose any one of these templates, or they can cycle through them, i.e., first selecting "OK" as the text to be read, then selecting "All Right," and so on. There are no specific limitations here. The multiple pre-set stylized templates provide users with a more varied and engaging experience, offering unpredictable feedback.

[0093] (2) Dynamic structured data.

[0094] In one possible implementation, when it is determined that the type of the content to be broadcast corresponding to the user's intent is dynamic structured data, the structured data corresponding to the content to be broadcast is obtained.

[0095] Determine the preset stylized template corresponding to the user's intent under the user's selected dialogue style, and replace the corresponding content of the preset stylized template with the parameters in the structured data to obtain the text to be broadcast;

[0096] The text to be broadcast is synthesized into speech and then broadcast through the speaker of the user's device.

[0097] The replacement process is as follows: Figure 5 As shown:

[0098] S501: Determine the dynamic preset stylized template corresponding to the user intent under the dialogue style selected by the user, and the fields in the dynamic preset stylized template.

[0099] For dynamic structured data, dynamic preset stylized templates differ from static preset stylized templates. Static preset stylized templates contain fixed content, while preset stylized templates for dynamic structured data include both fixed and replaceable content.

[0100] S502: Determine the target field in the dynamic preset stylized template that matches the structured data, and replace the parameter corresponding to the target field with the parameter corresponding to the structured data.

[0101] Taking a user's travel intention as an example, the corresponding dynamic structured data is shown in Table 1:

[0102] Table 1

[0103] mode of transportation airplane Train number, class number MU2357 Time-Date 20200325 Time - Hour 0720 seat 8A

[0104] Using a pre-set stylized template for this type of intent in the functional business platform, the message reads: "You've booked {travel mode}. The flight number is {train number, flight number}, and the departure time is {time-date}. Please wake up early, as the departure time is {time-hour}. Your seat number is {seat}, don't forget! If you need to set an alarm, you can tell me to set an alarm for {{time-hour}} in the morning." Then, replace the parameters in Table 1 with the corresponding curly braces "{}".

[0105] In one possible implementation, the parameters in the structured data are replaced with the corresponding content of the preset stylized template to obtain the text to be broadcast, such as... Figure 6 As shown, this includes the following two cases:

[0106] 601: When it is determined that the semantic expression style of the parameters in the structured data is consistent with the dialogue style selected by the user, the parameters in the structured data are used to replace the corresponding content of the preset stylized template to obtain the text to be broadcast;

[0107] 602: When it is determined that the semantic expression style of the parameters in the structured data is inconsistent with the dialogue style selected by the user, the parameters are converted into parameters with the same semantics and consistent with the dialogue style selected by the user, and then the corresponding content of the preset stylized template is replaced to obtain the text to be broadcast.

[0108] Among the dialog styles selected by the user for the key information after semantic understanding, for dialog styles with dialect characteristics, some parameters are expressed in Mandarin, but when there are inconsistent semantic expressions in the dialect, it is necessary to perform synonym replacement on the parameters. For example, "tortoise" is expressed as "wu ju" in Shanghainese, and "holding an umbrella" is expressed as "dam ze" in Cantonese.

[0109] Different from simple synonym replacement, for some dialog styles, it is necessary to convert specific time points into relatively vague semantic expressions, for example, converting 7:20 into "morning", and in some scenarios, when the alarm clock is scheduled at 0:00, the specific time is converted into the corresponding Shi Chen (two-hour period) in classical Chinese style: Zi Shi, or unit of measurement conversion, for example, the conversion between kilometer and li (Chinese mile) for distance in classical Chinese style, converting 1 kilometer into 2 li.

[0110] (3) Dynamic unstructured data.

[0111] The process of stylizing unstructured data can be as follows, such as Figure 7 shown in the figure:

[0112] S701: A third-party skill service platform or a functional service platform generates dynamic unstructured content to be broadcast.

[0113] S702: Extract key information using a semantic understanding model.

[0114] In a possible implementation, when it is determined that the type of the acquired content to be broadcast corresponding to the user intent is dynamic unstructured data, a semantic understanding model is used to extract key information associated with the user intent from the dynamic unstructured data, and determine the fields and parameters corresponding to the key information, so as to obtain structured data corresponding to the content to be broadcast.

[0115] The semantic understanding model is obtained by training in the following manner, such as Figure 8 shown in the figure, including:

[0116] S801: Acquire training samples, where the training samples include dynamic unstructured data, corresponding user intents, and fields and parameters of key information in the dynamic unstructured data.

[0117] S802: Input the dynamic unstructured data and the corresponding user intent in the training samples into the semantic understanding model, and perform training with the goal of outputting the fields and parameters of the corresponding key information in the training samples.

[0118] The text information received by the semantic understanding model herein is the content to be broadcast generated by the functional service platform when meeting user needs or requesting from a third-party skill service platform to meet user needs.

[0119] In addition to training a semantic understanding model using the methods described above, different semantic understanding models can be trained for different user intents. For example, for a user intent to query the weather, a weather semantic understanding model is trained, meaning the training samples include fields and parameters of key information from dynamic unstructured data and dynamic structured data corresponding to the user intent to query the weather; for a user intent to travel, a travel semantic understanding model is trained, meaning the training samples include fields and parameters of key information from dynamic unstructured data and dynamic structured data corresponding to the user intent to travel.

[0120] In the entire voice interaction system, the functional business platform and the third-party skill service platform have a limited number of functions and responses that can meet user needs. Furthermore, when understanding user intent, we already know which functions these platforms are currently invoking to satisfy the user's needs. Compared to semantic understanding of the user's voice input flow, the type of the text to be processed is already clear, requiring named entity extraction under specific service types. For example, if the system ultimately determines that the user's intent is to query the weather, and the weather report text retrieved by the third-party skill service platform is known to be weather report text before invoking the semantic understanding model, then during information extraction, we focus on extracting key information such as temperature, wind speed, air quality, PM2.5, and weather conditions from the weather expression.

[0121] For example, if the feedback to be broadcast is "Today's weather is sunny, temperature 15-25 degrees Celsius, light breeze, good air quality, PM2.5 is 70", for this type of cloud feedback, the key information in the text is first extracted using a semantic understanding model to form structured data, as shown in Table 2:

[0122] Table 2

[0123]

[0124]

[0125] After the structured data is formed, the same scheme as scheme (2) is adopted. For the pre-stylized template of the voice interaction system service content, the content corresponding to the pre-stylized template in Table 2 is replaced.

[0126] S703: Obtain the preset stylized template corresponding to the user intent under the user-selected dialogue style.

[0127] First, a template suitable for the feedback corpus service type needs to be randomly extracted from the user-selected pre-defined stylized template library. Multiple templates are pre-defined for each dialogue style, and these can be dynamically updated to give users a varied and engaging experience.

[0128] S704: Parameter replacement processing.

[0129] The replacement process is the same as that for the dynamic structured data in (2), and will not be repeated here.

[0130] S705: Feedback on the text to be broadcast that has been processed.

[0131] like Figure 9 The following is a detailed process for switching voice interaction dialogue styles:

[0132] 1. The user inputs voice information into the user device;

[0133] 2. The voice information is sent to the voice capability platform, which performs voice recognition and converts the voice information into text information in real time or non-real time.

[0134] 3. The voice capability platform feeds back the text information from the speech recognition to the user's device;

[0135] 4. The user equipment sends the text information to the functional service platform;

[0136] 5. The functional business platform performs semantic understanding of text information, determines the corresponding service type, and sends a request to a third-party skill service platform based on the service type. For example, it requests weather or news media content. This type of real-time dynamic information is provided by a dedicated weather website or news website.

[0137] 6. The third-party skills service platform responds to the request, obtains the media assets waiting to be broadcast that meet the user's needs, and feeds back to the functional business platform;

[0138] 7. After receiving the content to be broadcast, the functional business platform will use different stylization processing schemes according to the type of the content to be broadcast, generate the corresponding text to be broadcast, and send it back to the user's device.

[0139] 8. After receiving the text to be played from the functional service platform, the user equipment sends a speech synthesis request to the speech capability platform.

[0140] 9. The speech synthesis platform then sends the synthesized speech to the speech device.

[0141] 10. The voice interaction device will play the received audio message through a speaker;

[0142] 11. Optionally, the functional business platform can specify the tone of voice for text broadcasting in this style. When the user interaction device initiates a speech synthesis request, it requests the tone of voice in the corresponding style for speech synthesis.

[0143] The language capability platform and functional business platform are located on the first server, while the third-party skills service platform is located on the second server. Figure 10As shown, the first server requests data from the second server, the second server sends the content to be broadcast back to the first server, the first server processes the content to be broadcast to generate the text to be broadcast, and then sends it back to the user device, which then broadcasts it to the user through the speaker.

[0144] If the user's device has limited functionality, such as an air conditioner, the data in the content playback library of the second server can be migrated to the content playback library of the first server. Figure 11 As shown, all user needs can be provided by the first server. The specific interaction flow is as follows: Figure 12 As shown:

[0145] 1. The user inputs voice information into the user device;

[0146] 2. The voice information is sent to the voice capability platform, which performs voice recognition and converts the voice information into text information in real time or non-real time.

[0147] 3. The voice capability platform feeds back the text information from the speech recognition to the user's device;

[0148] 4. The user equipment sends the text information to the functional service platform;

[0149] 5. The functional business platform performs semantic understanding on the text information, determines the corresponding service type, and requests data from the language capability platform based on the service type;

[0150] 6. The language capability platform responds to requests, obtains media assets that meet user needs and await broadcast content, and feeds it back to the functional business platform;

[0151] 7. After receiving the content to be broadcast, the functional business platform will use different stylization processing schemes according to the type of the content to be broadcast, generate the corresponding text to be broadcast, and send it back to the user's device.

[0152] 8. After receiving the text to be played from the functional service platform, the user equipment sends a speech synthesis request to the speech capability platform.

[0153] 9. The speech synthesis platform then sends the synthesized speech to the speech device.

[0154] 10. The voice interaction device will play the received audio through the speaker.

[0155] 11. Optionally, the functional business platform can specify the tone of voice for text broadcasting in this style. When the user interaction device initiates a speech synthesis request, it requests the tone of voice in the corresponding style for speech synthesis.

[0156] If the text generation module does not require significant algorithmic resources for semantic understanding, and the stylized template library does not need frequent changes, the function of generating the text to be broadcast can be pre-installed in the user's device, including the following two forms:

[0157] 1) Migrate the data from the second server's content playback library to the first server's content playback library. The specific process is as follows: Figure 13 As shown:

[0158] Step 11 involves the user device sending a speech synthesis request to the language capability platform; the remaining steps are the same. Figure 12 The process is the same, so I will not repeat it here.

[0159] 2) Includes the first server and the second server.

[0160] The language proficiency platform and functional business platform are deployed on the first server, while the third-party skills service platform is deployed on the second server, such as... Figure 14 As shown, the first server requests data from the second server, the second server sends the content to be broadcast back to the first server, the first server sends the content to be broadcast back to the user device, the user device performs stylization processing and then sends it back to the first server for speech synthesis, and the synthesized speech is broadcast to the user through the user device's speaker. The specific process is as follows: Figure 15 As shown:

[0161] Step 11 involves the user device sending a speech synthesis request to the language capability platform; the remaining steps are the same. Figure 9 The process is the same, so I will not repeat it here.

[0162] Another solution is to integrate the voice capability platform with the functional business platform, such as... Figure 16 As shown, this solution has a relatively simple system architecture, relying on close cooperation between the voice capability platform and the functional business platform. The voice capability platform can customize requests. The specific steps are as follows:

[0163] 1. The user inputs voice information into the user device;

[0164] 2. The voice information is sent to the voice capability platform, which performs voice recognition and converts the voice information into text information in real time or non-real time.

[0165] 3. The voice capability platform sends the text information from the voice recognition to the functional business platform;

[0166] 4. The functional business platform performs semantic understanding of text information, determines the corresponding service type, and sends a request to the third-party skill service platform based on the service type. For example, it requests weather or news media content to be broadcast. This type of real-time dynamic information is provided by a dedicated weather website or news website.

[0167] 5. The third-party skills service platform responds to the request, obtains media asset content that meets the user's needs, and feeds it back to the functional business platform;

[0168] 6. After receiving the content to be broadcast, the functional business platform will generate the corresponding text to be broadcast based on the type of the content and use different technical solutions, and then feed it back to the language capability platform.

[0169] 7. The speech synthesis platform then sends the synthesized speech to the speech device.

[0170] 8. The voice interaction device will play the received audio message through the speaker;

[0171] 9. Optionally, the functional business platform can specify the tone of voice for text broadcasting in this style. When the user interaction device initiates a speech synthesis request, it requests the tone of voice in the corresponding style for speech synthesis.

[0172] Based on the same inventive concept, this application also provides a device 1700 for switching voice interaction dialogue styles, applied to a server, such as... Figure 17 As shown, the device includes:

[0173] The recognition module 1701 is used to receive and recognize voice information sent by the user equipment to obtain text information corresponding to the voice information;

[0174] The service type determination module 1702 is used to determine the user intent based on the text information and determine the corresponding service type based on the user intent.

[0175] The acquisition module 1703 is used to acquire the content to be played from the corresponding content playback library according to the service type, which corresponds to the user's intent;

[0176] The structured data module 1704 is used to determine the structured data corresponding to the content to be broadcast, wherein the structured data is the fields and parameters corresponding to the key information in the broadcast content;

[0177] Replacement module 1705 is used to determine the preset stylized template corresponding to the user's intent under the dialogue style selected by the user, and replace the corresponding content of the preset stylized template with the parameters in the structured data to obtain the text to be broadcast;

[0178] The synthesis module 1706 is used to synthesize the text to be broadcast into speech and broadcast it through the speaker of the user equipment.

[0179] In one possible implementation, the structured data determination module 1704 is used to determine the structured data corresponding to the content to be broadcast, including:

[0180] When it is determined that the type of the content to be broadcast corresponding to the user's intent is dynamic structured data, the structured data corresponding to the content to be broadcast is obtained.

[0181] When it is determined that the type of the content to be broadcast corresponding to the user intent is dynamic unstructured data, a semantic understanding model is used to extract key information related to the user intent from the dynamic unstructured data, and the fields and parameters corresponding to the key information are determined to obtain the structured data corresponding to the content to be broadcast.

[0182] In one possible implementation, when the structured data module 1704 determines that the type of the content to be broadcast corresponding to the user intent is static data, it acquires a static preset stylized template corresponding to the user intent under the style selected by the user, and uses the content of the static preset stylized template as the text to be broadcast.

[0183] In one possible implementation, the model training module is used to train the semantic understanding model in the following manner:

[0184] Obtain training samples, which include dynamic unstructured data and corresponding user intents, as well as fields and parameters of key information in dynamic structured data;

[0185] The dynamic unstructured data in the training samples and the corresponding user intent are input into the semantic understanding model, and the model is trained with the goal of outputting the fields and parameters of the corresponding key information in the training samples.

[0186] In one possible implementation, the replacement module 1705 is used to determine a preset stylized template corresponding to the user's intent under the user's selected dialogue style, and replace the corresponding content of the preset stylized template with parameters in the structured data, including:

[0187] Determine the dynamic preset stylized template corresponding to the user intent under the user's selected dialogue style, and the fields in the dynamic preset stylized template;

[0188] Identify the target field in the dynamically preset stylized template that matches the structured data, and replace the parameter corresponding to the target field with the parameter corresponding to the structured data.

[0189] In one possible implementation, the acquisition module 1703 is configured to acquire, according to the service type, the content to be played from the corresponding content playback library, which corresponds to the user intent, including:

[0190] When the service type is determined to be a local preset service type, the content to be played corresponding to the user intent is obtained from the local preset content playback library;

[0191] When the service type is determined to be a non-local service type, the content to be played corresponding to the user's intent is obtained from the content playback library of other servers;

[0192] In this process, the service type is matched with the content playback library in advance.

[0193] In one possible implementation, the replacement module 1705 is used to replace the corresponding content of the preset stylized template with parameters in the structured data to obtain the text to be broadcast, including:

[0194] When it is determined that the semantic expression style of the parameters in the structured data is consistent with the dialogue style selected by the user, the parameters in the structured data are used to replace the corresponding content of the preset stylized template to obtain the text to be broadcast.

[0195] When it is determined that the semantic expression style of the parameters in the structured data is inconsistent with the dialogue style selected by the user, the parameters are converted into parameters with the same semantics and consistent with the dialogue style selected by the user, and then the corresponding content of the preset stylized template is replaced to obtain the text to be broadcast.

[0196] Based on the same inventive concept, embodiments of this application also provide a device for switching voice interaction dialogue styles, such as... Figure 18 As shown, the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute any of the methods for switching voice interaction dialogue styles on the server side in the above embodiments.

[0197] The following reference Figure 18 To describe an electronic device 130 according to this embodiment of the present application. Figure 18 The electronic device 18 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0198] like Figure 18 As shown, the electronic device 130 is presented in the form of a general-purpose electronic device. The components of the electronic device 130 may include, but are not limited to: at least one processor 131, at least one memory 132, and a bus 133 connecting different system components (including memory 132 and processor 131).

[0199] The processor 131 is used to read and execute instructions from the memory 132, so that the at least one processor can execute the method for switching voice interaction dialogue styles on the server side provided in the above embodiments.

[0200] Bus 133 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0201] Electronic device 130 can also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with electronic device 130, and / or with any device that enables electronic device 130 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 135. Furthermore, electronic device 130 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 136. As shown, network adapter 136 communicates with other modules used in electronic device 130 via bus 133. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0202] In some possible implementations, the various aspects of the method for switching voice interaction dialogue styles applied to a server provided in this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps of the method for switching voice interaction dialogue styles applied to a server according to the various exemplary embodiments of this application described above.

[0203] Based on the same inventive concept, embodiments of this application also provide a user equipment, such as... Figure 19 As shown, the user equipment includes: a speaker, a memory, and a processor;

[0204] The speaker 1905 is configured to broadcast the synthesized text to be broadcast, fed back by the server.

[0205] The processor 1901 is used to read instructions from the memory 1902 and execute a method for switching voice interaction dialogue styles applied to the user device.

[0206] The memory 1902 is used to store various instructions and programs for the method of switching voice interaction dialogue style applied to user equipment provided in the above embodiments.

[0207] Communication interface 1903: Used for communication between processor 1901, memory 1902 and speaker 1905.

[0208] The 1904 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 19 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0209] Processor 1901 can be a central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), or any combination of CPU, NP, and GPU. It can also be a hardware chip. The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0210] In addition, this application also provides a computer-readable storage medium, such as Figure 20 The computer storage medium shown stores a computer program that causes the computer to perform any of the methods described in the above embodiments.

[0211] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0212] The memory may also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0213] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0214] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0215] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0216] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for switching voice interaction dialogue styles, characterized in that, Applied to the server side, the method includes: Receive and recognize voice information sent by a user equipment to obtain text information corresponding to the voice information; The user intent is determined based on the text information, and the corresponding service type is determined based on the user intent. According to the service type, retrieve the content to be played from the corresponding content playback library that corresponds to the user's intent; When it is determined that the type of the content to be broadcast corresponding to the user intent is dynamic unstructured data, a semantic understanding model is used to extract key information related to the user intent from the dynamic unstructured data, and the fields and parameters corresponding to the key information are determined to obtain the structured data corresponding to the content to be broadcast; wherein, the structured data is the fields and parameters corresponding to the key information in the broadcast content, which is data logically expressed and implemented by a two-dimensional table structure, while unstructured data is data with irregular or incomplete data structure and no predefined data model; Determine the preset stylized template corresponding to the user's intent under the user's selected dialogue style, and replace the corresponding content of the preset stylized template with the parameters in the structured data to obtain the text to be broadcast; The text to be broadcast is synthesized into speech and then broadcast through the speaker of the user's device.

2. The method according to claim 1, characterized in that, Also includes: When it is determined that the type of the content to be broadcast corresponding to the user's intent is dynamic structured data, the structured data corresponding to the content to be broadcast is obtained.

3. The method according to claim 1 or 2, characterized in that, Also includes: When it is determined that the type of the content to be broadcast corresponding to the user intent is static data, a static preset stylized template corresponding to the user intent under the style selected by the user is obtained, and the content of the static preset stylized template is used as the text to be broadcast.

4. The method according to claim 1, characterized in that, The semantic understanding model is trained in the following manner, including: Obtain training samples, which include dynamic unstructured data and corresponding user intents, as well as fields and parameters of key information in dynamic structured data; The dynamic unstructured data in the training samples and the corresponding user intent are input into the semantic understanding model, and the model is trained with the goal of outputting the fields and parameters of the corresponding key information in the training samples.

5. The method according to claim 1, characterized in that, Determine the preset stylized template corresponding to the user's intent under the user's selected dialogue style, and replace the corresponding content of the preset stylized template with parameters in the structured data, including: Determine the dynamic preset stylized template corresponding to the user intent under the user's selected dialogue style, and the fields in the dynamic preset stylized template; Identify the target field in the dynamically preset stylized template that matches the structured data, and replace the parameter corresponding to the target field with the parameter corresponding to the structured data.

6. The method according to claim 1, characterized in that, Based on the service type, retrieve the content to be played corresponding to the user's intent from the corresponding content playback library, including: When the service type is determined to be a local preset service type, the content to be played corresponding to the user intent is obtained from the local preset content playback library; When the service type is determined to be a non-local service type, the content to be played corresponding to the user's intent is obtained from the content playback library of other servers; In this process, the service type is matched with the content playback library in advance.

7. The method according to claim 1, characterized in that, The parameters in the structured data are replaced with the corresponding content of the preset stylized template to obtain the text to be broadcast, including: When it is determined that the semantic expression style of the parameters in the structured data is consistent with the dialogue style selected by the user, the parameters in the structured data are used to replace the corresponding content of the preset stylized template to obtain the text to be broadcast. When it is determined that the semantic expression style of the parameters in the structured data is inconsistent with the dialogue style selected by the user, the parameters are converted into parameters with the same semantics and consistent with the dialogue style selected by the user, and then the corresponding content of the preset stylized template is replaced to obtain the text to be broadcast.

8. A device for switching voice interaction dialogue styles, characterized in that, Applied to the server side, the device includes: The recognition module is used to receive and recognize voice information sent by the user equipment to obtain the text information corresponding to the voice information; The service type determination module is used to determine the user intent based on the text information and determine the corresponding service type based on the user intent. The acquisition module is used to acquire the content to be played from the corresponding content playback library according to the service type, which corresponds to the user's intent; The structured data module is used to extract key information related to the user intent from the dynamic unstructured data when the type of the acquired content to be broadcast corresponding to the user intent is determined to be dynamic unstructured data using a semantic understanding model, and to determine the fields and parameters corresponding to the key information to obtain the structured data corresponding to the content to be broadcast; wherein, the structured data is the fields and parameters corresponding to the key information in the broadcast content, which is data logically expressed and implemented by a two-dimensional table structure, while unstructured data is data with irregular or incomplete data structure and no predefined data model; The replacement module is used to determine the preset stylized template corresponding to the user's intent under the user's selected dialogue style, and replace the corresponding content of the preset stylized template with the parameters in the structured data to obtain the text to be broadcast; The synthesis module is used to synthesize the text to be broadcast into speech and broadcast it through the speaker of the user equipment.

9. A device for switching voice interaction dialogue styles, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program that enables the computer to perform the method as described in any one of claims 1-7.

Images