Display device, server and voice interaction method

Abstract

The embodiment of the invention discloses a display device, a server and a voice interaction method, and the method comprises the steps: obtaining a voice text corresponding to voice data when the voice data is received; based on a preset custom lexicon and the voice text, obtaining a target custom vocabulary contained in the voice text and a target standard name and a target scene label mapped by the target custom vocabulary; the preset custom lexicon comprises a mapping relation among standard names, custom vocabularies and scene labels; sending a voice recognition request to a server to request to obtain a voice interaction instruction generated by the server based on the user-defined recognition result, the target standard name and the general semantic recognition result; the user-defined recognition result is output based on a target scene label and a target user-defined vocabulary, and the universal semantic recognition result is output after semantic understanding is performed on the voice text; and the interaction content indicated by the voice interaction instruction is displayed, so that the voice interaction efficiency and precision are improved, and personalization and universality are considered.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a display device, a server, and a speech interaction method. Background Technology

[0002] When users interact with display devices via voice, they typically utter speech containing keywords so that the semantic understanding system can accurately interpret the speech content and recognize the user's intent. For example, when a user says "switch to HDMI1," the keyword is "HDMI1," and the voice recognition result is to switch the signal source to HDMI1.

[0003] Users may customize names for certain technical terms based on their preferences. For example, if HDMI1 connects to a game console and HDMI2 connects to a set-top box, the user might name HDMI1 "My Game Console" and HDMI2 "Home Theater," causing the general semantic understanding system to fail to recognize these custom terms. Furthermore, the general semantic understanding system may not accurately interpret the user's intent and command requirements related to these custom terms, leading to biased recognition results and potentially causing erroneous responses to voice commands. This results in the voice interaction content presented by the display device not meeting the user's expectations, impacting the efficiency and quality of voice interaction. Summary of the Invention

[0004] Some embodiments of this application provide a display device, a server, and a voice interaction method. By using a pre-built custom lexicon, custom words contained in the voice text are accurately identified. The server performs parallel general semantic recognition and custom recognition, and sends more comprehensive and user-preferred voice interaction commands to the display device by combining the two recognition results. This enables the display device to present more accurate voice interaction content, improves the efficiency and accuracy of voice interaction, and balances the personalization and universality of voice interaction.

[0005] In a first aspect, some embodiments of this application provide a display device, including: The display is configured to show interactive content; A voice acquisition device is configured to collect voice data input by the user; A communication device configured to communicate with a server; The controller is configured as follows: Upon receiving the voice data, obtain the corresponding voice text. Based on a preset custom lexicon and the voice text, the target custom words contained in the voice text, as well as the target standard names and target scene tags mapped by the target custom words are obtained; wherein, the preset custom lexicon includes the mapping relationship between standard names, custom words and scene tags, and the scene tags are used to mark the application scene to which the custom words belong; A speech recognition request is sent to the server; the speech recognition request includes the speech text, the target custom vocabulary, the target standard name, and the target scene tag; The system receives a voice interaction command from the server in response to the voice recognition request; the voice interaction command is generated by the server based on a custom recognition result, the target standard name, and a general semantic recognition result; the custom recognition result is the recognition result output by the server based on the target scene label and the target custom vocabulary, and the general semantic recognition result is the recognition result output by the server after performing semantic understanding on the voice text; In response to the voice interaction command, the display is controlled to show the interactive content indicated by the voice interaction command.

[0006] The beneficial effects of the embodiments of the first aspect described above are as follows: The display device can pre-build a preset custom dictionary, which includes custom words and their mapped standard names and scene tags. The scene tags are the application scenarios corresponding to the custom words. For example, the standard name mapped to "my game console" is HDMI1, and the mapped scene tag is signal source. When the display device receives voice data input by the user, it obtains the voice text converted from the voice data and, based on the preset custom dictionary, identifies whether the voice text contains custom words. If it contains target custom words, it sends a voice recognition request to the server. The voice recognition request contains the voice text, the target custom words, and the target standard name and target scene tag mapped to the target custom words. In this way, the server reads the target scene tag and target custom words from the voice recognition request, performs custom recognition, and provides a custom recognition result. It also performs general semantic understanding on the voice text and provides a general semantic recognition result. Finally, it combines the two recognition results to decide on the generation of voice interaction commands. The display device displays the corresponding voice interaction content based on the voice interaction command. Since the voice interaction content integrates custom recognition of target custom words and their scene tags with general semantic recognition, the voice interaction content is more comprehensive. It not only meets the usage scenario preferences of custom words, but also takes into account the command intent of general semantic recognition. That is, it takes into account both the personalization and universality of voice interaction, so that users can select the interactive content they are interested in as needed, thereby improving the efficiency and accuracy of voice interaction.

[0007] In some embodiments of the first aspect, the controller obtains target custom words contained in the speech text based on a preset custom word library and the speech text, specifically configured to: if the preset custom word library contains a first custom word whose similarity to a first word is greater than a threshold, then the first custom word is determined as the target custom word; wherein, the first word is a noun identified from the speech text.

[0008] The beneficial effects of this embodiment are as follows: By comparing the similarity of nouns contained in the speech text with custom words in a preset custom lexicon, the first custom word with a similarity greater than a threshold is found in the preset custom lexicon. Since there may be errors when converting speech data to speech text, such as converting "my game console" to "my game chicken", the preset custom lexicon is used to screen and determine that the first custom word is "my game console". If the first word is "my game chicken", then the first custom word is determined as the target custom word contained in the speech text, instead of the first word being determined as the target custom word. This corrects the misidentified custom words that occur during speech text conversion, improves the accuracy of subsequent custom recognition and general semantic understanding, and thus improves the accuracy of voice interaction, avoiding problems such as misrecognition and inability to recognize voice commands.

[0009] In some embodiments of the first aspect, the controller is further configured to: replace the first word in the speech text with the target custom word before sending the speech recognition request to the server, to obtain a rewritten speech text; and create the speech recognition request based on the rewritten speech text, the target custom word, the target standard name, and the target scene tag.

[0010] The beneficial effects of this embodiment are as follows: Since errors may occur when converting voice data into voice text, after identifying the target custom word through a preset custom word library, the target custom word replaces the first word, for example, replacing "my game chicken" with "my game console", while keeping the other content of the voice text unchanged, thereby correcting the voice text, avoiding misrecognition in subsequent general semantic understanding, and thus improving the accuracy of voice interaction.

[0011] In some embodiments of the first aspect, the controller, in response to the voice interaction command, controls the display to display the interactive content indicated by the voice interaction command. Specifically, it is configured to: if the voice interaction command is a first voice interaction command, control the display to display the first interactive content corresponding to the custom recognition result in a first window, and the second interactive content corresponding to the general semantic recognition result in a second window; wherein, the first voice interaction command is sent by the server when the custom recognition result contains only the target custom vocabulary, or, the first interactive command is sent by the server when both the custom recognition result and the general semantic recognition result contain a clear intent, and the instruction action indicated by the clear intent can be effectively executed.

[0012] The beneficial effects of this embodiment are as follows: When the custom recognition result only contains the target custom vocabulary, it indicates that the custom recognition path has not recognized a clear instruction intent. In this case, a general semantic recognition result can be attached, and the interactive content of the two recognition results can be presented in a split window on the same screen, allowing the user to select the desired interactive content as needed. When both the custom recognition result and the general semantic recognition result contain a clear intent, since it is uncertain which recognition result better meets the user's expectations and preferences, the interactive content corresponding to the two recognition results is also presented in a split window on the same screen, allowing the user to select as needed, avoiding misrecognition or situations where a single recognition result does not meet the user's requirements.

[0013] In some embodiments of the first aspect, the controller, in response to the voice interaction command, controls the display to display the interactive content indicated by the voice interaction command, specifically configured to: if the voice interaction command is a second voice interaction command, control the display to display the first interactive content corresponding to the custom recognition result in full-screen mode; wherein, the second voice interaction command is sent by the server when the custom recognition result contains a clear intent, and the general semantic recognition result contains a chatty intent or does not contain a clear intent.

[0014] The beneficial effects of this embodiment are as follows: when the recognition result is defined to contain a clear intent, if the general semantic recognition result contains a casual conversation intent or does not contain a clear intent, it indicates that the general semantic recognition cannot provide effective interactive content or interactive content that matches the user's preferences. In this way, the server filters out the general semantic recognition result and generates voice interaction commands only based on the custom recognition result, so that the display device only displays the interactive content corresponding to the custom recognition result, thus avoiding interference of general semantic recognition on voice interaction.

[0015] Secondly, some embodiments of this application also provide a server, including: A communicator used for communication connections with display devices; The processor is configured as follows: The system receives a speech recognition request sent by the display device. The speech recognition request includes speech text, target custom words, and target standard names and target scene tags mapped to the target custom words. The target custom words are custom words contained in the speech text, and the target scene tags are used to mark the application scene to which the target custom words belong. Based on the target scene label and the target custom vocabulary, a custom recognition result is obtained; The speech text is semantically understood to obtain a general semantic recognition result; Based on the custom recognition result, the target standard name, and the general semantic recognition result, generate voice interaction commands; The voice interaction command is sent to the display device so that the display device displays the interactive content indicated by the voice interaction command.

[0016] The beneficial effects of the embodiments in the second aspect above are as follows: When the server receives a speech recognition request, it can execute two recognition lines in parallel. The first recognition line reads the target scene tags and target custom words, performs custom recognition, and provides a custom recognition result; the second recognition line performs general semantic understanding on the speech text and provides a general semantic recognition result. The server combines the two recognition results to decide on the generation of the speech interaction command. Based on the speech interaction command, the display device displays the corresponding speech interaction content. Because the speech interaction content integrates custom recognition of the target custom words and their scene tags with general semantic recognition, the speech interaction content is more comprehensive. It not only meets the usage scenario preferences of the custom words but also takes into account the command intent of general semantic recognition, that is, it takes into account both the personalization and universality of speech interaction, allowing users to select the interactive content they are interested in as needed, thereby improving the efficiency and accuracy of speech interaction.

[0017] In some embodiments of the second aspect, the processor obtains a custom recognition result based on the target scene label and the target custom vocabulary, specifically configured to: determine the intent generation rule matching the target scene label, the intent generation rule including explicit intent rules and fuzzy intent rules; if the intent generation rule matching the target scene label is the fuzzy intent rule, then output the custom recognition result containing only the target custom vocabulary; if the intent generation rule matching the target scene label is the explicit intent rule, then obtain the target action vocabulary associated with the target scene label; combine the target custom vocabulary and the target action vocabulary to form an instruction sentence containing explicit intent, and output the custom recognition result containing the instruction sentence; the instruction action indicated by the explicit intent can be effectively executed.

[0018] The beneficial effects of this embodiment are as follows: The server can construct intent generation rules for different scene tags. Some scene tags are configured with explicit intent rules, while others are configured with fuzzy intent rules. For example, when the scene tag is "signal source," the typical usage scenario for a signal source is "turn on the signal source" or "switch to the signal source," which corresponds to an explicit intent. Conversely, if the scene tag is "umm," the intent is ambiguous, corresponding to a fuzzy intent. Thus, when the server receives a speech recognition request and executes recognition path one, it reads the category of the intent generation rule corresponding to the target scene tag. If it corresponds to a fuzzy intent rule, the custom recognition result only contains the target custom vocabulary. If it corresponds to an explicit intent rule, it obtains the target action vocabulary associated with the target scene tag. For example, the action vocabulary for a signal source is "turn on" or "switch to." The target custom vocabulary and action vocabulary are then combined to form a command sentence with a clear intent (e.g., "turn on my game console"), and the custom recognition result outputs this command sentence. In this way, through custom vocabulary and scene tags, the recognition results can be matched to different categories of intent generation rules, thereby attaching possible action vocabulary that conforms to the usage habits and intent preferences of most users to the custom vocabulary, making voice command recognition and interaction more personalized.

[0019] In some embodiments of the second aspect, the processor generates voice interaction instructions based on the custom recognition result, the target standard name, and the general semantic recognition result. Specifically, it is configured to: if the custom recognition result contains only the target custom vocabulary, or if both the custom recognition result and the general semantic recognition result contain a clear intent, then generate a first sub-command based on the custom recognition result and the target standard name; generate a second sub-command based on the general semantic recognition result; and generate a first voice interaction instruction based on the first sub-command and the second sub-command. The first interaction instruction instructs the display device to display first interactive content corresponding to the custom recognition result in a first window in response to the first sub-command, and to display second interactive content corresponding to the general semantic recognition result in a second window in response to the second sub-command.

[0020] The beneficial effects of this embodiment are as follows: When the custom recognition result only contains the target custom vocabulary, it indicates that the custom recognition path has not recognized a clear instruction intent. In this case, a general semantic recognition result can be attached, and the interactive content of the two recognition results can be presented in a split-window mode on the same screen, allowing the user to select the desired interactive content as needed. When both the custom recognition result and the general semantic recognition result contain clear intent, since it is uncertain which recognition result better meets the user's expectations and preferences, the interactive content corresponding to the two recognition results is also presented in a split-window mode on the same screen, avoiding misrecognition or situations that do not meet user requirements due to a single recognition result. In this way, a first sub-command is generated based on the custom recognition result and the target standard name, and a second sub-command is generated based on the general semantic recognition result, so that the first voice interaction command integrates the sub-commands of the two recognition results. The display device responds to the two sub-commands and displays two interactive contents in a split-window mode on the same screen, allowing the user to select as needed, thereby improving the quality of voice interaction and user satisfaction.

[0021] In some embodiments of the second aspect, the processor generates voice interaction instructions based on the custom recognition result, the target standard name, and the general semantic recognition result. Specifically, it is configured to: if the custom recognition result contains a clear intent, and the general semantic recognition result contains a casual conversation intent or does not contain a clear intent, then generate a second voice interaction instruction based on the instruction phrase and the target standard name; wherein the second interaction instruction is used to instruct the display device to display the first interaction content corresponding to the custom recognition result in full-screen mode.

[0022] The beneficial effects of this embodiment are as follows: when the recognition result is defined to contain a clear intent, if the general semantic recognition result contains a casual conversation intent or does not contain a clear intent, it indicates that the general semantic recognition cannot provide effective interactive content or interactive content that matches the user's preferences. In this way, the server filters out the general semantic recognition result and generates voice interaction commands only based on the custom recognition result, so that the display device only displays the interactive content corresponding to the custom recognition result, thus avoiding interference of general semantic recognition on voice interaction.

[0023] Thirdly, some embodiments of this application also provide a voice interaction method, including: Upon receiving voice data, obtain the corresponding voice text. Based on a preset custom lexicon and the voice text, the target custom words contained in the voice text, as well as the target standard names and target scene tags mapped by the target custom words are obtained; wherein, the preset custom lexicon includes the mapping relationship between standard names, custom words and scene tags, and the scene tags are used to mark the application scene to which the custom words belong; A speech recognition request is sent to the server; the speech recognition request includes the speech text, the target custom vocabulary, the target standard name, and the target scene tag; The system receives a voice interaction command from the server in response to the voice recognition request; the voice interaction command is generated by the server based on a custom recognition result, the target standard name, and a general semantic recognition result; the custom recognition result is the recognition result output by the server based on the target scene label and the target custom vocabulary, and the general semantic recognition result is the recognition result output by the server after performing semantic understanding on the voice text; In response to the voice interaction command, the interactive content indicated by the voice interaction command is displayed.

[0024] The beneficial effects of the embodiments in the third aspect above are as follows: The display device can pre-build a preset custom dictionary, which includes custom words and their mapped standard names and scene tags. The scene tags are the application scenarios corresponding to the custom words. For example, the standard name mapped to "my game console" is HDMI1, and the mapped scene tag is signal source. When the display device receives voice data input by the user, it obtains the voice text converted from the voice data and identifies whether the voice text contains custom words based on the preset custom dictionary. If it contains the target custom words, it sends a voice recognition request to the server. The voice recognition request contains the voice text, the target custom words, and the target standard name and target scene tag mapped to the target custom words. In this way, the server reads the target scene tag and the target custom words from the voice recognition request, performs custom recognition, and gives the custom recognition result. It also performs general semantic understanding on the voice text and gives the general semantic recognition result. Combining the two recognition results, it decides on the generation of voice interaction commands. The display device displays the corresponding voice interaction content based on the voice interaction command. Since the voice interaction content integrates custom recognition of target custom words and their scene tags with general semantic recognition, the voice interaction content is more comprehensive. It not only meets the usage scenario preferences of custom words, but also takes into account the command intent of general semantic recognition. That is, it takes into account both the personalization and universality of voice interaction, so that users can select the interactive content they are interested in as needed, thereby improving the efficiency and accuracy of voice interaction. Attached Figure Description

[0025] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application; Figure 3 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application; Figure 4 A schematic diagram of a custom word settings page provided in some embodiments of this application; Figure 5 A schematic diagram illustrating a list of signal sources provided for some embodiments of this application; Figure 6 A flowchart illustrating a voice interaction method provided in some embodiments of this application; Figure 7 A schematic diagram illustrating the principle of obtaining target custom vocabulary as provided in some embodiments of this application; Figure 8 A schematic diagram illustrating the principle of server execution line one provided in some embodiments of this application; Figure 9 A schematic diagram of a display device for displaying voice interaction content provided in some embodiments of this application. Figure 1 ; Figure 10 A schematic diagram of a display device for displaying voice interaction content provided in some embodiments of this application. Figure 2 ; Figure 11 A schematic diagram of a display device for displaying voice interaction content provided in some embodiments of this application. Figure 3 ; Figure 12 A schematic diagram of a display device for displaying voice interaction content provided in some embodiments of this application. Figure 4 ; Figure 13 Timing signaling diagram for some embodiments of the voice interaction method provided in this application. Detailed Implementation

[0026] In this application embodiment, "display device" generally refers to a device with screen display and data processing capabilities. For example, display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0027] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, a user can operate the display device 200 via touch operation and control device 100. The control device 100 receives user input commands and converts them into control commands that the display device 200 can recognize and respond to. For example, the control device 100 can be a remote control, stylus, gamepad, etc. The control device 100 may have a voice button; by pressing and holding this button, user voice data can be input. The control device 100 receives the voice data and sends it to the display device 200.

[0028] like Figure 1 The diagram also shows that the display device 200 communicates with the server 300 via various communication methods, allowing the server 300 to execute related voice processing programs, including ASR (Automatic Speech Recognition), semantic understanding, intent recognition, and voice command generation. The display device 200 can communicate via a local area network (LAN), wireless local area network (WLAN), and other networks.

[0029] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0030] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface 280.

[0031] In some embodiments, detector 230 is used to acquire signals from the external environment or interactions with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds, and during voice interaction, the sound acquisition device can acquire voice data input by the user. The sound acquisition device may include a microphone array (built-in far field) to support far-field voice interaction. If the sound acquisition device does not include a microphone array, the display device can connect an external microphone array (external far field) via a peripheral interface (e.g., a USB interface).

[0032] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0033] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 300 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0034] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0035] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0036] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0037] In some embodiments, the display device 200 may run an operating system to perform user interaction. The operating system is a computer program used to manage and control the hardware and software resources of the display device 200. The operating system can be divided into different modules or levels according to the functions it implements, for example, such as... Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0038] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0039] In some embodiments, see Figure 3 The application layer can run multiple applications, such as Application 1, Application 2, etc. For voice interaction scenarios, the application layer can run a voice assistant application, a signal source playback application (e.g., LiveTV), and other applications that can interact with the voice assistant application. The user can wake up the voice assistant application using a wake word. Once awakened, the voice assistant application can acquire voice data through a sound acquisition device and respond to commands corresponding to the voice data, such as answering questions or displaying voice interaction content on a display. The controller can run at least one related application within the application layer to implement the technical solution provided in the embodiments of this application.

[0040] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0041] See Figure 3 In this embodiment of the application, the application framework layer includes a view system, managers, and content providers. The view system can design and implement the application's interface and interaction. The view system includes lists, grids, text boxes, buttons, etc.

[0042] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and navigating back. The Window Manager manages all windowed applications, such as obtaining screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes in the displayed windows.

[0043] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0044] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 3 As shown, hardware drivers can be configured in the kernel layer. The kernel layer can contain at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB (Universal Serial Bus) driver, HDMI (High Definition Multimedia Interface) driver, and power driver, etc.

[0045] Users can control and interact with the display device 200 via voice commands or through voice assistant applications. When users input voice data, they typically utter speech containing keywords so that the semantic understanding system (configurable on the server side) can accurately understand the speech content and recognize the user's voice intent. For example, when a user says "switch to HDMI1," the keyword is "HDMI1," and the voice recognition result is to switch the signal source to HDMI1.

[0046] Users may customize the names of certain technical terms based on their usage habits and preferences. In some application scenarios, the display device 200 has multiple HDMI ports, with HDMI1 connecting to game consoles and HDMI2 connecting to set-top boxes. However, users may not be able to accurately distinguish which HDMI port an external device is connected to. In this case, users may customize the names of the HDMI ports, for example, naming HDMI1 "My Game Console" and HDMI2 "Home Theater".

[0047] Because the general semantic understanding system does not include user-defined words, it cannot establish a mapping relationship between user-defined words and standard names. As a result, the general semantic understanding system cannot recognize user-defined words. In addition, the general semantic understanding system may not be able to accurately understand the user's intent and instruction requirements for user-defined words, resulting in deviations in the recognition results of the general semantic understanding system. This may lead to erroneous responses to voice commands, causing the voice interaction content presented by the display device 200 to not meet the user's expectations. This may result in the user re-entering voice data containing standard names or correcting the error through manual operation, affecting the efficiency and quality of voice interaction.

[0048] Figure 4This is a schematic diagram of a custom word settings page provided in some embodiments of this application.

[0049] In some embodiments, see Figure 4 Users can access the custom word settings page 40 through preset methods. These preset methods can include clicking a preset button on the control device 100, inputting a voice command or gesture to open the custom word settings page, or opening the settings menu through the overall settings button and selecting the custom word settings option from the settings menu to open the next level of the custom word settings page.

[0050] In some embodiments, see Figure 4 The custom word settings page 40 includes a first input box control 41, a second input box control 42, a third input box control 43, a submit button 44, and a cancel button 45.

[0051] The first input box control 41 is used to input a standard name, the second input box control 42 is used to input custom words mapped to the standard name, and the third input box control 43 is used to input scene labels mapped to the custom words. The scene labels are the application scenarios to which the custom words belong. For example, see... Figure 4 For example, the standard name is "HDMI1", the custom word mapped to the standard name is "my game console", and the scene label mapped to the custom word is signal source.

[0052] In some embodiments, the controller 250 can automatically match scene labels based on standard names and custom terms entered by the user, and automatically fill the text content of the scene labels into the third input box control 43. For example, if the scene label for HDMI1 and my game console is "Source", then the text "Signal Source" of the scene label will be filled into the third input box control 43. In this way, the user does not need to manually enter the scene label, and the user can modify or reset the scene label through the third input box control 43 if the scene label is incorrectly recognized or does not meet the user's expectations.

[0053] In some embodiments, in response to the user clicking the submit button 44, the controller controls the display to close the custom word settings page 40 and saves the mapping relationship between custom words, standard names and scene tags.

[0054] In some embodiments, the display device may maintain and create a preset custom thesaurus, which stores custom setting data submitted by the user through the custom word setting page 40, i.e., stores the mapping relationship between custom words, standard names, and scene tags. The preset custom thesaurus may store data in the form of a list, as shown in Table 1: Table 1

[0055] In some embodiments, in response to a user clicking the cancel button 45, the controller controls the display to close the custom word settings page 40 and does not save the custom settings data.

[0056] In some embodiments, the display device can connect to an external CEC (Consumer Electronics Control) device via an HDMI interface. The controller can monitor the connection status of each HDMI interface. When it detects that any target HDMI has switched from an unconnected state to a connected state, it obtains the device name of the CEC device connected to that target HDMI and automatically sets the custom terminology for the target HDMI mapping to that device name. Referring to Table 1, when HDMI3 is connected to a CEC device, the controller obtains the device name of the CEC device as "Xbox", and determines the custom terminology for HDMI3 mapping to be Xbox; when HDMI4 is connected to a CEC device, the controller obtains the device name of the CEC device as "Apple TV", and determines the custom terminology for HDMI3 mapping to be Apple TV. Since HDMI all belong to the Source category, the scene label for Xbox and Apple TV mapping is automatically configured as signal source. In this way, the HDMI interface is automatically named when the CEC device is connected, thus eliminating the need for the user to set the custom terminology for HDMI mapping through the custom terminology setting page 40.

[0057] Figure 5 This is a schematic diagram illustrating a list of signal sources provided for some embodiments of this application.

[0058] In some embodiments, see Figure 5 The signal source list 50 includes multiple signal source options 51, which are used to display signal source information. This information can include at least the signal source name. The display format for the signal source name can be set to a standard name / custom term, such as HDMI1 / My Game Console, HDMI2 / Home Theater, HDMI3 / My Xbox, etc. HDMI1 / My Game Console indicates that the custom term mapped to HDMI1 is "My Game Console".

[0059] In this way, users can clearly see the custom words mapped to each HDMI signal source through the signal source list 50, and control the display device to turn on / switch to the corresponding HDMI signal source by saying "turn on / switch to the custom words" during voice interaction. For example, if the user says "switch to my game console", the controller can switch to the HDMI1 signal source mapped to "my game console" by executing setCurrentSource(HDMI1).

[0060] Figure 6 A flowchart illustrating a voice interaction method provided in some embodiments of this application.

[0061] See Figure 6 The interactive objects involved in this voice interaction process include a display device and a server. The display device includes a sound acquisition unit, a display, a communication device, and a controller. The controller receives voice data through the sound acquisition unit, communicates and interacts with the server through the communication device, and controls the display to display the voice interaction content. The controller is configured to execute the software program on the display device side.

[0062] In some embodiments, see Figure 6 The controller receives voice data through the sound acquisition device and obtains the corresponding voice text.

[0063] In some embodiments, the display device 200 may be configured with an ASR module, which is used to convert human speech into computer-readable text. In this way, the controller sends speech data to the ASR module, which then converts the speech data into corresponding speech-to-text.

[0064] In some embodiments, the server may be configured with an ASR service. The controller may send an ASR request to the server, which includes voice data. In response to the ASR request, the server retrieves the voice data from the request and sends it to the ASR service. The ASR service converts the voice data into speech-to-text, and the server then sends the speech-to-text to the display device. This embodiment executes the ASR program on the server side, reducing the computational load on the display device and ensuring its performance.

[0065] In some embodiments, see Figure 6 The controller retrieves target custom words contained in the speech text based on a pre-defined custom lexicon. In this way, the controller can obtain the target standard name and target scene label mapped to the target custom word from the pre-defined custom lexicon.

[0066] Figure 7 This is a schematic diagram illustrating the principle of obtaining target custom vocabulary as provided in some embodiments of this application.

[0067] In some embodiments, see Figure 7 The controller can identify nouns (hereinafter referred to as "first words") contained in the spoken text and compare the similarity of the first words with custom words in a preset custom word library to find first custom words in the preset custom word library whose similarity to the first words is greater than a threshold. The threshold value is not limited, for example, it is 95%.

[0068] In some embodiments, if there is only one first custom word, the controller can directly determine that first custom word as the target custom word. If there are multiple first custom words, the controller can determine the first custom word with the highest similarity to the first word as the target custom word.

[0069] In some embodiments, the controller can calculate the similarity between text using methods such as edit distance. The edit distance algorithm (Levenshtein Distance) is a classic method for calculating the similarity between two strings. It quantifies the difference by calculating the minimum number of editing operations (including insertion, deletion, or replacement of characters) required to transform one string into another. The smaller the edit distance, the higher the string similarity. See Figure 7 For example, suppose the spoken text is "Open my game chicken". The system recognizes the noun "my game chicken" as the first word in the spoken text. Then, it compares "my game chicken" with the pre-defined custom words in the custom word library. i Perform a similarity comparison, that is, calculate the similarity between "my game chicken" and "CustomizedWord". i Similarity between i ,in i Indicates the sequence number of the custom vocabulary, 1≤ i ≤M, where M represents the total number of custom words in the preset custom word library, CustomizedWord( i ) indicates the first i A custom vocabulary.

[0070] This involves calculating the similarity scores: 1 between "My Game Chicken" and "My Game Console", 2 between "My Game Chicken" and "Home Theater", 3 between "My Game Chicken" and "Xbox", 4 between "My Game Chicken" and "Apple TV", and 5 between "My Game Chicken" and "Penguin". A similarity threshold is then used for screening and matching. If a similarity score of 1 is greater than the threshold, then "My Game Console" is selected as the first custom keyword.

[0071] ASR (Automatic Speech Recognition) is based on matching user pronunciation with corresponding text. However, due to homophones (the same pronunciation may map to multiple characters), ASR may introduce errors when converting speech data into speech-text. For example, "chicken" and "machine" are homophones, so "my game console" might be converted to "my game chicken." By using a pre-defined custom vocabulary library for word similarity screening, and determining that the first custom word is "my game console," this first custom word is identified as the target custom word contained in the speech-text, rather than being directly identified as the target custom word. This corrects the incorrect custom words that occur during ASR conversion, improving the accuracy of subsequent custom recognition and general semantic understanding, thereby improving the accuracy of voice interaction and avoiding problems such as misrecognition and inability to recognize voice commands. Thus, participants... Figure 7 The controller determines the target custom term as "my game console", the target standard name as "HDMI1", and the target scene label as signal source.

[0072] In some embodiments, see Figure 7 The controller can replace the first word in the spoken text with a target custom word to obtain the rewritten spoken text. For example, if the original spoken text is "Open my game chicken", the controller can replace "my game chicken" with "my game console" while keeping the other text content unchanged, resulting in the rewritten spoken text "Open my game console". In this way, by screening the similarity between the name in the spoken text and custom words in a preset custom vocabulary library, the accuracy of custom word recognition is improved, and errors in ASR-recognized spoken text are corrected to avoid misrecognition in subsequent general semantic understanding, thereby improving the accuracy of voice interaction.

[0073] In some embodiments, see continue to see Figure 6 The controller can create a speech recognition request and send it to the server. The speech recognition request includes the spoken text, target custom vocabulary, target standard name, and target scene label. An example of a JSON structure for a speech recognition request is shown below: {retext:“Open my game chicken”; rewrite: "Turn on my game console"; HDMISetting:{HDMI1:My game console;} Scene label:Source}; request:1.0} Here, `retext` represents the original speech text, such as "Open my game chicken". `rewrite` represents the rewritten speech text, such as "Open my game console". `HDMISetting` includes the mapping between target custom terms and target standard names (e.g., HDMI1: my game console), and also includes a Scene label, which represents the target scene label (e.g., Source / signal source). `request` represents the request version of the speech recognition request, such as HTTP 1.0.

[0074] In some embodiments, the speech recognition request may contain only the rewritten speech text, without necessarily including the original speech text. When the server responds to the speech recognition request, it performs general semantic understanding and recognition based on the rewritten speech text, which can improve the accuracy of speech command recognition and intent analysis, avoid misrecognition problems caused by ASR bias, and improve the accuracy of voice interaction.

[0075] In some embodiments, when a target custom word is recognized in the speech text, the controller can write the data of the preset custom word library into the speech recognition request, and synchronize the data of the preset custom word library to the server by sending the speech recognition request to the server, so that the server can store and maintain the preset custom word library.

[0076] In some embodiments, see continue to see Figure 6 When a server responds to a speech recognition request, it can execute two speech recognition lines in parallel, namely Line 1 and Line 2.

[0077] Among them, Line 1 corresponds to custom semantic recognition, which is configured to obtain custom recognition results based on target scene tags and target custom vocabulary. Custom semantic recognition dynamically constructs the first instruction intent for the target custom vocabulary by combining the target scene tags.

[0078] Line 2 corresponds to general semantic understanding. Line 2 is configured to: perform semantic understanding on the rewritten speech text, predict candidate intentions with different confidence levels, and determine the candidate intention with the highest confidence level as the intention contained in the general semantic recognition result. That is, perform semantic understanding and intention analysis on the complete speech text to obtain the second instruction intention carried by the speech data itself. The second instruction intention may be the same as or different from the first instruction intention.

[0079] In some embodiments, the server can be configured with a custom semantic recognition model and a general semantic understanding model, wherein the custom semantic recognition model is used to execute line one and the general semantic understanding model is used to execute line two, thereby ensuring that the two lines run in parallel and do not interfere with each other.

[0080] In some embodiments, see continue to see Figure 6 The server can aggregate the recognition results from two lines to generate voice interaction commands and transmit these commands to the display device based on a communication protocol. The server generates voice interaction commands based on custom recognition results, general semantic recognition results, and target standard names. These commands balance personalization and universality, standardizing the voice interaction commands by mapping the custom vocabulary involved in the dual-line recognition results to target standard names, thus facilitating accurate command response from the computer on the display device.

[0081] In some embodiments, see continue to see Figure 6 The controller responds to voice interaction commands and controls the display to show the interactive content indicated by the voice interaction command. Taking the voice data as "Turn on my game console" as an example, if the voice interaction command indicates "Switch Source to HDMI1", the controller will switch the signal source to HDMI1 and control the display to show the game screen transmitted through HDMI1. This game screen is the interactive content.

[0082] The display device pre-builds a custom lexicon to provide data support for subsequent custom word recognition and text rewriting of speech-text. When the display device receives user-input speech data, it acquires the converted speech-text and, based on the custom lexicon, identifies whether the speech-text contains custom words. If it does, it sends a speech recognition request to the server. The server responds to the speech recognition request via parallel lines one and two, aggregating the recognition results from both lines to determine the generation of voice interaction commands. Based on these commands, the display device displays corresponding voice interaction content. Because the voice interaction content integrates custom semantic recognition and general semantic understanding of the target custom words and their contextual tags, it is more comprehensive. It conforms to shared contextual preferences for custom words while also considering the command intent of general semantic recognition, thus balancing personalization and universality in voice interaction. Based on the mapping relationship between target custom words and target standard names, it generates standardized, machine-responsive voice interaction commands, avoiding erroneous responses and response failures. This allows users to select interactive content of interest as needed, improving the efficiency and accuracy of voice interaction.

[0083] In some embodiments, if the preset custom lexicon does not contain a first custom word whose similarity to the first word is greater than a threshold, the controller can determine that the speech text does not contain the custom word. In this way, the speech recognition request generated by the controller only contains the original speech text recognized by ASR; in response to the speech recognition request, the server only executes line two and does not execute line one, and feeds back the voice interaction command to the server based on the general semantic recognition result.

[0084] Figure 8 This is a schematic diagram illustrating the principle of server execution line one provided in some embodiments of this application.

[0085] In some embodiments, see Figure 8 The server can set scene tags based on business needs. j Corresponding intent generation rules j .in, j Indicates the sequence number of the scene label, 1≤ j ≤N, where N represents the total number of scene tags set on the server side.

[0086] In some embodiments, see Figure 8 Intent generation rules include fuzzy intent rules and explicit intent rules.

[0087] In some embodiments, see Figure 8 The fuzzy intent rule represents an instruction whose attribute is fuzzy, meaning it does not have a clear intent.

[0088] In some embodiments, see Figure 8 Explicit intent rules (also known as hard intent rules) represent instructions with explicit / hard attributes. This means that the server sets the intent and instruction actions based on the target scene tags and the target object (i.e., the target custom vocabulary).

[0089] In some embodiments, if the intention is to generate rules j To clarify the intent rules, the server pre-sets scene tags. j The associated action vocabulary accurately conveys the intent and indicates the action to be performed in response to the command. For example, if the scene label... j As a signal source, in general scenarios, users usually turn on a signal source or switch to a signal source, so the action word can be set to "turn on" or "switch to".

[0090] In some embodiments, see Figure 8 When the server executes line one, it retrieves the target custom vocabulary and target scene tags from the speech recognition request and determines the intent generation rule for matching the target scene tags. Assume the target scene tags and scene tags... j If they are consistent, then the target scene label and the intent generation rule are correct. j Matching, if the intention is to generate rules j For fuzzy intent rules, The server then uses fuzzy intent rules without attaching any custom intents, so that the custom recognition results output by Line 1 only contain the target custom words.

[0091] In some embodiments, see Figure 8 If the intention is to generate rules j To define the intent rule, the server obtains the target scene tag / scene tag based on the defined intent rule. j The server associates target action words with custom target words and semantically combines them to obtain a command sentence containing a clear intent. Thus, the custom recognition result output by Line 1 includes this command sentence. The server can construct intent generation rules for different scene tags. Some scene tags are configured with explicit intent rules, while others are configured with fuzzy intent rules. For example, when the scene tag is "signal source," the typical usage scenario for a signal source is "turn on the signal source" or "switch to the signal source," which corresponds to an explicit intent. The server can pre-associate action words for this type of scene tag. On the other hand, the scene tag is "umm," which has an ambiguous intent, corresponding to a fuzzy intent. For this type of scene tag, the server does not pre-associate action words.

[0092] When the server receives a speech recognition request and executes recognition path one, it reads the category of the intent generation rule corresponding to the target scene label. If it corresponds to a fuzzy intent rule, the custom recognition result only contains the target custom words, such as "{my game console}". If it corresponds to a specific intent rule, it obtains the target action words associated with the target scene label. For example, if the action words corresponding to the signal source scene label are "open" or "switch to", then the target custom words and action words are combined to form a command sentence with a specific intent, such as "{open my game console}" or "{switch to my game console}", and the custom recognition result outputs this command sentence. In this way, the category of intent generation rule can be matched through scene labels, and custom recognition results can be output based on different intent generation rules. For specific intent rules, possible target action words that conform to the usage habits and intent preferences of most users can be attached to the target custom words, making voice command recognition and interaction more personalized, rather than being limited to the results of general semantic understanding and recognition.

[0093] Figure 9 A schematic diagram of a display device for displaying voice interaction content provided in some embodiments of this application. Figure 1 .

[0094] In some embodiments, see Figure 9 If the custom recognition result output by Line 1 only contains the target custom words, it indicates that Line 1 has not defined a clear instruction intent (ambiguous intent). In this case, the general semantic recognition result can be merged to generate a voice interaction command that integrates the recognition results of the two lines, so as to avoid the voice interaction not meeting the user's expectations due to the ambiguity of the intent of Line 1.

[0095] In some embodiments, see Figure 9The server generates a first sub-command corresponding to the first line path based on the custom recognition result and the target standard name, and generates a second sub-command corresponding to the second line path based on the general semantic recognition result. Based on the first and second sub-commands, the server generates a first voice interaction command.

[0096] In some embodiments, since the custom recognition result only contains target custom words in this scenario, and the target custom words map to target standard names, the server can generate a first sub-command based on the target standard names. The server can set a first instruction identifier for the first sub-command and a second instruction identifier for the second sub-command, where the first instruction identifier indicates that the first sub-command is a custom scene instruction generated by line one, and the second instruction identifier indicates that the second sub-command is a general scene instruction generated by line two. The server merges the first and second sub-commands to obtain a first voice interaction instruction, and transmits the first voice interaction instruction to the display device through the communication protocol between the display device and the server.

[0097] In some embodiments, the controller 250 receives a first voice interaction command and can identify subcommands through command identifiers. See also Figure 9 In response to the first sub-command, the controller 250 controls the display to show the first interactive content corresponding to the custom recognition result in the first window W1, and in response to the second sub-command, controls the display to show the second interactive content corresponding to the general semantic recognition result in the second window W2.

[0098] In some embodiments, see Figure 9 The first window W1 can be a full-screen window, and the second window W2 can be a floating thumbnail window, meaning the second window W2 floats above the first window W1, and the size of the second window W2 is smaller than the size of the first window W1. This embodiment defaults to displaying the second interactive content corresponding to Line 2 in a full-screen view, while displaying the first interactive content corresponding to Line 1 in a small window, forming a "picture-in-picture" mode.

[0099] In some embodiments, the display device can set the priority of custom recognition results and general semantic recognition results. If the priority of the custom recognition result is higher than the priority of the general semantic recognition result, the controller 250 can control the display to display the second interactive content corresponding to the general semantic recognition result in the first window W1 and the first interactive content corresponding to the custom recognition result in the second window W2.

[0100] In some embodiments, for example, assuming the user says "penguin," the target custom word is "penguin," the target standard name is "TX Video," and the target scene tag is "App." If Line 1 determines that the target scene tag matches a fuzzy intent rule, the custom recognition result output by Line 1 only contains "penguin." The server detects the fuzzy attribute of the custom recognition result and writes the App identifier (e.g., TXVideo) of the target standard name "TX Video" mapped to "penguin" into the generated first sub-command. Line 2 performs semantic understanding and intent analysis on the spoken text "penguin." If the predicted intent is, for example, to search for penguins, the instruction content (e.g., Search Penguin) is written into the generated second sub-command. Thus, in response to the first sub-command, the display device displays the first interactive content in the first window W1. The first interactive content is, for example, the application icon of "TX Video," which the user can trigger to launch the TX Video application. In response to the second sub-command, the display device displays the second interactive content in the second window W2. The second interactive content is, for example, search results for penguins (including penguin encyclopedia, penguin pictures, etc.). This allows the content from both lines to be displayed on the same screen, enabling users to browse simultaneously and watch as needed.

[0101] Figure 10 A schematic diagram of a display device for displaying voice interaction content provided in some embodiments of this application. Figure 2 .

[0102] In some embodiments, see Figure 10 Upon receiving a first voice interaction command, controller 250 can control the display to show the first and second interactive content in a split-screen mode. Specifically, in response to the first sub-command, controller 250 controls the display to show the first interactive content corresponding to the customized recognition result in the first split-screen window SCW1, and in response to the second sub-command, controls the display to show the second interactive content corresponding to the general semantic recognition result in the second split-screen window SCW2.

[0103] In some embodiments, see Figure 10 For example, if both the first split-screen window SCW1 and the second split-screen window SCW2 are landscape windows, which means that the width of the window is greater than the height of the window, then the sum of the widths of the first split-screen window SCW1 and the second split-screen window SCW2 can be equal to the screen width, and the heights of the first split-screen window SCW1 and the second split-screen window SCW2 can be the same, thus making the two split-screen windows more compact.

[0104] Figure 11 A schematic diagram of a display device for displaying voice interaction content provided in some embodiments of this application. Figure 3 .

[0105] In some embodiments, see Figure 11If the custom recognition result output by Line 1 contains a complete instruction sentence, it indicates that Line 1 has defined a clear instruction intent (explicit intent / hard intent). If the general recognition result output by Line 2 also contains an explicit intent, since it is uncertain which recognition result better meets the user's expectations and preferences, in this case, a voice interaction command that integrates the recognition results of both lines can be generated to avoid misrecognition or failure to meet user requirements caused by a single recognition result. Ultimately, the user can select the interaction content as needed.

[0106] In some embodiments, see Figure 11 The server generates a first sub-command corresponding to Line 1 based on the custom recognition result and the target standard name, and a second sub-command corresponding to Line 2 based on the general semantic recognition result. Based on the first and second sub-commands, it generates a first voice interaction command. The controller 250 receives the first voice interaction command. (See below) Figure 11 In response to a first sub-command, controller 250 controls the display to show the first interactive content corresponding to the custom recognition result in a first window W1, and in response to a second sub-command, controls the display to show the second interactive content corresponding to the general semantic recognition result in a second window W2. The size, display position, and layout of the first window W1 and the second window W2 are not limited.

[0107] In some embodiments, for example, assuming the user says "Turn on my game console," the target custom word is "my game console," the target standard name is "HDMI1," and the target scene label is the signal source. If Line 1 determines that the target scene label matches the explicit intent rule, the custom recognition result output by Line 1 contains the instruction phrase "Turn on my game console." The server detects the hard attribute of the custom recognition result and writes the instruction content (e.g., OpenHDMI1) to indicate turning on the HDMI1 signal language in the generated first sub-command. Line 2 performs semantic understanding and intent analysis on the voice text "Turn on my game console," recognizing "my game console" as a game console application (Gamemachine), and then writes the instruction content (e.g., Open Gamemachine App) in the generated second sub-command. In this way, the display device responds to the first sub-command and displays the first interactive content in the first window W1. The first interactive content is a dynamic screen of switching to HDMI1, including the progress of switching to HDMI1, the HDMI1 switching success prompt message, and the video screen transmitted by HDMI1, etc. In response to the second sub-command, the display device shows the second interactive content in the second window W2. The second interactive content is a dynamic display of the game console application, including the application's startup animation, homepage, and game screen. This allows for the simultaneous display of content from two different sources, enabling users to browse and view content as needed.

[0108] In some embodiments, see Figure 9 and Figure 11 Users can select the focus window using the control device 100 (e.g., a remote control). For example, after the first window W1 gains focus, the user can input a confirmation command (e.g., click the OK button). In response to this confirmation command, the controller can control the display to close the second window W2 and reset the size of the first window W1, switching it to a full-screen window, thus displaying the first interactive content in full-screen mode. In this way, users can select the interactive content they wish to view as needed.

[0109] In some embodiments, see Figure 10 For split-screen mode, users can switch to full-screen mode, and also adjust the size and display position of the first split-screen window SCW1 and the second split-screen window SCW2.

[0110] When the custom recognition result contains only the target custom vocabulary, it indicates that the custom recognition path has not recognized a clear instruction intent. In this case, a general semantic recognition result can be attached, and the interactive content of the two recognition results can be presented in a split-window display on the same screen, allowing the user to select the desired interactive content as needed. When both the custom recognition result and the general semantic recognition result contain a clear intent, since it is uncertain which recognition result better meets the user's expectations and preferences, the interactive content corresponding to the two recognition results can also be presented in a split-window display on the same screen to avoid misrecognition or situations that do not meet user requirements due to a single recognition result. In this way, a first sub-command is generated based on the custom recognition result and the target standard name, and a second sub-command is generated based on the general semantic recognition result. This allows the first voice interaction command to integrate the sub-commands of the two recognition results. The display device responds to the two sub-commands by displaying two interactive contents in a split-window display mode on the same screen, allowing the user to select as needed, thereby improving the quality of voice interaction and user satisfaction.

[0111] Figure 12 A schematic diagram of a display device for displaying voice interaction content provided in some embodiments of this application. Figure 4 .

[0112] In some embodiments, see Figure 12 If the custom recognition result output by Line 1 contains a complete instruction sentence, it indicates that Line 1 has defined a clear instruction intent (explicit intent / hard intent). If the general recognition result output by Line 2 contains a chatty intent or does not contain an explicit intent, it indicates that the general semantic understanding cannot provide effective interactive content or interactive content that matches the user's preferences. In this way, the server filters out / discards the general semantic recognition result to avoid interference from the general semantic recognition result to voice interaction.

[0113] In some embodiments, see Figure 12, the server filters out / discards the general semantic recognition results, and only generates a second voice interaction instruction based on the instruction sentence pattern and the target standard name, and transmits the second voice interaction instruction to the display device through the communication protocol. The display device responds to the second voice interaction instruction and displays the first interaction content corresponding to the custom recognition result in full-screen mode.

[0114] In some embodiments, for example, the voice text is "MOMO". The display device, based on the preset custom word library, recognizes that the voice text contains the target custom vocabulary "MOMO", the target standard name is "MOMO", and the target scene label is App. If Line 1 determines that the target scene label matches the explicit intention rule, the custom recognition result output by Line 1 includes the instruction sentence pattern "Open MOMO". Line 2, through semantic understanding and intention analysis of the voice text "MOMO", believes that "MOMO" is an expression of a chatting tone word, that is, it contains a chatting intention, and then discards the general semantic recognition result of Line 2. In this way, the server writes the instruction content (such as Open MOMO App) for instructing to open the MOMO App into the generated second voice interaction instruction. In this way, the display device responds to the second voice interaction instruction and controls the display to display the first interaction content in full-screen mode. The first interaction content is the dynamic screen for starting the Momo App, including the startup animation of the Momo App, the application home page, etc. In this way, the interaction content corresponding to the general semantic recognition result of the chatting type or without valid business intention is filtered out, and only the interaction content corresponding to the custom recognition result is displayed.

[0115] Figure 13 This is the timing signaling diagram of the voice interaction method provided by some embodiments of this application.

[0116] In some embodiments, refer to Figure 13 , the interaction objects of this timing signaling logic include a display device and a server. The display device runs a controller and a UI layer, and the server is configured with an ASR service, a service system, a custom scene recognition module, a general semantic understanding module, and an instruction decision module.

[0117] In some embodiments, refer to Figure 13 , the user can preset custom vocabulary, and the controller stores and maintains the preset custom word library. When the controller receives the voice data input by the user, it can send an ASR request to the server, and the ASR request includes the voice data. The ASR service of the server responds to the ASR request, converts the voice data into voice text, and sends the voice text to the display device.

[0118] In some embodiments, refer to Figure 13The controller identifies target custom words in the spoken text based on a preset custom dictionary. If the target custom word does not match the first word, the first word is replaced with the target custom word, thereby rewriting the spoken text.

[0119] In some embodiments, see Figure 13 The controller sends a speech recognition request to the server. This request includes information such as the rewritten speech text, target custom vocabulary, target standard names, and target scene tags. In response to this speech recognition request, the service system initiates parallel processing on Line 1 and Line 2.

[0120] In some embodiments, see Figure 13 The custom scene recognition module executes line one and outputs the custom recognition result to the service system. The general semantic understanding module executes line two and outputs the general semantic recognition result to the service system. The service system submits the custom recognition result and the general semantic recognition result to the instruction decision module, which then decides on the generation of voice interaction instructions.

[0121] In some embodiments, see Figure 13 When the custom recognition result contains only the target custom words (i.e. the instruction attribute is fuzzy / vogue), or when both the custom recognition result and the general semantic recognition result contain a clear intent, the instruction decision module generates a first voice interaction instruction and sends the first voice interaction instruction to the display device.

[0122] In some embodiments, see Figure 13 The controller responds to the first voice interaction command and controls the UI layer to display the first interactive content corresponding to the custom recognition result in the first window, and to display the second interactive content corresponding to the general semantic recognition result in the second window.

[0123] In some embodiments, see Figure 13 When the custom recognition result contains an instruction sentence (i.e., the instruction attribute is explicit / hard), and the general semantic recognition result contains a chatty intent or an invalid intent, the instruction decision module generates a second voice interaction instruction and sends the second voice interaction instruction to the display device.

[0124] In some embodiments, see Figure 13 The controller responds to the second voice interaction command and controls the UI layer to display the first interactive content corresponding to the custom recognition result in full-screen mode.

[0125] In some embodiments, see Figure 13 When users modify, add, or delete custom words, the controller synchronously updates the preset custom word library to ensure the recognition accuracy of subsequent Line 1.

[0126] In some embodiments, a computer storage medium is also provided, which may store a program. When the computer storage medium is configured in the display device 200, the program, when executed, may include the program steps involved in the voice interaction methods in the above embodiments. The computer storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

Claims

1. A display device, characterized in that, include: The display is configured to show interactive content; A voice acquisition device is configured to collect voice data input by the user; A communication device configured to communicate with a server; The controller is configured as follows: Upon receiving the voice data, obtain the corresponding voice text. Based on a preset custom lexicon and the voice text, the target custom words contained in the voice text, as well as the target standard names and target scene tags mapped by the target custom words are obtained; wherein, the preset custom lexicon includes the mapping relationship between standard names, custom words and scene tags, and the scene tags are used to mark the application scene to which the custom words belong; A speech recognition request is sent to the server; the speech recognition request includes the speech text, the target custom vocabulary, the target standard name, and the target scene tag; The system receives a voice interaction command from the server in response to the voice recognition request; the voice interaction command is generated by the server based on a custom recognition result, the target standard name, and a general semantic recognition result; the custom recognition result is the recognition result output by the server based on the target scene label and the target custom vocabulary, and the general semantic recognition result is the recognition result output by the server after performing semantic understanding on the voice text; In response to the voice interaction command, the display is controlled to show the interactive content indicated by the voice interaction command.

2. The display device according to claim 1, characterized in that, The controller, based on a preset custom lexicon and the voice text, obtains the target custom vocabulary contained in the voice text, specifically configured as follows: If the preset custom word library contains a first custom word whose similarity to the first word is greater than a threshold, then the first custom word is determined as the target custom word; wherein, the first word is a noun identified from the speech text.

3. The display device according to claim 2, characterized in that, Before sending a speech recognition request to the server, the controller is also configured to: The first word in the speech text is replaced with the target custom word to obtain the rewritten speech text; The speech recognition request is created based on the rewritten speech text, the target custom vocabulary, the target standard name, and the target scene label.

4. The display device according to claim 1, characterized in that, The controller, in response to the voice interaction command, controls the display to show the interactive content indicated by the voice interaction command, specifically configured as follows: If the voice interaction command is a first voice interaction command, the display is controlled to display the first interaction content corresponding to the custom recognition result in the first window, and the second interaction content corresponding to the general semantic recognition result in the second window; Wherein, the first voice interaction command is sent by the server when the custom recognition result contains only the target custom vocabulary, or the first interaction command is sent by the server when both the custom recognition result and the general semantic recognition result contain a clear intent, and the instruction action indicated by the clear intent can be effectively executed.

5. The display device according to claim 1, characterized in that, The controller, in response to the voice interaction command, controls the display to show the interactive content indicated by the voice interaction command, specifically configured as follows: If the voice interaction command is a second voice interaction command, control the display to display the first interaction content corresponding to the custom recognition result in full-screen mode; The second voice interaction command is sent by the server when the custom recognition result contains a clear intent, and the general semantic recognition result contains a casual conversation intent or does not contain a clear intent.

6. A server, characterized in that, include: A communicator used for communication connections with display devices; The processor is configured as follows: Receive a voice recognition request sent by the display device; The speech recognition request includes the speech text, the target custom vocabulary, and the target standard name and target scene label mapped to the target custom vocabulary; The target custom vocabulary refers to the custom vocabulary contained in the voice text, and the target scene tag is used to mark the application scene to which the target custom vocabulary belongs; Based on the target scene label and the target custom vocabulary, a custom recognition result is obtained; The speech text is semantically understood to obtain a general semantic recognition result; Based on the custom recognition result, the target standard name, and the general semantic recognition result, generate voice interaction commands; The voice interaction command is sent to the display device so that the display device displays the interactive content indicated by the voice interaction command.

7. The server according to claim 6, characterized in that, The processor, based on the target scene label and the target custom vocabulary, obtains a custom recognition result, specifically configured as follows: Determine the intent generation rules for matching the target scene label, wherein the intent generation rules include explicit intent rules and fuzzy intent rules; If the intent generation rule matching the target scene label is the fuzzy intent rule, then the output is the custom recognition result containing only the target custom vocabulary; If the intent generation rule matched by the target scene tag is the explicit intent rule, then the target action vocabulary associated with the target scene tag is obtained; The target custom vocabulary and the target action vocabulary are combined to form an instruction sentence containing a clear intent, and the custom recognition result containing the instruction sentence is output; wherein the instruction action indicated by the clear intent can be effectively executed.

8. The server according to claim 7, characterized in that, The processor generates voice interaction commands based on the custom recognition result, the target standard name, and the general semantic recognition result, specifically configured as follows: If the custom recognition result contains only the target custom vocabulary, or if both the custom recognition result and the general semantic recognition result contain a clear intent, then a first sub-command is generated based on the custom recognition result and the target standard name; Based on the general semantic recognition result, a second sub-command is generated; Based on the first sub-command and the second sub-command, a first voice interaction instruction is generated; wherein, the first interaction instruction is used to instruct the display device to respond to the first sub-command by displaying the first interaction content corresponding to the custom recognition result in a first window, and to respond to the second sub-command by displaying the second interaction content corresponding to the general semantic recognition result in a second window.

9. The server according to claim 7, characterized in that, The processor generates voice interaction commands based on the custom recognition result, the target standard name, and the general semantic recognition result, specifically configured as follows: If the custom recognition result contains a clear intent, and the general semantic recognition result contains a casual conversation intent or does not contain a clear intent, then a second voice interaction instruction is generated based on the instruction phrase and the target standard name; wherein, the second interaction instruction is used to instruct the display device to display the first interaction content corresponding to the custom recognition result in full-screen mode.

10. A voice interaction method, characterized in that, include: Upon receiving voice data, obtain the corresponding voice text. Based on a preset custom lexicon and the voice text, the target custom words contained in the voice text, as well as the target standard names and target scene tags mapped by the target custom words are obtained; wherein, the preset custom lexicon includes the mapping relationship between standard names, custom words and scene tags, and the scene tags are used to mark the application scene to which the custom words belong; A speech recognition request is sent to the server; the speech recognition request includes the speech text, the target custom vocabulary, the target standard name, and the target scene tag; The system receives a voice interaction command from the server in response to the voice recognition request; the voice interaction command is generated by the server based on a custom recognition result, the target standard name, and a general semantic recognition result; the custom recognition result is the recognition result output by the server based on the target scene label and the target custom vocabulary, and the general semantic recognition result is the recognition result output by the server after performing semantic understanding on the voice text; In response to the voice interaction command, the interactive content indicated by the voice interaction command is displayed.

Images