A display device, a server, and a voice command execution method

Abstract

The application provides a display device, a server and a voice instruction execution method, which comprises the following steps: receiving voice interaction data input by a user; generating a corresponding instruction set according to the voice interaction data; obtaining a preset instruction directed acyclic graph, wherein the preset instruction directed acyclic graph comprises multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions comprise first instructions; determining a target path composed of at least one first instruction according to the preset instruction directed acyclic graph; determining a target execution order between the at least one first instruction according to the target path; and executing the corresponding first instructions according to the target execution order. This process can receive voice interaction data of a user, analyze and determine a corresponding instruction set, then find a target path in a preset instruction directed acyclic graph, determine the execution order of the instructions, and execute the instruction set according to the order, thereby improving the interactive experience of the user and ensuring the correctness and reliability of the instructions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice understanding, and in particular to a display device, a server and a voice instruction execution method. BACKGROUND

[0002] At present, a display device can extract multiple intents from a voice instruction of a user by using voice recognition and intent understanding technology, generate an instruction set corresponding to each intent, and execute actions corresponding to the instructions in the instruction set according to a generation order of the instructions in the instruction set.

[0003] However, the generation order of the instructions in the instruction set can be different from an order required to obtain a result expected by the user, resulting in a result obtained by the display device by executing actions corresponding to the instructions according to the generation order being inconsistent with the result expected by the user, and affecting the experience of the user. SUMMARY

[0004] The present application provides a display device, a server and a voice instruction execution method, which can determine a correct execution order of instructions in an instruction set according to a preset instruction directed acyclic graph, improve the interactive experience of the user, and ensure the correctness and reliability of the instructions.

[0005] In a first aspect, an embodiment of the present application provides a display device, comprising: a display configured to display a user interface; a controller coupled to the display and configured to: receive voice interaction data input by a user; generate a corresponding instruction set according to the voice interaction data; the instruction set comprising at least one first instruction; obtain a preset instruction directed acyclic graph, the preset instruction directed acyclic graph comprising a plurality of paths, the paths representing an execution order of preset instructions; the preset instructions comprising the first instructions; determine a target path composed of the at least one first instruction according to the preset instruction directed acyclic graph; determine a target execution order between the at least one first instruction according to the target path; and execute corresponding first instructions according to the target execution order.

[0006] In the technical solution provided by the present application, the controller receives voice interaction data input by a user, generates a corresponding instruction set according to the voice interaction data, the instruction set comprising at least one first instruction, obtains a preset instruction directed acyclic graph, the preset instruction directed acyclic graph comprising a plurality of paths, the paths representing an execution order of preset instructions, the preset instructions comprising the first instructions, determines a target path composed of the at least one first instruction according to the preset instruction directed acyclic graph, determines a target execution order between the at least one first instruction according to the target path, and executes corresponding first instructions according to the target execution order. This process can receive voice interaction data of a user, parse and determine a corresponding instruction set, then find a target path in a preset instruction directed acyclic graph, determine an execution order of instructions, and execute the instruction set according to the order, improving the interactive experience of the user and ensuring the correctness and reliability of the instructions.

[0007] In some possible embodiments, the controller performs determining the target path according to the preset instruction directed acyclic graph, and the determining is specifically configured to: for each first instruction in the instruction set, traversing the preset instruction directed acyclic graph, selecting all first paths including the first node matched with the at least one first instruction; if there are multiple first paths, performing a de-duplication and a merging equivalent path operation on the multiple first paths to obtain second paths; if there are multiple second paths, determining a number of hit rules of each second path according to the number of first nodes included in the second path; sorting the multiple second paths based on the number of hit rules of each second path; and determining the target path according to the second path with the maximum number of hit rules. According to the above embodiments, the de-duplication and the merging of the found first paths can reduce redundant paths and improve processing efficiency. The sorting of the second paths according to the number of hit rules in the sorting result can determine the path including all first instructions in the instruction set, so that the accuracy and reliability of the execution of the first instructions can be improved.

[0008] In some possible embodiments, the controller performs determining the target path according to the second path with the maximum number of hit rules, and the determining is specifically configured to: determining whether the second path with the maximum number of hit rules includes a target node; the target node is a node matched with a second instruction executed before the execution of the first instruction; in the case that the second path includes at least one target node, adding the second instruction corresponding to the target node to the instruction set and taking the second path as a candidate path; in the case that the second path does not include any target node, taking the second path as a candidate path; and determining the target path from all candidate paths.

[0009] According to the above embodiments, the instruction set can be completed according to whether there is a target node in the second path, so that the correctness of the instruction execution can be ensured.

[0010] In some possible embodiments, the controller performs determining whether the second path with the maximum number of hit rules includes a target node, and the determining is specifically configured to: determining whether the second path includes at least one second node; the second node is a previous node of each first node in the second path, and the previous node is different from any first node; in the case that the second path includes any second node, determining whether the second node is in a first state; the first state indicates that the instruction corresponding to the second node is executed before the execution of the first instruction; and when the second node is in the first state, determining that the second node is the target node.

[0011] According to the above embodiments, it can be determined whether the second path includes the second node, and it can be determined whether the second path includes the target node of the first node by determining the first state of the second node, so that the execution order of the instructions can be ensured to comply with all necessary dependency relationships, thereby avoiding execution failure caused by incorrect dependency relationships.

[0012] In some possible embodiments, the controller is configured to determine the target path from all candidate paths, and specifically configured to: if there are multiple candidate paths, obtain user historical access data of each node in each candidate path; calculate a total historical access value of the corresponding candidate path based on the user historical access data of each node; determine the candidate path with the largest total historical access value from the multiple candidate paths as the target path; or randomly select one candidate path from the multiple candidate paths as the target path. According to the above embodiments, the path with the largest total historical access value can be selected to better match the user's usage habits and improve user satisfaction. The target path can also be randomly selected, without the need for additional historical data and complex calculations, and each candidate path has an equal probability of being selected, ensuring the fairness of the selection.

[0013] In some possible embodiments, the controller is configured to determine the target path composed of at least one first instruction according to the preset instruction directed acyclic graph, and specifically configured to: in the case that the target path includes the target node, determine the target execution order of the first instruction and the second instruction according to the node order on the target path.

[0014] According to the above embodiments, the controller can ensure that all necessary dependencies are correctly handled when executing the instruction set, thereby ensuring the smoothness and effectiveness of the entire operation process.

[0015] In some possible embodiments, the controller is further configured to: obtain historical user instructions; map each historical user instruction to a node in the directed acyclic graph, and take the instruction type corresponding to the historical user instruction as a node label; take the execution value of the historical user instruction, the terminal scene after the current historical user instruction is executed, and the node state as node attributes of the directed acyclic graph; create a directed edge between the nodes corresponding to the historical user instructions according to the execution order of the historical user instructions; and construct the preset instruction directed acyclic graph according to the node label, the node attributes, and the directed edge. According to the above embodiments, the preset instruction directed acyclic graph can be established based on the historical user instructions, and the preset instruction directed acyclic graph can clearly indicate the dependencies between the preset instructions, ensuring that the instructions are executed in the correct order and improving the execution efficiency of the preset instructions.

[0016] In some possible embodiments, the controller is configured to determine the corresponding instruction set according to the voice interaction data, and specifically configured to: convert the voice interaction data into text data based on voice recognition technology; analyze the text data by using natural language processing technology to obtain user intent and entity information; find the first instruction corresponding to the user intent and the entity information in a preset rule library based on the user intent and the entity information; the preset rule library is used to record the mapping relationship between the user intent and the first instruction; and determine the instruction set based on the first instruction.

[0017] According to the above-mentioned embodiments, the speech interaction data is processed by using the speech recognition technology and the natural language processing technology to obtain the text data, the user intent and the entity information, which can improve the accuracy of the display device in recognizing and understanding the user intent. Further, the instructions corresponding to different intents and entity information are recorded in the preset rule library, and by searching the rule library, it can be ensured that the generated first instructions are highly consistent with the actual needs of the user, which reduces the possibility of misoperation, and at the same time, the accuracy and response speed of the display device in executing corresponding actions can be improved.

[0018] In a second aspect, the embodiments of the present application provide a display device, comprising: a display; a communicator configured to be communicatively connected with a server; a controller coupled with the display and the communicator, and configured to: receive speech interaction data input by a user; control the communicator to send the speech interaction data to the server, so that the server determines a corresponding instruction set according to the speech interaction data, and determines a target execution order between first instructions in the instruction set based on a preset instruction directed acyclic graph; wherein the preset instruction directed acyclic graph comprises a plurality of paths, and the paths represent execution orders of preset instructions; the preset instructions comprise the first instructions; control the communicator to receive the instruction set and the corresponding target execution order from the server; and execute the corresponding first instructions according to the target execution order.

[0019] In a third aspect, the embodiments of the present application provide a server, comprising: a communicator configured to be communicatively connected with a display device; a processor coupled with the communicator, and configured to: control the communicator to receive speech interaction data from the display device, and generate a corresponding instruction set according to the speech interaction data; the instruction set comprises at least one first instruction; obtain a preset instruction directed acyclic graph, wherein the preset instruction directed acyclic graph comprises a plurality of paths, and the paths represent execution orders of preset instructions; the preset instructions comprise the first instructions; determine a target path composed of the at least one first instruction according to the preset instruction directed acyclic graph; determine a target execution order between the at least one first instruction according to the target path; and control the communicator to send the instruction set and the target execution order to the display device, so that the display device executes the corresponding first instructions according to the target.

[0020] In a fourth aspect, the embodiments of the present application provide an execution method of a voice instruction, which is applied to a display device, and comprises: receiving speech interaction data input by a user; generating a corresponding instruction set according to the speech interaction data; the instruction set comprises at least one first instruction; obtaining a preset instruction directed acyclic graph, wherein the preset instruction directed acyclic graph comprises a plurality of paths, and the paths represent execution orders of preset instructions; the preset instructions comprise the first instructions; determining a target path composed of the at least one first instruction according to the preset instruction directed acyclic graph; determining a target execution order between the at least one first instruction according to the target path; and executing the corresponding first instructions according to the target execution order.

[0021] It can be understood that the beneficial effects achieved by the technical solutions provided in the second aspect to the fourth aspect can refer to the beneficial effects in the first aspect and any optional implementation manner thereof, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0023] Figure 1 A schematic diagram of an operation scenario between a display device and a control device is provided for the embodiments of the present application.

[0024] Figure 2 A schematic diagram of a hardware configuration of a display device is provided for the embodiments of the present application.

[0025] Figure 3 A schematic diagram of a software configuration of a display device is provided for the embodiments of the present application.

[0026] Figure 4 A flowchart of determining an execution order of a first instruction in an instruction set according to a preset directed acyclic graph for a display device is provided for the embodiments of the present application.

[0027] Figure 5 A flowchart of generating an instruction set for a display device is provided for the embodiments of the present application.

[0028] Figure 6 A flowchart of constructing a preset instruction directed acyclic graph for a display device is provided for the embodiments of the present application.

[0029] Figure 7 A structural diagram of a preset instruction directed acyclic graph is provided for the embodiments of the present application.

[0030] Figure 8 A flowchart of searching for a target path according to an instruction set for a display device is provided for the embodiments of the present application.

[0031] Figure 9 A flowchart of determining a target path for a display device is provided for the embodiments of the present application.

[0032] Figure 10 A flowchart of communicating with a server to determine an instruction execution order and an instruction set according to voice interaction data is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0033] The embodiments will be described in detail below with reference to examples thereof as illustrated in the accompanying drawings. When the description below refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise described. The implementations described in the following embodiments are not meant to represent all implementations consistent with the present disclosure. Rather, they are merely examples of systems and methods consistent with some aspects of the present disclosure as detailed in the claims.

[0034] It should be noted that the brief description of terms in the present disclosure is only for the convenience of understanding the implementations described next, and is not intended to limit the implementations of the present disclosure. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

[0035] The terms "first", "second", "third", and the like in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar or like objects or entities, and do not necessarily mean a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchanged under appropriate circumstances.

[0036] The terms "include" and "have" and any variations thereof are intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not have to be limited to all components clearly listed, but can include other components not clearly listed or inherent to these products or devices. The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code capable of performing functions related to the element.

[0037] The display device 200 provided by the embodiments of the present disclosure generally refers to a device having the ability of picture display and data processing, which can have various implementation forms, for example, can be a television, a smart television, a computer, a laser projection device, a monitor, an electronic bulletin board, an electronic table, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0038] Figure 1 The schematic diagram of the operation scenario between the display device 200 and the control device 100 provided by the embodiments of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the user can operate the display device 200 through the terminal device 300 or the control device 100. Figure 1

[0039] ​In some embodiments, the control device 100 is used to receive user input operation instructions and convert the operation instructions into control instructions that the display device 200 can recognize and respond to. The control device 100 can be a remote controller, a stylus, a handle, etc., wherein the communication between the remote controller and the display device 200 includes infrared protocol communication or Bluetooth protocol communication, and other short-distance communication methods, to control the display device 200 in a wireless or wired manner. The user can input user instructions through buttons on the remote controller, voice input, control panel input, etc., to control the display device 200.

[0040] In some embodiments, the terminal device 300 (such as a mobile terminal, a tablet computer, a computer, a notebook computer, etc.) can also be used as a control device to perform human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device to establish a communication connection with the display device 200 and perform data interaction. The mobile terminal 300 can install the same software application as the display device 200, and then realize connection communication through a network communication protocol, to achieve the purpose of one-to-one control operation and data communication. The mobile terminal 300 can also display audio and video content on the display device 200 to realize synchronous display function.

[0041] In some embodiments, the mobile terminal 300 or other electronic device can also simulate the function of the control device 100 by running an application program for controlling the display device 200.

[0042] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the terminal device 300, for example, the display device 200 can directly receive user voice instructions through a voice instruction acquisition module configured inside the display device 200, or the display device 200 can receive user voice instructions through a voice control device arranged outside the display device 200. The display device 200 can also receive user control through touch or gesture, etc.

[0043] In some embodiments, the display device 200 also communicates with the server 400 through various communication methods. The display device 200 can be allowed to communicate through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various content and interaction to the display device 200. The server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0044] The display device 200 can provide a broadcast receiving television function, and can additionally provide a smart network television function with computer support, including but not limited to a network television, a smart television, an Internet protocol television (IPTV), etc.

[0045] Figure 2 A hardware configuration block diagram of a control device 100 according to some embodiments is shown. For example... Figure 2 As shown, the control device 100 includes a controller 250, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0046] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0047] like Figure 2 As shown, the display device 200 may include at least one of the following: modem 210, communication device 220, detector 230, external device interface 240, controller 250, display 260, audio output interface 270, memory, power supply, and user interface.

[0048] In some embodiments, modem 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals.

[0049] In some embodiments, the communication device 220 is a component used to communicate with external devices or servers according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connection communication, it needs to have a communication device 220 with Bluetooth functionality. The communication device 220 may include at least one of a WiFi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communication device 220.

[0050] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0051] In some embodiments, the detector 230 is configured to collect signals of the external environment or the external interaction. For example, the detector 230 includes a light receiver configured to collect an ambient light intensity; or the detector 230 includes an image collector, such as a camera, configured to collect an external environment scene, a user attribute, or a user interaction gesture; or the detector 230 includes a sound collector, such as a microphone, configured to receive an external sound.

[0052] In some embodiments, the external device interface 240 can include, but is not limited to, any one or more of a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. The external device interface 240 can also be a composite input / output interface formed by a plurality of the above interfaces.

[0053] In some embodiments, the controller 250 controls the operation of the display device 200 and the response to the user's operation by storing various software control programs in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object displayed on the display 260, the controller 250 can perform an operation related to the object selected by the user command.

[0054] In some embodiments, the controller 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (random access memory), a ROM (read-only memory), a first interface to an n-th interface for input / output, a communication bus, etc. The controller 250 controls the operation of the display device and the response to the user's operation by storing various software control programs in the memory. The controller 250 controls the overall operation of the display device 200.

[0055] In some embodiments, the controller 250 and the modem 210 can be located in different split devices, i.e., the modem 210 can also be in a peripheral device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0056] In some embodiments, the user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

[0057] In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving the display of the picture. The display 260 is configured to receive an image signal output from the controller 250 for display. For example, the display 260 can be configured to display video content, image content, and components of a menu control interface, as well as a user control UI interface, etc.

[0058] In some embodiments, the "user interface" is a medium interface for interaction and information exchange between an application program or an operating system and a user, which realizes the conversion between the internal form of information and the form acceptable by the user. The commonly used form of the user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an icon, window, control, etc. interface element displayed in the display screen of an electronic device, wherein the control can include an icon, button, menu, tab, text box, dialog box, status bar, navigation bar, Widget, etc. visual interface element.

[0059] In some embodiments, the display 260 can be a liquid crystal display, an OLED display, and a projection display, and can also be a projection device and a projection screen.

[0060] In some embodiments, the audio output device can be a native loudspeaker of the display device 200, or can be an audio output device connected to the display device 200. For the audio output device connected to the display device 200, the display device 200 can also be provided with an external audio output terminal, and the audio output device can be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0061] In some embodiments, the user interface can be configured to receive a control signal of the control device 100 (such as an infrared remote controller, etc.).

[0062] In some embodiments, the display device can include a microphone array for obtaining the user's voice. In the process of receiving the user's voice data, the microphone array can collect the user's voice interaction data and convert the voice interaction data into an electrical signal, so that the display device can accurately understand and respond.

[0063] To perform user interactions, in some embodiments, the display device 200 can run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources of the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0064] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0065] The operating system can be divided into different modules or levels according to the functions implemented, for example, as shown in FIG. 1, in some embodiments, the system is divided into four layers, from top to bottom, the Applications layer (referred to as the "application layer"), the Application Framework layer (referred to as the "framework layer"), the system library layer, and the kernel layer. Figure 3

[0066] In some embodiments, the application layer is used to provide services and interfaces for applications, so that the display device 200 can run the application and interact with the user based on the application. At least one application can be run in the application layer, which can be a window (Window) program, a system setting program or a clock program provided by the operating system, or an application developed by a third-party developer. In specific implementation, the application package in the application layer is not limited to the above examples.

[0067] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some pre-defined functions. The application framework layer is equivalent to a processing center that decides which application in the application layer to act. The application can access the resources in the system and obtain the services of the system through the API interface during execution.

[0068] Figure 3 The software configuration schematic diagram of the display device 200 provided by the embodiments of the present application is shown in FIG. 1.

[0069] As shown in FIG. 1, the software configuration of the display device 200 includes the following layers: the application layer, the framework layer, the system library layer, and the kernel layer. Figure 3 ​As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0070] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0071] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0072] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 3 As shown, hardware drivers can be configured in the kernel layer. The kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0073] It should be noted that the above examples are only a simple division of the operating system functions, and do not constitute a limitation on the specific operating system form of the display device 200 in the embodiments of the present application. According to the function of the display device, the type of operating system, and other factors, the number and specific level types of the levels contained in the operating system can be in other forms.

[0074] With the rapid development of digitalization and intelligentization technology, display devices with voice interaction function have become an important part of modern work and life. Through natural language processing technology, display devices can understand and respond to users' voice commands, greatly improving the convenience and efficiency of human-computer interaction. Users can complete various operations such as playing music, watching videos, adjusting device settings, etc. through simple voice instructions. For example, a user can say to the display device: "play XXX songs in sequence", "play Wolf Warrior 2 in full screen", "play songs in sequence", etc. voice instructions, and the display device can recognize the voice instructions and execute the actions corresponding to the voice instructions.

[0075] Currently, display devices can use voice recognition and intent understanding technology to extract multiple intents from users' voice instructions, generate execution instructions corresponding to each intent, and execute corresponding actions in the order of generating execution instructions. However, display devices execute corresponding actions according to the recognized execution order, which may result in the effect achieved by display devices executing instructions in sequence not matching the actual effect the user wants to achieve. For example, for the instruction "play XXX songs in sequence", the display device can successfully recognize two execution instructions: one is to set the play mode to sequence play, and the other is to query and play XXX songs. The display device will first set the play mode to sequence play, and then query XXX songs according to the order of the above instructions. However, the user's actual expectation is to first find the list of XXX songs, and then set it to sequence play. This difference in execution order of execution instructions not only affects the user experience, but also may cause some unnecessary errors or confusion.

[0076] Secondly, existing technologies also face challenges when processing context-dependent instructions. For example, when a user says "play songs in sequence", the system will usually recognize the instruction to set the play mode to sequence play. However, if the user is not currently in a song playing scenario, this instruction will not be executed, causing the user to feel confused. In this case, the user's real intention may be to first start song playing, and then set the play mode to sequence play. Existing technologies often lack sufficient understanding of the user's current context, and thus cannot accurately judge and execute the user's actual needs.

[0077] In view of the above problems, the embodiments of the present application provide a display device, which can determine a corresponding instruction set based on voice interaction data of a user, determine an execution order of instructions in the instruction set by using a preset directed acyclic graph of instructions, and then execute the instructions according to the execution order.

[0078] In some embodiments, Figure 4 The flowchart for determining the execution order of the first instruction in the instruction set according to the preset directed acyclic graph is provided for the display device of the embodiments of the present application. As shown in Figure 4 The display device 200 provided by the embodiments of the present application includes a display and a controller. The controller 250 is coupled to the display 260 and is configured to perform the following program steps.

[0079] S100: Receive voice interaction data input by a user.

[0080] In some embodiments, the display device can perform corresponding actions based on the voice interaction data input by the user. For example, the display device can capture the audio signal of the user's voice input, i.e., the voice interaction data, through the built-in microphone array, process the voice interaction data, and implement corresponding functions or services. This process can be applied to multiple application scenarios such as information query, entertainment function, personal assistant service, and health management. For example, the user can control the display device to query the weather of the day through a simple voice instruction, or control the display device to play music, video, or start an entertainment project such as a game.

[0081] S200: Generate a corresponding instruction set according to the voice interaction data.

[0082] The controller can analyze the voice interaction data input by the user, identify the user's intention, and convert the user's intention into an understandable and executable instruction set, wherein the instruction set includes at least one first instruction.

[0083] In order to generate the instruction set, in some embodiments, the controller can convert the voice interaction data into text data based on voice recognition technology, analyze the text data by using natural language processing technology, obtain the user's intention and entity information, then find the first instruction corresponding to the user's intention and entity information in the preset rule library based on the user's intention and entity information, and finally determine the instruction set based on the first instruction.

[0084] Exemplarily, after capturing the voice interaction data of the user through the microphone array, the display device performs preprocessing such as noise reduction and echo cancellation on the voice interaction data to improve the accuracy of subsequent voice recognition. Then, the audio signal of the voice interaction data can be converted into text data by using a deep learning model. Subsequently, the text data is analyzed by using natural language processing technology, such as word segmentation, part-of-speech tagging, named entity recognition, and syntax analysis, to extract the user intent and entity information in the text data. Further, a first instruction matching the user intent and entity information is searched for in a preset rule library, and the first instruction can be a specific operation instruction or an API request for calling other services. Here, the preset rule library is a database recording the mapping relationship between the user intent and the first instruction. Finally, the instruction set is constructed according to the first instruction searched for in the preset rule library.

[0085] Exemplarily, when the controller receives the voice interaction data input by the user as "sequentially playing songs of XXX", the controller converts the voice interaction data into text data "sequentially playing songs of XXX", and performs intent recognition and entity recognition on sequentially playing songs of XXX (a certain singer) to obtain the user intent as sequentially playing songs and the entity information as XXX. Further, the controller determines two first instructions in the preset rule library based on the user intent and the entity information, which are sequentially playing songs and playing songs of XXX, respectively, and constructs the instruction set according to the two first instructions.

[0086] In this way, the voice interaction data is processed by using voice recognition technology and natural language processing technology to obtain text data, user intent, and entity information, which can improve the accuracy of the display device in recognizing and understanding the user intent. Further, the instructions corresponding to different intents and entity information are recorded in the preset rule library, and the first instruction generated by searching for the rule library can be highly consistent with the actual demand of the user, reducing the possibility of misoperation, and at the same time, the accuracy and response speed of the display device in performing corresponding actions can be improved.

[0087] In some embodiments, Figure 5 A flowchart of the display device generating an instruction set is provided for the embodiments of the present application. As shown in Figure 5 The display device receives an audio signal input by the user through a microphone array, and sends the audio signal to a controller. The controller converts the audio signal into voice interaction data, and sends the voice interaction data to a voice recognition module. The voice recognition module converts the voice interaction data into text data, and sends the text data to a semantic understanding module. The semantic understanding module performs intent recognition and entity recognition on the text data to obtain user intent and entity information, and sends the user intent and the entity information to an instruction generation module. The instruction generation module searches for a corresponding first instruction in a preset rule library according to the user intent and the entity information, and constructs an instruction set, and returns the instruction set to the controller.

[0088] S300: Obtain a preset instruction directed acyclic graph.

[0089] In some embodiments, the preset instruction directed acyclic graph includes a plurality of paths, and each path represents an execution order of preset instructions, and the preset instructions include a first instruction. In the preset instruction directed acyclic graph, each node represents a preset instruction, and a directed edge represents a dependency relationship between preset instructions, and the execution order between preset instructions can be determined through the preset instruction directed acyclic graph.

[0090] In some embodiments, Figure 6 The display device provided in the embodiments of the present application provides a flowchart for constructing a preset instruction directed acyclic graph. As shown in Figure 6 The controller constructing the preset instruction directed acyclic graph can be configured to perform the following steps:

[0091] S601: Obtain historical user instructions, and map each historical user instruction to a node in a directed acyclic graph.

[0092] In some embodiments, the historical user instructions can include a plurality of types based on application scenarios, such as application control (such as opening / closing an application, switching an application, etc.), playback control (such as playing / pausing / stopping media playback, jumping to a specified time point, switching a playback mode, etc.), device control (turning on / off a device, switching a network connection, etc.), and the like, but are not limited thereto. When selecting historical user instructions, instructions with execution orders are preferentially selected, such as playing a song, sequentially playing a song, etc. Instructions without execution orders, such as increasing the volume, turning off the device, etc., do not need to be added.

[0093] Illustratively, the controller can obtain historical user instructions, and each historical user instruction includes an instruction type, an instruction value, a terminal scene after execution of the instruction, and a node state. The instruction type can be determined according to the application scenario, the execution value is the instruction text, i.e., the original instruction input by the user, the terminal scene is the state or scene of the terminal device after execution, and the node state is whether the current instruction has a strong dependency relationship with the next instruction. Here, the node state is related to the execution order between the historical user instructions, and the node state of the node can be determined according to whether the node is a necessary prerequisite for execution of the next node. Illustratively, for historical user instructions being two instructions of opening a music application and sequentially playing a song, the correct execution order is to open the application first and then sequentially play the song. For this reason, for the instruction of opening the application, it is a necessary prerequisite for the next node of sequentially playing the song to execute, and therefore, the node state of the node corresponding to the instruction of opening the application is set to the first state.

[0094] S602: Take the instruction type corresponding to the historical user instruction as a label of the node.

[0095] The instruction type can be determined according to the application scenario. Each historical user instruction is mapped to a node in the directed acyclic graph, and the label of the node is the instruction type, the field name is label, and the field type is a string.

[0096] S603: The execution value of the historical user instruction, the terminal scene after execution of the current historical user instruction, and the node state are taken as node attributes of the directed acyclic graph.

[0097] The execution value of the historical user instruction, the terminal scene after execution, and the node state of the instruction are taken as attributes of the node, wherein the field name of the instruction value is command_param, the field type is a Json structure, the field name of the terminal scene after execution is executed_state, the character type is a string, and the field name of the node state is necessary, the character type is a Boolean value.

[0098] S604: According to the execution order of the historical user instruction, a directed edge is created between the nodes corresponding to the historical user instruction.

[0099] According to the execution order between the historical user instructions, a directed edge is created between the corresponding nodes, so that the directed edge can be used to represent that the instruction corresponding to the current node depends on the execution result of the instruction corresponding to the previous node.

[0100] S605: The preset instruction directed acyclic graph is constructed according to the node label, the node attribute, and the directed edge.

[0101] After the nodes, node attributes, and directed edges in the directed acyclic graph are determined, the directed acyclic graph is created according to the nodes and the directed edges between the nodes.

[0102] After the directed acyclic graph is constructed, it can be checked whether the preset instruction directed acyclic graph is acyclic. The first way can determine whether the preset instruction directed acyclic graph is acyclic based on topological sorting. Illustratively, an in-degree table can be created to record the in-degree of each node in the preset instruction directed acyclic graph, that is, how many edges point to the node. Then, all nodes with an in-degree of 0 are added to a queue. The nodes in the queue can then be processed, illustratively, a node is taken out of the queue, and the in-degree of all adjacent nodes of the node is reduced by 1. If the in-degree of a certain adjacent node becomes 0, the node is added to the queue. Finally, when all nodes have been processed, the directed acyclic graph is acyclic.

[0103] The second approach determines whether a predefined directed acyclic graph (DAG) is acyclic based on a depth-first search. For example, a color table can be created to record the access status of each node (e.g., white indicates unvisited, gray indicates being visited, and black indicates visited). A depth-first search is then performed starting from each unvisited node. If a gray node is encountered during the traversal, a cycle exists; if no gray node is encountered throughout the traversal, the predefined directed acyclic graph is acyclic.

[0104] Figure 7 This application provides a schematic diagram of the structure of a directed acyclic graph of preset instructions, as shown in the embodiments of this application. Figure 7 As shown, the preset instruction directed acyclic graph includes nodes 701-707, where node 701 is the head node, and nodes 702-707 each include node labels and node data. The node label represents the instruction type, and the node attributes include the instruction's execution value, the terminal scenario after the current historical user instruction execution, and the node status. Figure 7 In the diagram, node 702 is labeled "Application Control," its execution value is "Open Music Application," and after execution, the terminal device's state is "Music," and the node state is "First State." Node 703 is labeled "Music Playback Control," its execution value is "Sequential Playback," and after execution, the terminal device's state is "Music," and the node state is "First State." Node 704 is labeled "Application Control," its execution value is "Open First Music Application," and after execution, the terminal device's state is "Music," but the node state is not "First State." Node 705 is labeled "Music Priority Search," and after execution, the terminal device's state is "Music," and the node state is "First State." Node 706 is labeled "Music Playback Control," its execution value is "Sequential Playback," and after execution, the terminal device's state is "Music," and the node state is "First State." Node 707 is labeled "Application Control," its execution value is "Open Second Music Application," and after execution, the terminal device's state is "Music," and the node state is "First State."

[0105] In this way, a directed acyclic graph of preset instructions can be built using historical user instructions. The directed acyclic graph of preset instructions can clearly indicate the dependencies between preset instructions, ensuring that instructions are executed in the correct order and improving the execution efficiency of preset instructions.

[0106] S400: Determine the target path consisting of at least one first instruction based on the preset directed acyclic graph of instructions.

[0107] In order to determine the correct execution order of the first instruction in the instruction set based on a preset directed acyclic graph of instructions, in some embodiments, Figure 8 A flowchart illustrating the process of determining a target path consisting of at least one first instruction for a display device provided in this application embodiment is shown below. Figure 8As shown, the controller can be configured to perform the following steps:

[0108] S801: For each first instruction in the instruction set, traverse the preset instruction directed acyclic graph to select all first paths including the first node matched with the at least one first instruction.

[0109] In some embodiments, for each first instruction in the instruction set, the controller can traverse the preset instruction directed acyclic graph by depth-first search or breadth-first search to find all first paths containing the first node corresponding to the first instruction.

[0110] Illustratively, when the instruction set includes two first instructions, for the first first instruction, traverse the preset instruction directed acyclic graph to find all first paths including the first node corresponding to the instruction. For the second first instruction, traverse the preset instruction directed acyclic graph to find all first paths including the first node corresponding to the instruction.

[0111] S802: If there are multiple first paths, perform deduplication and equivalent path merging operations on the multiple first paths to obtain second paths.

[0112] In some embodiments, when there are multiple first paths, perform deduplication processing on all collected first paths to remove duplicate paths, ensure that each first path is unique, and remove first paths that are completely identical in node order and attributes to obtain second paths. The number of second paths can be multiple.

[0113] Illustratively, taking the example of an instruction set including two first instructions, perform deduplication and equivalent path merging on the first path containing the first first instruction and the first path containing the second first instruction to obtain second paths.

[0114] S803: If there are multiple second paths, determine the number of hit rules of the second paths according to the number of first nodes included in the second paths.

[0115] In some embodiments, when there are multiple second paths, the number of first nodes included in each second path can be used as the number of hit rules of each second path. Illustratively, taking the example of an instruction set including two first instructions, the second paths include 3, and the number of first nodes included in each second path is 2, 2, and 1. The number of hit rules corresponding to each second path is 2, 2, and 1, respectively.

[0116] S804: Sort the multiple second paths based on the number of hit rules of each second path.

[0117] In some embodiments, the controller can sort the multiple second paths in descending order according to the number of hit rules.

[0118] Exemplarily, the number of hit rules corresponding to each second path is respectively 2, 2 and 1, and the second paths are sorted according to the number of hit rules.

[0119] S805: determining the target path according to the second path with the largest number of hit rules.

[0120] In some embodiments, the path with the largest number of hit rules is determined as the target path from the sorting result, and the target path represents the correct execution order of the first instructions in the instruction set.

[0121] Exemplarily, when the instruction set of the second path includes two first instructions, and there are two second paths with the number of hit rules being 2, the target path can be determined according to the two second paths with the number of hit rules being 2.

[0122] In this way, the de-duplication and merging of the found first paths can reduce redundant paths and improve processing efficiency. The sorting of the second paths according to the number of hit rules can determine the path including all the first instructions in the instruction set, so that the accuracy and reliability of the execution of the first instructions can be improved.

[0123] In some embodiments, Figure 9 A flowchart for determining the target path of the display device provided in the embodiments of the present application is shown in FIG. 9, and the controller performing the step of determining the target path according to the second path with the largest number of hit rules can be configured to perform the following steps: Figure 9

[0124] S901: determining whether the second path with the largest number of hit rules includes the target node.

[0125] In some embodiments, the target node is a node matched with the second instruction executed before the execution of the first instruction, i.e., a strong dependency node. In the process of determining the target path according to the second path with the largest number of hit rules, the target path can be determined according to whether the first node corresponding to the first instruction in the instruction set is included in the target node in the second path with the largest number of hit rules. The target node is a node before the first node in the second path, and is different from any first node, and the state of the target node is a first state (e.g., True), which represents that the target node is the target node of the next node. That is, the target node is a node that does not exist in the instruction set and has a necessary execution order with the first node.

[0126] ​Exemplarily, the controller traverses each node in the second path, checks whether the previous node of each first node therein is the second node, and whether the instruction corresponding to the second node does not exist in the instruction set. That is, there is a directed edge between the second node and the first node, and the controller can execute the instruction corresponding to the second node before executing the first instruction corresponding to the first node. Further, if the second path includes the second node, the controller obtains the node state of the second node. If the node state is the first state, it is determined that the node is the target node. In the case where the second node is the target node, the controller must execute the instruction corresponding to the second node before executing the first instruction corresponding to the first node.

[0127] In view of this, in some embodiments, the controller performing the determination of whether the second path with the maximum hit rule data includes the target node can be configured to: determine whether the second path includes at least one second node, the second node being the previous node of each first node in the second path and being different from any first node; in the case where the second path includes any second node, determine whether the second node is in the first state; the first state indicating that the instruction corresponding to the second node is executed before the first instruction; and when the second node is in the first state, determine that the second node is the target node.

[0128] According to the above embodiments, whether the second path includes the target node of the first node can be determined by judging whether the second path includes the second node and whether the state of the second node is the first state, so that the execution order of the instructions can be ensured to comply with all necessary dependency relationships, thereby avoiding execution failure due to incorrect dependency relationships.

[0129] S902: In the case where the second path includes at least one target node, adding the second instruction corresponding to the target node to the instruction set and taking the second path as a candidate path.

[0130] Exemplarily, when the second path includes at least one target node, it indicates that the execution of the first instruction in the second path depends on the execution result of the target node, that is, the controller needs to execute the second instruction corresponding to the target node before executing the first instruction corresponding to the first node, and then the second instruction needs to be added to the instruction set to complete the instructions in the instruction set and ensure the integrity of the execution of the instructions in the instruction set. After the second instruction is added to the instruction set, the second path is taken as a candidate path.

[0131] S903: In the case where the second path does not include any target node, taking the second path as a candidate path.

[0132] Exemplarily, in the case where the second path does not include any target node, it indicates that the first instruction in the instruction set is complete and the instruction does not need to be supplemented, and then the second path is taken as a candidate path.

[0133] S904: determining a target path from all candidate paths.

[0134] There can be multiple candidate paths determined according to the above method, and thus a final target path needs to be determined from the multiple candidate paths.

[0135] To determine the target path, in some embodiments, the controller can be configured to, if there are multiple candidate paths, acquire user historical access data of each node in each candidate path; calculate a historical access amount total value of the corresponding candidate path based on the user historical access data of each node; determine the candidate path with the largest historical access amount total value from the multiple candidate paths as the target path; or randomly select one candidate path from the multiple candidate paths as the target path.

[0136] Exemplarily, to determine the target path, when there are multiple candidate paths, the controller can select the target path by using one of the following two strategies: 1. selecting the target path based on historical access data, and 2. randomly selecting the target path.

[0137] When the target path is selected based on historical access data, the controller can acquire, for each node in each candidate path, historical access data of the node from a database or a log. The historical access data can include a number of times, a frequency, etc. that a user historically executes an instruction represented by the node. Then, for each candidate path, a historical access amount total value of all nodes in the path is calculated, which can be determined by accumulating the historical access number of each node in the path. Finally, the historical access amount total values of all candidate paths are compared, and the path with the largest total value is selected as the target path. In this way, selecting the path with the largest historical access amount total value can better match the user's usage habits and improve user satisfaction. Moreover, a path that is frequently accessed is more likely to be the correct path for the user's current intention, which can improve the accuracy and efficiency of instruction execution.

[0138] When the target path is randomly selected, the controller uses a random number generator to randomly select a path from the candidate paths as the target path. In this way, no additional historical data and complex calculations are required, and each candidate path has an equal probability of being selected, ensuring the fairness of the selection.

[0139] S500: determining a target execution order between at least one first instruction according to the target path.

[0140] In some embodiments, each first instruction in the instruction set corresponds to a node on the target path, and the target execution order between the first instructions is determined according to the node order of the first instructions on the target path.

[0141] For example, when the first instruction in the instruction set is full screen playing and playing XXX movie, the target path found according to the instruction set includes a first node corresponding to playing XXX movie and a first node corresponding to full screen playing, and the first node corresponding to playing XXX movie is before the first node corresponding to full screen playing. Therefore, according to the node order on the target path, the target execution order of the first instruction is determined to be playing XXX movie first and then full screen playing.

[0142] If the target path includes a target node, the controller will determine the target execution order of the first instruction and the second instruction according to the node order on the target path. The determination of the target execution order is based on the arrangement of the nodes in the target path, where each node represents an instruction or operation. For each node on the target path, the controller will arrange the execution of the instructions in the order from the starting point to the end point. If a node is the target node of another node, the instruction of the target node will be executed before the first instruction that depends on it.

[0143] For example, when the first instruction in the instruction set is playing a song in sequence, there is a target node before the first node corresponding to playing a song in sequence in the target path found according to the instruction set, and the second instruction corresponding to the target node is opening a music program. Therefore, the second instruction needs to be supplemented to the instruction set, and the first instruction and the second instruction in the instruction set are executed in the order of the nodes corresponding to the first instruction and the second instruction, i.e., opening a music program first and then playing a song in sequence.

[0144] In view of this, in some embodiments, the controller executing the preset instruction directed acyclic graph to determine the target path composed of at least one first instruction can be configured to:

[0145] In the case where the target path includes a target node, the target execution order of the first instruction and the second instruction is determined according to the node order on the target path.

[0146] According to the above embodiments, the controller can ensure that all necessary dependencies are correctly handled when executing the instruction set, thereby ensuring the smoothness and effectiveness of the entire operation process.

[0147] S600: Execute the corresponding first instruction according to the target execution.

[0148] After determining the target execution order, the controller executes the first instruction between the instruction sets according to the determined target execution order. In this way, it is ensured that each first instruction can be executed correctly, improving the reliability and stability of the system.

[0149] According to the above embodiment, the controller receives voice interaction data input by a user; generates a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction; a preset instruction directed graph is obtained, the preset instruction directed graph includes multiple paths, and the paths represent execution sequences of preset instructions; the preset instructions include the first instructions; according to the preset instruction directed graph, a target path composed of the at least one first instruction is determined; according to the target path, a target execution sequence between the at least one first instruction is determined; and the corresponding first instructions are executed in the target execution sequence. This process can receive voice interaction data of a user, analyze and determine a corresponding instruction set, then find a target path in a preset instruction directed graph, determine a target execution sequence of instructions, and execute the instruction set according to the target sequence, improving the user's interaction experience and ensuring the correctness and reliability of the instructions.

[0150] In some embodiments, Figure 10 A flowchart for communicating with a server to determine an instruction execution sequence and an instruction set according to voice interaction data is provided for the embodiments of the present application. As shown in Figure 10 The present application provides a display device, which includes a display, a communicator configured to be communicatively connected with a server, and a controller configured to:

[0151] The controller controls the communicator to send voice interaction data to the server, so that the server determines a corresponding instruction set according to the voice interaction data, and determines a target execution sequence between first instructions of the instruction set based on a preset instruction directed graph; the preset instruction directed graph includes multiple paths, and the paths represent execution sequences of preset instructions; the preset instructions include the first instructions.

[0152] The controller controls the communicator to receive the instruction set and the corresponding target execution sequence from the server;

[0153] The corresponding first instructions are executed in the target execution sequence.

[0154] The display device 200 can be a smart TV with high computing power, which can obtain voice interaction data to generate an instruction set and execute it through the computing chip mounted thereon. However, in some application scenarios, the display device 200 can be a low-power device such as an advertising machine, and the low-power terminal device generates an instruction set and executes it at a slow speed, or cannot mount a computing model.

[0155] Based on the above problems, in some embodiments, the display device 200 can also interact with the server in real time to generate an instruction set through the server, and then the controller executes the instruction set. The server can be arranged based on the cloud, or arranged based on the local network, or arranged based on the edge computing search server. The way of obtaining the goddess of communication with the server alleviates the problem that the display device 200 cannot generate and execute the instruction set in time according to the obtained voice interaction data due to insufficient computing power.

[0156] In some embodiments, the present application also provides a server, comprising: a communicator configured to be communicatively connected with a display device; a processor connected with the communicator and configured to:

[0157] control the communicator to receive voice interaction data from the display device, and generate a corresponding instruction set according to the voice interaction data; the instruction set comprises at least one first instruction;

[0158] obtain a preset instruction directed acyclic graph, the preset instruction directed acyclic graph comprising a plurality of paths, the paths representing an execution order of preset instructions; the preset instructions include the first instructions;

[0159] determine a target path composed of the at least one first instruction according to the preset instruction directed acyclic graph;

[0160] determine a target execution order between the at least one first instruction according to the target path;

[0161] control the communicator to send the instruction set and the target execution order to the display device, so that the display device executes the corresponding first instructions according to the target execution order.

[0162] The server can receive the voice interaction data forwarded by the display device 200, and determine the corresponding instruction set according to the voice interaction data. And the instruction set and the corresponding execution order are sent to the display device 200, so that the display device 200 executes the instruction set according to the execution order. Alleviate the problem that the display device 200 with low computing power has difficulty in generating and determining the execution order of the instruction set. In some embodiments, the present application also provides a voice instruction execution method applied to a display device with high computing power, comprising:

[0163] receiving voice interaction data input by a user;

[0164] generating a corresponding instruction set according to the voice interaction data; the instruction set comprises at least one first instruction;

[0165] obtaining a preset instruction directed acyclic graph, the preset instruction directed acyclic graph comprising a plurality of paths, the paths representing an execution order of preset instructions; the preset instructions include the first instructions;

[0166] According to the preset instruction directed acyclic graph, a target path composed of at least one first instruction is determined;

[0167] According to the target path, a target execution order between the at least one first instruction is determined;

[0168] The corresponding first instruction is executed according to the target execution order.

[0169] The method is applied to a display device 200, such as a smart television, a computer, etc., which is equipped with a high-performance operation chip.

[0170] It can be understood that in some embodiments, a voice instruction execution method applied to a display device with low computing power is also provided. The controller 250 of the display device 200 forwards the user data to the server, and the server executes the process of generating the multimedia resource recommendation result, and sends the generated multimedia resource recommendation result to the display device 200. The display device 200 displays the multimedia resource recommendation result through the display 260.

[0171] From the above technical content, it can be seen that the present application provides a display device, a server and a voice instruction execution method. The method receives voice interaction data input by a user; generates a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction; a preset instruction directed acyclic graph is obtained, the preset instruction directed acyclic graph includes multiple paths, and the path represents an execution order of the preset instruction; the preset instruction includes the first instruction; according to the preset instruction directed acyclic graph, a target path composed of at least one first instruction is determined; according to the target path, a target execution order between the at least one first instruction is determined; and the corresponding first instruction is executed according to the target execution order. This process can receive voice interaction data of a user, analyze and determine a corresponding instruction set, then find a target path in a preset instruction directed acyclic graph, determine an execution order of the instruction, and execute the instruction set according to the order. In this way, the user's interactive experience is improved, and the correctness and reliability of the instruction are ensured.

[0172] The similar parts among the embodiments provided by the present application can be referred to each other, and the specific embodiments provided above are only a few examples under the general concept of the present application, and do not limit the protection scope of the present application. For those skilled in the art, any other embodiments extended according to the present application scheme without creative labor are within the protection scope of the present application.

Claims

1. A display device, characterized by comprising: The application relates to a voice interaction method and device. The application comprises: a display configured to display a user interface; a controller coupled with the display and configured to: receive voice interaction data input by a user; identify at least one user intent from the voice interaction data and convert the at least one user intent into an instruction set; the instruction set comprises at least one first instruction corresponding to the at least one user intent one by one; obtain a preset instruction directed acyclic graph, wherein the preset instruction directed acyclic graph comprises a plurality of paths, and the paths represent an execution order of preset instructions; the preset instructions comprise the first instructions; determine a target path composed of the at least one first instruction according to the preset instruction directed acyclic graph; determine a target execution order between the at least one first instruction according to the target path; 2. The display device of claim 1, wherein, execute the corresponding first instructions in the target execution order. The controller configured to determine the target path composed of the at least one first instruction according to the preset instruction directed acyclic graph, specifically comprises: for each first instruction in the instruction set, traverse the preset instruction directed acyclic graph to select all first paths comprising a first node matching the at least one first instruction; if there are a plurality of first paths, perform a de-duplication and equivalent path merging operation on the plurality of first paths to obtain second paths; if there are a plurality of second paths, determine hit rule data of the second paths according to the number of first nodes included in the second paths; sort the plurality of second paths based on the hit rule data of each second path; 3. The display device of claim 2, wherein, determine the target path according to the second path with the maximum hit rule data. The controller configured to determine the target path according to the second path with the maximum hit rule data, specifically comprises: determine whether the second path with the maximum hit rule data comprises a target node; the target node is a node matching a second instruction executed before the execution of the first instruction; in the case that the second path comprises at least one target node, add the second instruction corresponding to the target node to the instruction set and take the second path as a candidate path; in the case that the second path does not comprise any target node, take the second path as the candidate path; 4. The display device of claim 3, wherein, determine the target path from all the candidate paths. The controller configured to determine whether the second path with the maximum hit rule data comprises a target node, specifically comprises: determine whether the second path comprises at least one second node; the second node is a previous node of each first node in the second path, and the previous node is different from any first node; in the case that the second path comprises any second node, judge whether the second node is in a first state; the first state represents that the instruction corresponding to the second node is executed before the execution of the first instruction; when the second node is in the first state, determine that the second node is the target node.

5. The display device of claim 3, wherein, The controller is configured to determine the target path from all the candidate paths, and specifically configured to: if there are multiple candidate paths, obtain user historical access data of each node in each candidate path; based on the user historical access data of each node, calculate the historical access amount total value of the corresponding candidate path; determine the candidate path with the largest historical access amount total value as the target path from the multiple candidate paths; or if there are multiple candidate paths, randomly select one candidate path from the multiple candidate paths as the target path.

6. The display device of claim 3, wherein, The controller is configured to determine the target path composed of the at least one first instruction according to the preset instruction directed acyclic graph, and specifically configured to: if the target node is included in the target path, determine the target execution order of the first instruction and the second instruction according to the node order on the target path.

7. The display device of claim 1, wherein, The controller is further configured to: obtain historical user instructions; map each historical user instruction to a node in the directed acyclic graph, and take the instruction type corresponding to the historical user instruction as a node label; take the execution value of the historical user instruction, the terminal scene after the current historical user instruction is executed, and the node state as the node attributes of the directed acyclic graph; create a directed edge between the nodes corresponding to the historical user instructions according to the execution order of the historical user instructions; construct the preset instruction directed acyclic graph according to the node label, node attribute, and directed edge.

8. The display device of claim 1, wherein, The controller is configured to determine the corresponding instruction set according to the voice interaction data, and specifically configured to: convert the voice interaction data into text data based on voice recognition technology; analyze the text data using natural language processing technology to obtain user intent and entity information; based on the user intent and the entity information, find the first instruction corresponding to the user intent and the entity information in a preset rule library; the preset rule library is used to record the mapping relationship between the user intent and the first instruction; determine the instruction set based on the first instruction.

9. A display device, characterized by It includes: a display; a communicator configured to be communicatively linked with a server; a controller coupled to the display and the communicator, and configured to: receive user input voice interaction data; control the communicator to send the voice interaction data to the server, so that the server identifies at least one user intent according to the voice interaction data, converts at least one user intent into an instruction set, and determines the target execution order between the first instructions in the instruction set based on a preset instruction directed acyclic graph; the instruction set includes at least one first instruction, and at least one first instruction corresponds to at least one user intent one by one; wherein the preset instruction directed acyclic graph includes multiple paths, and the path represents the execution order of the preset instruction; the preset instruction includes the first instruction; control the communicator to receive the instruction set and the corresponding target execution order from the server; execute the corresponding first instruction according to the target execution order.

10. A server, characterized by It includes: The communicator is configured to be communicatively connected with the display device. The processor is coupled with the communicator and is configured to: control the communicator to receive voice interaction data from the display device, identify at least one user intent from the voice interaction data, and convert the at least one user intent into an instruction set; the instruction set includes at least one first instruction, and the at least one first instruction corresponds to the at least one user intent one by one; obtain a preset instruction directed acyclic graph, the preset instruction directed acyclic graph includes a plurality of paths, and the paths represent an execution order of preset instructions; the preset instructions include the first instructions; determine a target path composed of the at least one first instruction according to the preset instruction directed acyclic graph; determine a target execution order between the at least one first instruction according to the target path; control the communicator to send the instruction set and the target execution order to the display device, so that the display device executes the corresponding first instructions according to the target execution.

11. A method of executing a voice command, the method comprising: The execution method is applied to a display device, including: receiving voice interaction data input by a user; identifying at least one user intent from the voice interaction data, and converting the at least one user intent into an instruction set; the instruction set includes at least one first instruction, and the at least one first instruction corresponds to the at least one user intent one by one; obtaining a preset instruction directed acyclic graph, the preset instruction directed acyclic graph includes a plurality of paths, and the paths represent an execution order of preset instructions; the preset instructions include the first instructions; determining a target path composed of the at least one first instruction according to the preset instruction directed acyclic graph; determining a target execution order between the at least one first instruction according to the target path; executing the corresponding first instructions according to the target execution order.

Images