Display device, server and voice instruction execution method

Abstract

The invention provides a display device, a server and a voice instruction execution method. The method comprises the following steps: receiving voice interaction data input by a user; generating a corresponding instruction set according to the voice interaction data; a preset instruction directed acyclic graph is obtained, the preset instruction directed acyclic graph comprises a plurality of paths, and the paths represent the execution sequence of the preset instruction; the preset instruction comprises a first instruction; determining a target path formed by at least one first instruction according to a preset instruction directed acyclic graph; determining a target execution sequence between the at least one first instruction according to the target path; and executing the corresponding first instruction according to the target execution sequence. According to the process, the voice interaction data of the user can be received, the corresponding instruction set can be analyzed and determined, then the target path is searched in the preset instruction directed acyclic graph, the execution sequence of the instruction is determined, the instruction set is executed according to the sequence, the interaction experience of the user is improved, and the correctness and reliability of the instruction are ensured.

Description

Technical Field

[0001] The present application relates to the field of speech understanding technology, and in particular to a display device, a server, and a method for executing a speech instruction. Background Art

[0002] Currently, display devices can use speech recognition and intent understanding technology to extract multiple intents from the user's voice commands, generate instructions corresponding to each intent, obtain an instruction set, and execute the actions corresponding to the instructions in the order in which the instructions in the instruction set are generated.

[0003] However, the generation order of the instructions in the instruction set may be different from the order required to obtain the user's expected results, resulting in the display device executing the actions corresponding to the instructions according to the generation order, and the results obtained are inconsistent with the user's expected results, affecting the user experience. Summary of the invention

[0004] The present application provides a display device, a server, and a method for executing voice commands, which can determine the correct execution order of commands in an instruction set according to a preset instruction directed acyclic graph, improve the user's interactive experience, and ensure the correctness and reliability of the commands.

[0005] In a first aspect, an embodiment of the present application provides a display device, comprising: a display, configured to display a user interface; a controller, coupled to the display and configured to: receive voice interaction data input by a user; generate a corresponding instruction set based on the voice interaction data; the instruction set includes at least one first instruction; obtain a preset instruction directed acyclic graph, the preset instruction directed acyclic graph includes multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions include a first instruction; according to the preset instruction directed acyclic graph, determine a target path composed of at least one first instruction; determine a target execution order between at least one first instruction based on the target path; and execute the corresponding first instructions according to the target execution order.

[0006] In the technical solution provided by the present application, the controller receives the voice interaction data input by the user; generates a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction; obtains a preset instruction directed acyclic graph, the preset instruction directed acyclic graph includes multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions include a first instruction; according to the preset instruction directed acyclic graph, determines the target path composed of at least one first instruction; determines the target execution order between at least one first instruction according to the target path; executes the corresponding first instruction according to the target execution order. This process can receive the user's voice interaction data, parse and determine the corresponding instruction set, then search for the target path in the preset instruction directed acyclic graph, determine the execution order of the instructions, and execute the instruction set according to the order, so as to improve the user's interactive experience and ensure the correctness and reliability of the instructions.

[0007] In some feasible embodiments, the controller executes a preset instruction directed acyclic graph to determine a target path composed of at least one first instruction, and is specifically configured as follows: for each first instruction in the instruction set, traverse the preset instruction directed acyclic graph, and select all first paths including the first node that matches at least one first instruction; if there are multiple first paths, perform deduplication and merge equivalent path operations on the multiple first paths to obtain a second path; if there are multiple second paths, determine the number of hit rules of the second path according to the number of first nodes included in the second path; sort the multiple second paths based on the number of hit rules of each second path; and determine the target path according to the second path with the largest hit rule data. According to the above embodiment, deduplication and merging of the found first paths can reduce redundant paths and improve processing efficiency. Sorting the second paths according to the number of hit rules in the sorting result can determine the path including all the first instructions in the instruction set, so that the accuracy and reliability of the execution of the first instruction can be improved.

[0008] In some feasible embodiments, the controller executes to determine the target path based on the second path with the largest hit rule data, and is configured to: determine whether the second path with the largest hit rule data includes a target node; the target node is a node that matches the second instruction executed before executing the first instruction; when the second path includes at least one target node, add the second instruction corresponding to the target node to the instruction set, and use the second path as a candidate path; when the second path does not include any target node, use the second path as a candidate path; and determine the target path among all candidate paths.

[0009] According to the above embodiment, the instruction set can be completed according to whether there is a target node in the second path, so as to ensure the correctness of the instruction execution.

[0010] In some feasible embodiments, the controller executes a determination as to whether the second path with the largest hit rule data includes a target node, and is configured to: determine whether the second path includes at least one second node, the second node being the previous node of each first node in the second path, and the previous node is different from any first node; when the second path includes any second node, determine whether the second node is in a first state; the first state indicates that the instruction corresponding to the second node is executed before the first instruction is executed; when the second node is in the first state, determine that the second node is a target node.

[0011] According to the above embodiment, it is possible to first determine whether the second path includes the second node, and then determine whether the second path includes the target node of the first node based on the first state of the second node. In this way, it is possible to ensure that the execution order of the instructions meets all necessary dependencies, thereby avoiding execution failures caused by dependency errors.

[0012] In some feasible embodiments, the controller determines the target path among all candidate paths, and is specifically configured as follows: if there are multiple candidate paths, obtain the user's historical access data for each node in each candidate path; calculate the total historical access value of the corresponding candidate path based on the user's historical access data for each node; determine the candidate path with the largest total historical access value among the multiple candidate paths as the target path; or if there are multiple candidate paths, randomly select one of the multiple candidate paths as the target path. According to the above embodiment, the path with the largest total historical access value can be selected to better match the user's usage habits and improve user satisfaction. The target path can also be randomly selected without the need for additional historical data and complex calculations, and each candidate path has the same probability of being selected, ensuring the fairness of the selection.

[0013] In some feasible embodiments, the controller executes a directed acyclic graph based on a preset instruction to determine a target path consisting of at least one first instruction, and is configured to: when the target path includes a target node, determine the target execution order of the first instruction and the second instruction according to the order of nodes on the target path.

[0014] According to the above embodiment, the controller can ensure that all necessary dependencies are correctly processed when executing the instruction set, thereby ensuring the smoothness and effectiveness of the entire operation process.

[0015] In some feasible embodiments, the controller is further configured to: obtain historical user instructions; map each historical user instruction to a node in a directed acyclic graph, and use the instruction type corresponding to the historical user instruction as a node label; use the execution value of the historical user instruction, the terminal scene after the current historical user instruction is executed, and the node state as the node attributes of the directed acyclic graph; create directed edges between the nodes corresponding to the historical user instructions according to the execution order of the historical user instructions; and construct a preset instruction directed acyclic graph according to the node labels, node attributes, and directed edges. According to the above embodiment, the preset instruction directed acyclic graph can be established based on the historical user instructions, and the preset instruction directed acyclic graph can clearly indicate the dependency relationship between the preset instructions, ensure that the instructions are executed in the correct order, and improve the execution efficiency of the preset instructions.

[0016] In some feasible embodiments, the controller executes an instruction set corresponding to the voice interaction data, and is specifically configured to: convert the voice interaction data into text data based on voice recognition technology; use natural language processing technology to analyze the text data to obtain user intention and entity information; based on the user intention and entity information, search for a first instruction corresponding to the user intention and entity information in a preset rule library; the preset rule library is used to record the mapping relationship between the user intention and the first instruction; and determine the instruction set based on the first instruction.

[0017] According to the above embodiment, the speech interaction data is processed by speech recognition technology and natural language processing technology to obtain text data, user intention and entity information, which can improve the accuracy of the display device in identifying and understanding the user's intention. Furthermore, the instructions corresponding to different intentions and entity information are recorded in the preset rule library. By searching the rule library, it can be ensured that the generated first instruction is highly consistent with the actual needs of the user, reducing the possibility of misoperation. At the same time, the accuracy and response speed of the display device in executing the corresponding action can be improved.

[0018] In a second aspect, an embodiment of the present application provides a display device, comprising: a display; a communicator, configured to be communicatively connected to a server; a controller, coupled to the display and the communicator, and configured to: receive voice interaction data input by a user; control the communicator to send the voice interaction data to the server, so that the server determines a corresponding instruction set based on the voice interaction data, and determines a target execution order between the first instructions of the instruction set based on a preset instruction directed acyclic graph; wherein the preset instruction directed acyclic graph includes multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions include a first instruction; control the communicator to receive the instruction set and the corresponding target execution order from the server; and execute the corresponding first instruction according to the target execution order.

[0019] In a third aspect, an embodiment of the present application provides a server, comprising: a communicator, configured to be communicatively connected to a display device; a processor, coupled to the communicator, and configured to: control the communicator to receive voice interaction data from the display device, and generate a corresponding instruction set based on the voice interaction data; the instruction set includes at least one first instruction; obtain a preset instruction directed acyclic graph, the preset instruction directed acyclic graph includes multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions include a first instruction; according to the preset instruction directed acyclic graph, determine a target path composed of at least one first instruction; determine a target execution order between at least one first instruction based on the target path; control the communicator to send the instruction set and the target execution order to the display device, so that the display device executes the corresponding first instruction according to the target.

[0020] In a fourth aspect, an embodiment of the present application provides a method for executing voice commands, which is applied to a display device, including: receiving voice interaction data input by a user; generating a corresponding instruction set based on the voice interaction data; the instruction set includes at least one first instruction; obtaining a preset instruction directed acyclic graph, the preset instruction directed acyclic graph includes multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions include a first instruction; according to the preset instruction directed acyclic graph, determining a target path composed of at least one first instruction; determining a target execution order between at least one first instruction based on the target path; and executing the corresponding first instructions according to the target execution order.

[0021] It can be understood that the beneficial effects that can be achieved by the technical solutions provided in the second to fourth aspects mentioned above can refer to the beneficial effects in the first aspect and any optional implementation method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of the hardware configuration of a display device provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of software configuration of a display device provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of a flow chart of a display device according to an embodiment of the present application determining the execution order of the first instruction in an instruction set according to a preset directed acyclic graph;

[0027] Figure 5 A schematic diagram of a flow chart of generating an instruction set for a display device provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of a process for constructing a directed acyclic graph of preset instructions for a display device provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of a preset instruction directed acyclic graph provided in an embodiment of the present application;

[0030] Figure 8 A schematic diagram of a flow chart of a display device searching for a target path according to an instruction set provided in an embodiment of the present application;

[0031] Fig. 9 A schematic diagram of a process for determining a target path for a display device provided in an embodiment of the present application;

[0032] Fig.10 A flowchart of an embodiment of the present application providing communication with a server to determine an instruction execution order and an instruction set based on voice interaction data. DETAILED DESCRIPTION

[0033] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0034] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0035] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0036] The terms "including" and "having" and any variations thereof are intended to cover but not be exclusive, for example, a product or device comprising a list of components is not necessarily limited to all the components explicitly listed, but may include other components not explicitly listed or inherent to such products or devices. The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with the element.

[0037] The display device 200 provided in the embodiment of the present application generally refers to a device with image display and data processing capabilities, and can have various implementation forms, for example, it can be a television, a smart TV, a computer, a laser projection device, a display (monitor), an electronic whiteboard (electronic bulletin board), an electronic desktop (electronic table), a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0038] Figure 1 Schematic diagram of an operation scenario between a display device 200 and a control device 100 provided in an embodiment of the present application. Figure 1 As shown, the user can operate the display device 200 through the terminal device 300 or the control device 100 .

[0039] In some embodiments, the control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that the display device 200 can recognize and respond to. The control device 100 can be a remote control, a stylus, a handle, etc., wherein the communication between the remote control and the display device 200 includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user can input user instructions through buttons on the remote control, voice input, control panel input, etc. to control the display device 200.

[0040] In some embodiments, a terminal device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) can also be used as a control device to perform human-computer interaction between a user and the display device 200. The mobile terminal 300 can also be used as a communication device to establish a communication connection with the display device 200 and perform data interaction. The mobile terminal 300 can install the same software application as the display device 200, and then connect and communicate through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0041] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0042] In some embodiments, the display device 200 may also be controlled in a manner other than the control device 100 and the terminal device 300, for example, the user's voice command control may be directly received through a module for acquiring voice commands configured inside the display device 200, or the user's voice command control may be received through a voice control device provided outside the display device 200. The display device 200 may also receive user control through touch or gestures.

[0043] In some embodiments, the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0044] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0045] Figure 2 FIG. 1 shows a hardware configuration block diagram of a control device 100 according to some embodiments. Figure 2 As shown, the control device 100 includes a controller 250, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive input operation instructions from the user, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200.

[0046] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0047] like Figure 2 As shown, the display device 200 may include at least one of a modem 210, a communication device 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0048] In some embodiments, the modem 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0049] In some embodiments, the communication device 220 is a component for communicating with an external device or server according to various communication protocol types. The display device 200 can be provided with a plurality of communication devices 220 according to the different communication modes supported. For example, when the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function. The communication device 220 may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish the sending and receiving of control signals and data signals with the external control device 100 or the server 400 through the communication device 220.

[0050] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0051] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0052] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0053] In some embodiments, the controller 250 controls the operation of the display device 200 and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0054] In some embodiments, the controller 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a random access memory (RAM), a read-only memory (ROM), a first interface to an nth interface for input / output, a communication bus, etc. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0055] In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be in a peripheral device of the main device where the controller 250 is located, such as an external set-top box.

[0056] In some embodiments, the user may input a user command in a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0057] In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0058] In some embodiments, the "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0059] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

[0060] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0061] In some embodiments, the user interface may be used to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).

[0062] In some embodiments, the display device may include a microphone array for acquiring the user's voice. In the process of receiving the user's voice data, the microphone array may collect the user's voice interaction data and convert the voice interaction data into an electrical signal so that the display device can accurately understand and respond.

[0063] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling the hardware resources and software resources of the display device 200. The operating system may control the display device to provide a user interface, for example, the operating system may directly control the display device to provide a user interface, or may provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0064] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0065] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0066] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0067] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0068] Figure 3 A schematic diagram of the software configuration of the display device 200 provided in an embodiment of the present application.

[0069] like Figure 3As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0070] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0071] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0072] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0073] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0074] With the rapid development of digital and intelligent technologies, display devices with voice interaction functions have become an important part of modern work and life. Through natural language processing technology, display devices can understand and respond to users' voice commands, greatly improving the convenience and efficiency of human-computer interaction. Users can complete various operations such as playing music, watching videos, and adjusting device settings through simple voice commands. For example, users can say to the display device: play XXX's songs in sequence, play Wolf Warrior 2 in full screen, play songs in sequence, etc. The display device can recognize the voice commands and execute the corresponding actions of the voice commands.

[0075] At present, the display device can use voice recognition and intent understanding technology to extract multiple intentions from the user's voice instructions, generate execution instructions corresponding to each intention, and execute the corresponding actions in the order in which the execution instructions are generated. However, the display device executes the corresponding actions according to the recognized execution order, which will cause the display device to execute the actions corresponding to the instructions in sequence to achieve the effect that is inconsistent with the actual effect that the user wants to achieve. For example, for the instruction "play XXX's songs in sequence", the display device can successfully identify two execution instructions: one is to set the playback mode to sequential playback, and the other is to query and play XXX's songs. According to the order of the above instructions, the display device will first set the playback mode to sequential playback, and then query XXX's songs. However, the user's actual expectation is to find XXX's song list first, and then set it to sequential playback. This difference in the execution order of execution instructions not only affects the user experience, but may also cause some unnecessary errors or confusion.

[0076] Secondly, existing technologies also face challenges when processing context-dependent instructions. For example, when a user says "play songs in sequence", the system usually recognizes the instruction to set the playback mode to sequential playback. However, if the user is not currently in a song playback scenario, this instruction will not be executed, causing confusion to the user. In this case, the user's real intention may be to start the song playback first, and then set the playback mode to sequential playback. Existing technologies often lack a full understanding of the user's current context, and thus cannot accurately judge and execute the user's actual needs.

[0077] In view of the above problems, an embodiment of the present application provides a display device that can determine a corresponding instruction set based on the user's voice interaction data, use a preset instruction directed acyclic graph to determine the execution order of instructions in the instruction set, and then execute the instructions according to the execution order.

[0078] In some embodiments, Figure 4 A flow chart of a display device according to an embodiment of the present application determining the execution order of the first instruction in the instruction set according to a preset directed acyclic graph. Figure 4 As shown, the embodiment of the present application provides a display device 200, which includes: a display and a controller. The controller 250 is coupled to the display 260 and is configured to perform the following program steps:

[0079] S100: receiving voice interaction data input by a user.

[0080] In some embodiments, the display device may perform corresponding actions based on the voice interaction data input by the user. For example, the display device may capture the audio signal of the user's voice input, i.e., the voice interaction data, through a built-in microphone array, and process the voice interaction data to implement the corresponding function or service. This process can be applied to multiple application scenarios such as information query, entertainment functions, personal assistant services, and health management. For example, a user can control the display device to query the weather conditions of the day, or control the display device to play music, videos, or start games and other entertainment projects through simple voice commands.

[0081] S200: Generate a corresponding instruction set according to the voice interaction data.

[0082] The controller can identify the user's intention by analyzing the voice interaction data input by the user, and convert the user's intention into an understandable and executable instruction set, wherein the instruction set includes at least one first instruction.

[0083] In order to generate an instruction set, in some embodiments, the controller may convert the voice interaction data into text data based on voice recognition technology, and analyze the text data using natural language processing technology to obtain user intent and entity information. Afterwards, based on the user intent and entity information, the controller may search for a first instruction corresponding to the user intent and entity information in a preset rule library, and finally determine an instruction set based on the first instruction.

[0084] Exemplarily, after the display device captures the user's voice interaction data through a microphone array, it performs pre-processing such as noise reduction and echo cancellation on the voice interaction data to improve the accuracy of subsequent voice recognition. Then, the deep learning model can be used to convert the audio signal of the voice interaction data into text data. After that, the text data is analyzed using natural language processing technology, such as word segmentation, part-of-speech tagging, named entity recognition, and syntactic analysis, to extract user intent and entity information from the text data. Furthermore, a first instruction that matches the user intent and entity information is searched in the preset rule library. The first instruction can be a specific operation instruction or an API request to call other services. Here, the preset rule library is a database that records the mapping relationship between the user intent and the first instruction. Finally, an instruction set is constructed based on the first instruction found in the preset rule library.

[0085] For example, when the controller receives voice interaction data input by the user as "play XXX's songs in sequence", the controller converts the voice interaction data into text data "play XXX's songs in sequence", and performs intent recognition and entity recognition on the sequential playback of XXX (a certain singer)'s songs, and obtains the user's intent to sequentially play songs and play XXX's songs, and the entity information is XXX. Further, the controller determines two first instructions in the preset rule library based on the user's intent and entity information, namely, sequentially play songs and play XXX's songs, and constructs an instruction set based on the two first instructions.

[0086] In this way, the voice interaction data is processed using voice recognition technology and natural language processing technology to obtain text data, user intent and entity information, which can improve the accuracy of the display device in identifying and understanding the user's intent. Furthermore, the instructions corresponding to different intents and entity information are recorded in the preset rule library. By searching the rule library, it can be ensured that the generated first instruction is highly consistent with the user's actual needs, reducing the possibility of misoperation. At the same time, it can improve the accuracy and response speed of the display device in executing the corresponding action.

[0087] In some embodiments, Figure 5 A flow chart of a display device generating an instruction set provided in an embodiment of the present application. Figure 5 As shown, the display device receives the audio signal input by the user through the microphone array and sends the audio signal to the controller. The controller converts the audio signal into voice interaction data and sends the voice interaction data to the voice recognition module. The voice recognition module converts the voice interaction data into text data and sends it to the semantic understanding module. The semantic understanding module performs entity recognition for intent recognition on the text data, obtains user intent and entity information, and sends the user intent and entity information to the instruction generation module. The instruction generation module searches for the corresponding first instruction in the preset rule library according to the user intent and entity information, builds an instruction set, and returns the instruction set to the controller.

[0088] S300: Obtain a preset instruction directed acyclic graph.

[0089] In some embodiments, the preset instruction directed acyclic graph includes multiple paths, the paths represent the execution order of the preset instructions, and the preset instructions include a first instruction. In the preset instruction directed acyclic graph, each node represents a preset instruction, and the directed edges represent the dependency relationship between the preset instructions. The execution order between the preset instructions can be determined through the preset instruction directed acyclic graph.

[0090] In some embodiments, Figure 6 A schematic diagram of a flow chart of constructing a preset instruction directed acyclic graph for a display device provided in an embodiment of the present application. Figure 6 As shown, the controller constructs a preset instruction directed acyclic graph and can be configured to perform the following steps:

[0091] S601: Obtain historical user instructions, and map each historical user instruction to a node in a directed acyclic graph.

[0092] In some embodiments, historical user instructions may include various types based on application scenarios, such as application control (such as opening / closing applications, switching applications, etc.), playback control (such as playing / pausing / stopping media playback, jumping to a specified time point, switching playback modes, etc.), device control (opening / closing devices, switching network connections, etc.), etc., but not limited thereto. When selecting historical user instructions, instructions with an execution order are given priority, such as playing songs, playing songs in sequence, etc. Instructions without an execution order, such as increasing the volume and shutting down the device, do not need to be added.

[0093] Exemplarily, the controller can obtain historical user instructions, each of which includes an instruction type, an instruction value, a terminal scene after the instruction is executed, and a node state. Among them, the instruction type can be determined according to the application scenario, the execution value is the instruction text, that is, the original instruction input by the user, the terminal scene is the state or scene of the terminal device after execution, and the node state is whether the current instruction has a strong dependency relationship with the next instruction. Here, the node state is related to the execution order between historical user instructions, and the node state of the node can be determined based on whether the node is a necessary execution prerequisite for the next node. Exemplarily, for the two historical user instructions of opening a music application and playing songs in sequence, the correct execution order is: open the application first, and then play the songs in sequence. In this regard, for the instruction to open the application, it is a necessary prerequisite for the execution of the next node to play songs in sequence. In this regard, the node state of the node corresponding to the instruction to open the application needs to be set to the first state.

[0094] S602: Use the instruction type corresponding to the historical user instruction as a label of the node.

[0095] The instruction type can be determined according to the application scenario. Each historical user instruction is mapped to a node in a directed acyclic graph. The node label is the instruction type, its field name is label, and the field type is string.

[0096] S603: Use the execution value of the historical user instruction, the terminal scene after the current historical user instruction is executed, and the node state as the node attributes of the directed acyclic graph.

[0097] The execution value of the historical user instruction, the terminal scene after execution, and the node state of the instruction are used as attributes of the node, where the field name of the instruction value is command_param, the field type is Json structure, the field name of the terminal scene after execution is executed_state, the character type is string, and the field name of the node state is necessary, and the character type is Boolean value.

[0098] S604: Create directed edges between nodes corresponding to the historical user instructions according to the execution order of the historical user instructions.

[0099] According to the execution order between historical user instructions, directed edges are created between corresponding nodes, so that the directed edges can be used to indicate that the instruction corresponding to the current node depends on the execution result of the instruction corresponding to the previous node.

[0100] S605: Construct a preset instruction directed acyclic graph according to the node labels, node attributes and directed edges.

[0101] After determining the nodes, node attributes, and directed edges in the directed acyclic graph, a directed acyclic graph is created according to the nodes and the directed edges between the nodes.

[0102] After constructing the directed acyclic graph, it is possible to check whether the preset instruction directed acyclic graph is acyclic. The first method is to determine whether the preset instruction directed acyclic graph is acyclic based on topological sorting. Exemplarily, an in-degree table can be created to record the in-degree of each node in the preset instruction directed acyclic graph, that is, how many edges point to the node. Then, all nodes with an in-degree of 0 are added to the queue. The nodes in the queue can then be processed. Exemplarily, a node is taken out of the queue, and the in-degree of all its adjacent nodes is reduced by 1. If the in-degree of an adjacent node becomes 0, the node is added to the queue. Finally, when all nodes have been processed, the directed acyclic graph is acyclic.

[0103] The second method is to determine whether the preset instruction directed acyclic graph is acyclic based on a depth-first search. For example, a color table can be created to record the access status of each node (such as white for unvisited, gray for being visited, and black for visited). In a hurry, a depth-first search traversal is performed from each unvisited node. If a gray node is encountered during the traversal process, it means that there is a cycle. If no gray node is encountered during the whole process, the preset instruction directed acyclic graph is acyclic.

[0104] Figure 7 A schematic diagram of the structure of a preset instruction directed acyclic graph provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the preset instruction directed acyclic graph includes nodes 701-node 707, where node 701 is the head node, and nodes 702-node 707 all include node labels and node data, where the node label is the instruction type, and the node attributes are the execution value of the instruction, the terminal scene after the current historical user instruction is executed, and the node status. Figure 7 , the node label of node 702 is application control, the node execution value is to open the music application, the state of the terminal device after execution is music, and the node state is the first state. The node label of node 703 is music playback control, the node execution value is to play sequentially, the state of the terminal device after the instruction is executed is music, and the node state is the first state. The node label of node 704 is application control, the node execution value is to open the first music application, the state of the terminal device after execution is music, and the node state is not the first state. The node label of node 705 is music focus search, the state of the terminal device after execution is music, and the node state is the first state. The node label of node 706 is music playback control, the node execution value is to play sequentially, the state of the terminal device after the instruction is executed is music, and the node state is the first state. The node label of node 707 is application control, the node execution value is to open the second music application, and the state of the terminal device after execution is music, and the node state is the first state.

[0105] In this way, historical user instructions can be used to establish a preset instruction directed acyclic graph, which can clearly indicate the dependencies between the preset instructions, ensure that the instructions are executed in the correct order, and improve the execution efficiency of the preset instructions.

[0106] S400: Determine a target path consisting of at least one first instruction according to a preset instruction directed acyclic graph.

[0107] In order to determine the correct execution order of the first instruction in the instruction set according to the preset instruction directed acyclic graph, in some embodiments, Figure 8 A flow chart of a display device according to an embodiment of the present application determining a target path composed of at least one first instruction, such as Figure 8As shown, the controller can be configured to perform the following steps:

[0108] S801: For each first instruction in the instruction set, traverse a preset instruction directed acyclic graph, and select all first paths including at least one first node that matches the first instruction.

[0109] In some embodiments, for each first instruction in the instruction set, the controller may traverse the preset instruction directed acyclic graph through a depth-first search or a breadth-first search to find all first paths containing the first node corresponding to the first instruction.

[0110] Exemplarily, when the instruction set includes two first instructions, for the first first instruction, the preset instruction directed acyclic graph is traversed to find all first paths including the first node corresponding to the instruction. For the second first instruction, the preset instruction directed acyclic graph is traversed to find all first paths including the first node corresponding to the instruction.

[0111] S802: If there are multiple first paths, perform deduplication and merging equivalent path operations on the multiple first paths to obtain a second path.

[0112] In some embodiments, when there are multiple first paths, all collected first paths are deduplicated to remove duplicate paths, ensure that each first path is unique, and remove first paths that are exactly the same in node order and attributes to obtain second paths. There can be multiple second paths.

[0113] Exemplarily, taking the instruction set including two first instructions as an example, a first path including the first first instruction and a first path including the second first instruction are deduplicated and equivalent paths are merged to obtain a second path.

[0114] S803: If there are multiple second paths, determine the number of hit rules of the second paths according to the number of first nodes included in the second paths.

[0115] In some embodiments, when there are multiple second paths, the number of first nodes included in each second path can be used as the number of hit rules for each second path. For example, taking the instruction set including two first instructions as an example, the second paths include 3, and the number of first nodes included in each second path is 2, 2, and 1, then the number of hit rules corresponding to each second path is 2, 2, and 1 respectively.

[0116] S804: Sort the multiple second paths based on the number of hit rules of each second path.

[0117] In some embodiments, the controller may sort the plurality of second paths from largest to smallest according to the number of hit rules.

[0118] Exemplarily, the number of hit rules corresponding to each second path is 2, 2, and 1 respectively, and the second paths are sorted according to the number of hit rules.

[0119] S805: Determine the target path according to the second path with the largest rule data.

[0120] In some embodiments, a path with the largest number of hit rules from the sorting results is used as a target path, and the target path represents the correct execution order of the first instruction in the instruction set.

[0121] Exemplarily, when the instruction set of the second path includes two first instructions and there are two paths with a hit rule number of 2 in the second path, the target path can be determined according to the two second paths with a hit rule number of 2.

[0122] In this way, deduplication and merging of the found first paths can reduce redundant paths and improve processing efficiency. Sorting the second paths according to the number of hit rules can determine the path including all first instructions in the instruction set, thus improving the accuracy and reliability of the execution of the first instructions.

[0123] In some embodiments, Fig. 9 A schematic diagram of a process for determining a target path for a display device provided in an embodiment of the present application, such as Fig. 9 As shown, the controller determines the target path according to the second path with the largest hit rule data, and can be configured to perform the following steps:

[0124] S901: Determine whether the second path with the largest hit rule data includes a target node.

[0125] In some embodiments, the target node is a node that matches the second instruction executed before executing the first instruction, that is, a strongly dependent node. In the process of determining the target path according to the second path with the largest hit rule data, the target path can be determined according to whether the second path with the largest hit rule data includes the target node of the first node corresponding to the first instruction in the instruction set. Among them, the target node is the previous node of the first node in the second path, and the node is different from any first node, and the state of the node is the first state (for example, True), and the first state indicates that the node is the target node of the next node. In other words, the target node is a node that does not exist in the instruction set and has a necessary execution order with the first node.

[0126] Exemplarily, the controller traverses each node in the second path, checks whether the previous node of each first node is the second node, and the instruction corresponding to the second node does not exist in the instruction set. That is, there is a directed edge between the second node and the first node, and the controller can execute the instruction corresponding to the second node before executing the first instruction corresponding to the first node. Further, if the second path includes a second node, the controller obtains the node state of the second node. If the node state is the first state, the node is determined to be a target node. In the case where the second node is the target node, the controller must execute the instruction corresponding to the second node before executing the first instruction corresponding to the first node.

[0127] In view of this, in some embodiments, the controller executes to determine whether the second path with the largest hit rule data includes a target node, and can be configured to: determine whether the second path includes at least one second node, the second node is the previous node of each first node in the second path, and the previous node is different from any first node; when the second path includes any second node, determine whether the second node is in the first state; the first state indicates that the instruction corresponding to the second node is executed before the first instruction is executed; when the second node is in the first state, determine that the second node is the target node.

[0128] According to the above embodiment, it is possible to first determine whether the second path includes the second node, and then determine whether the state of the second node is the first state to determine whether the second path includes the target node of the first node. In this way, it can be ensured that the execution order of the instructions meets all necessary dependencies, thereby avoiding execution failures caused by dependency errors.

[0129] S902: When the second path includes at least one target node, add a second instruction corresponding to the target node to the instruction set, and use the second path as a candidate path.

[0130] Exemplarily, when the second path includes at least one target node, it means that the execution of the first instruction in the second path depends on the execution result of the target node, that is, before the controller executes the first instruction corresponding to the first node, it needs to execute the second instruction corresponding to the target node first, then the second instruction needs to be added to the instruction set to complete the instructions in the instruction set and ensure the integrity of the execution of the instructions in the instruction set. After the second instruction is added to the instruction set, the second path is used as a candidate path.

[0131] S903: When the second path does not include any target node, use the second path as a candidate path.

[0132] Exemplarily, when the second path does not include any target node, it means that the first instruction in the instruction set is complete and does not need to be completed, and the second path is used as a candidate path.

[0133] S904: Determine a target path among all candidate paths.

[0134] There may be multiple candidate paths determined according to the above method, so the final target path needs to be determined from the multiple candidate paths.

[0135] In order to determine the target path, in some embodiments, the controller can be configured to obtain the user's historical access data for each node in each candidate path if there are multiple candidate paths; calculate the total value of the historical access volume of the corresponding candidate path based on the user's historical access data for each node; determine the candidate path with the largest total value of historical access volume among the multiple candidate paths as the target path; or if there are multiple candidate paths, randomly select one of the multiple candidate paths as the target path.

[0136] Exemplarily, in order to determine the target path, when there are multiple candidate paths, the controller may adopt one of the following two strategies to select the target path: 1. Select the target path based on historical access data; 2. Select the target path randomly.

[0137] When selecting a target path based on historical access data, the controller can obtain the historical access data of the node from a database or log for each node in each candidate path. The historical access data may include the number of times and frequency of the user's historical execution of the instruction represented by the node. Next, for each candidate path, the total historical access value of all nodes in the path is calculated, and the total historical access value can be determined by accumulating the number of historical accesses to each node in the path. Finally, the total historical access values ​​of all candidate paths are compared, and the path with the largest total value is selected as the target path. In this way, selecting the path with the largest total historical access value can better match the user's usage habits and improve user satisfaction. In addition, frequently visited paths are more likely to be the correct path for the user's current intention, which can improve the accuracy and efficiency of instruction execution.

[0138] When randomly selecting a target path, the controller uses a random number generator to randomly select a path from the candidate paths as the target path. In this way, no additional historical data and complex calculations are required, and each candidate path has the same probability of being selected, ensuring the fairness of the selection.

[0139] S500: Determine a target execution order between at least one first instruction according to a target path.

[0140] In some embodiments, each first instruction in the instruction set corresponds to a node on the target path, and the target execution order between the first instructions is determined according to the node order of the first instructions on the target path.

[0141] Exemplarily, when the first instruction in the instruction set is full-screen playback and playing XXX movie, the target path found according to the instruction set includes the first node corresponding to playing XXX movie and the first node corresponding to full-screen playback, and the first node corresponding to playing XXX movie is before the first node corresponding to full-screen playback, then the target execution order of the first instruction is determined according to the node order on the target path to first execute playing XXX movie and then execute full-screen playback.

[0142] If the target path includes a target node, the controller will determine the target execution order of the first instruction and the second instruction based on the order of the nodes on the target path. The target execution order is determined based on the arrangement of the nodes in the target path, where each node represents an instruction or operation. For each node on the target path, the controller will arrange the execution of the instructions in order from the starting point to the end point. If a node is the target node of another node, the instruction of the target node will be executed before the first instruction that depends on it.

[0143] For example, when the first instruction in the instruction set is to play songs in sequence, there is a target node before the first node corresponding to the sequentially playing songs in the target path found by the instruction set, and the corresponding second instruction is to open the music program. In this way, the second instruction needs to be added to the instruction set and executed in the order of the nodes corresponding to the first and second instructions in the instruction set, that is, the music program is opened first, and then the sequentially playing songs is executed.

[0144] In view of this, in some embodiments, the controller executes the predetermined instruction directed acyclic graph to determine the target path composed of at least one first instruction, which may be configured as follows:

[0145] In the case where the target path includes the target node, the target execution order of the first instruction and the second instruction is determined according to the order of the nodes on the target path.

[0146] According to the above embodiment, the controller can ensure that all necessary dependencies are correctly processed when executing the instruction set, thereby ensuring the smoothness and effectiveness of the entire operation process.

[0147] S600: Execute the corresponding first instruction according to the target.

[0148] After determining the target execution sequence, the controller executes the first instruction among the instruction sets according to the determined target execution sequence, thus ensuring that each first instruction can be correctly executed, thereby improving the reliability and stability of the system.

[0149] According to the above embodiment, the controller receives the voice interaction data input by the user; generates a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction; obtains a preset instruction directed acyclic graph, the preset instruction directed acyclic graph includes multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions include a first instruction; according to the preset instruction directed acyclic graph, determines the target path composed of at least one first instruction; determines the target execution order between at least one first instruction according to the target path; executes the corresponding first instruction according to the target execution order. This process can receive the user's voice interaction data, parse and determine the corresponding instruction set, then search for the target path in the preset instruction directed acyclic graph, determine the target execution order of the instructions, and execute the instruction set according to the target order, thereby improving the user's interactive experience and ensuring the correctness and reliability of the instructions.

[0150] In some embodiments, Fig.10 A flow chart of communicating with a server to determine the instruction execution order and instruction set according to voice interaction data provided by an embodiment of the present application. Fig.10 As shown, the present application provides a display device, including: a display; a communicator, configured to communicate with a server; and a controller, configured to:

[0151] Controlling the communicator to send voice interaction data to the server, so that the server determines the corresponding instruction set according to the voice interaction data, and determines the target execution order between the first instructions of the instruction set based on the preset instruction directed acyclic graph; wherein the preset instruction directed acyclic graph includes multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions include the first instruction;

[0152] The control communicator receives the instruction set and the corresponding target execution sequence from the server;

[0153] Execute the corresponding first instruction in the target execution order.

[0154] The display device 200 may be a smart TV with high computing power, and may obtain voice interaction data through its own computing chip to generate and execute instruction sets. However, in some application scenarios, the display device 200 may be a low computing power device such as an advertising machine, and low computing power terminal devices may generate and execute instruction sets more slowly or may not be able to carry computing models.

[0155] Based on the above problems, in some embodiments, the display device 200 can also interact with the server in real time to generate an instruction set through the server, and then the controller executes the instruction set. The server can be a server deployed on the cloud, a server deployed on the local network, or a server searched based on edge computing. The method of communicating with the server to obtain Zhilingjie alleviates the problem that the display device 200 cannot generate and execute the instruction set in time according to the acquired voice interaction data due to insufficient computing power.

[0156] In some embodiments, the present application further provides a server, including: a communicator, configured to be communicatively connected to a display device; and a processor, connected to the communicator and configured to:

[0157] Controlling the communicator to receive voice interaction data from the display device, and generating a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction;

[0158] Obtaining a preset instruction directed acyclic graph, the preset instruction directed acyclic graph including a plurality of paths, the paths representing the execution order of the preset instructions; the preset instructions including a first instruction;

[0159] Determining a target path consisting of at least one first instruction according to a preset instruction directed acyclic graph;

[0160] determining a target execution order between at least one first instruction according to the target path;

[0161] The control communicator sends the instruction set and the target execution sequence to the display device, so that the display device executes the corresponding first instruction according to the target.

[0162] The server can receive the voice interaction data forwarded by the display device 200, and determine the corresponding instruction set according to the voice interaction data. And the instruction set and the corresponding execution order are sent to the display device 200, so that the display device 200 executes the instruction set according to the execution order. The problem of difficulty in generating and determining the execution order of the instruction set by the low computing power display device 200 is alleviated. In some embodiments, the present application also provides a method for executing voice instructions, which is applied to a display device with higher computing power, including:

[0163] Receive voice interaction data input by the user;

[0164] Generate a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction;

[0165] Obtaining a preset instruction directed acyclic graph, the preset instruction directed acyclic graph including a plurality of paths, the paths representing the execution order of the preset instructions; the preset instructions including a first instruction;

[0166] Determining a target path consisting of at least one first instruction according to a preset instruction directed acyclic graph;

[0167] determining a target execution order between at least one first instruction according to the target path;

[0168] Execute the corresponding first instruction in the target execution order.

[0169] The method is applied to a display device 200 equipped with a computing chip with relatively high computing power, such as a display device 200 such as a smart TV and a computer.

[0170] It is understandable that, in some embodiments, a method for executing a voice command applied to a display device with low computing power is also provided. The controller 250 of the display device 200 forwards the user data to the server, and the server executes the process of generating the multimedia resource recommendation result, and sends the generated multimedia resource recommendation result to the display device 200. The display device 200 displays the multimedia resource recommendation result through the display 260.

[0171] As can be seen from the above technical content, the present application provides a display device, a server and a method for executing voice instructions. The method receives voice interaction data input by a user; generates a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction; obtains a preset instruction directed acyclic graph, the preset instruction directed acyclic graph includes multiple paths, and the paths represent the execution order of the preset instructions; the preset instructions include a first instruction; according to the preset instruction directed acyclic graph, determine the target path composed of at least one first instruction; determine the target execution order between at least one first instruction according to the target path; execute the corresponding first instruction according to the target execution order. The process can receive the user's voice interaction data, parse and determine the corresponding instruction set, then search for the target path in the preset instruction directed acyclic graph, determine the execution order of the instructions, and execute the instruction set according to the order. In this way, the user's interactive experience is improved and the correctness and reliability of the instructions are ensured.

[0172] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; A controller is coupled to the display and is configured to: Receive voice interaction data input by the user; Generate a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction; Acquire a preset instruction directed acyclic graph, wherein the preset instruction directed acyclic graph includes a plurality of paths, and the paths represent an execution order of the preset instructions; the preset instructions include the first instruction; Determining a target path formed by the at least one first instruction according to the preset instruction directed acyclic graph; determining a target execution order between the at least one first instruction according to the target path; Execute the corresponding first instructions according to the target execution order.

2. The display device according to claim 1, characterized in that The controller determines a target path formed by the at least one first instruction according to the preset instruction directed acyclic graph, and is specifically configured as follows: For each of the first instructions in the instruction set, traverse the preset instruction directed acyclic graph, and select all first paths including at least one first node matching the first instruction; If there are multiple first paths, perform deduplication and merging equivalent path operations on the multiple first paths to obtain a second path; If there are multiple second paths, determine the number of hit rules of the second path according to the number of the first nodes included in the second path; sorting the plurality of second paths based on the number of hit rules of each second path; The target path is determined according to the second path having the largest hit rule data.

3. The display device according to claim 2, characterized in that The controller determines the target path according to the second path with the largest hit rule data, and is configured to: Determine whether the second path with the largest hit rule data includes a target node; the target node is a node that matches the second instruction executed before executing the first instruction; In the case where the second path includes at least one of the target nodes, adding the second instruction corresponding to the target node to the instruction set, and taking the second path as a candidate path; In a case where the second path does not include any of the target nodes, using the second path as the candidate path; The target path is determined among all the candidate paths.

4. The display device according to claim 3, characterized in that The controller determines whether the second path with the largest hit rule data includes a target node, and is configured to: determining whether the second path includes at least one second node, the second node being a previous node of each of the first nodes in the second path, and the previous node being different from any of the first nodes; In the case where the second path includes any of the second nodes, determining whether the second node is in a first state; The first state represents that the instruction corresponding to the second node is executed before the first instruction is executed; When the second node is in the first state, the second node is determined to be the target node.

5. The display device according to claim 3, characterized in that The controller determines the target path among all the candidate paths, and is specifically configured to: If there are multiple candidate paths, obtaining the user's historical access data of each node in each candidate path; Based on the user historical access data of each node, calculate the total historical access volume of the corresponding candidate path; Determine the candidate path with the largest total value of historical visits among the multiple candidate paths as the target path; or If there are multiple candidate paths, one of the multiple candidate paths is randomly selected as the target path.

6. The display device according to claim 3, characterized in that The controller determines a target path formed by the at least one first instruction according to the preset instruction directed acyclic graph, and is configured to: In a case where the target path includes the target node, the target execution order of the first instruction and the second instruction is determined according to the order of nodes on the target path.

7. The display device according to claim 1, characterized in that The controller is also configured to: Get historical user instructions; Mapping each of the historical user instructions to a node in a directed acyclic graph, and using the instruction type corresponding to the historical user instruction as a node label; Using the execution value of the historical user instruction, the terminal scene after the current historical user instruction is executed, and the node state as the node attributes of the directed acyclic graph; According to the execution order of the historical user instructions, creating directed edges between the nodes corresponding to the historical user instructions; The preset instruction directed acyclic graph is constructed according to the node labels, node attributes and directed edges.

8. The display device according to claim 1, characterized in that The controller executes the instruction set corresponding to the voice interaction data, and is specifically configured as follows: Converting the voice interaction data into text data based on voice recognition technology; Analyze the text data using natural language processing technology to obtain user intent and entity information; Based on the user intention and the entity information, searching a preset rule library for the first instruction corresponding to the user intention and the entity information; The preset rule base is used to record the mapping relationship between the user intention and the first instruction; The instruction set is determined based on the first instruction.

9. A display device, characterized in that: include: monitor; a communicator configured to communicate with the server; A controller is coupled to the display and the communicator and is configured to: Receive voice interaction data input by the user; The control communicator sends the voice interaction data to the server, so that the server determines the corresponding instruction set according to the voice interaction data, and determines the target execution order between the first instructions of the instruction set based on the preset instruction directed acyclic graph; wherein the preset instruction directed acyclic graph includes a plurality of paths, and the paths represent the execution order of the preset instructions; the preset instructions include the first instruction; the control communicator receives the instruction set and the corresponding target execution order from the server; Execute the corresponding first instructions according to the target execution order.

10. A server, characterized in that: include: A communicator configured to be communicatively connected with a display device; A processor is coupled to the communicator and is configured to: Controlling the communicator to receive voice interaction data from the display device, and generating a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction; Acquire a preset instruction directed acyclic graph, wherein the preset instruction directed acyclic graph includes a plurality of paths, and the paths represent an execution order of the preset instructions; the preset instructions include the first instruction; Determining a target path formed by the at least one first instruction according to the preset instruction directed acyclic graph; determining a target execution order between the at least one first instruction according to the target path; The communicator is controlled to send the instruction set and the target execution order to the display device, so that the display device executes the corresponding first instruction according to the target.

11. A method for executing a voice command, characterized in that: The execution method is applied to a display device, comprising: Receive voice interaction data input by the user; Generate a corresponding instruction set according to the voice interaction data; the instruction set includes at least one first instruction; Acquire a preset instruction directed acyclic graph, wherein the preset instruction directed acyclic graph includes a plurality of paths, and the paths represent an execution order of the preset instructions; the preset instructions include the first instruction; Determining a target path formed by the at least one first instruction according to the preset instruction directed acyclic graph; determining a target execution order between the at least one first instruction according to the target path; Execute the corresponding first instructions according to the target execution order.

Images