Display device and method for guiding user to use voice instruction

Abstract

The invention provides a display device and a method for guiding a user to use a voice instruction. The display device obtains at least one operation sequence when meeting a trigger condition for generating a guide language, screens out at least one target sub-sequence of which the occurrence frequency is greater than or equal to a frequency threshold value from the at least one operation sequence, and generates a guide language according to the correlation among operation events in the at least one target sub-sequence. And screening out at least one target operation event. And the display device generates a guide language corresponding to the at least one target operation event according to the semantic instruction corresponding to each target operation event, and the guide language can be accurately matched with the real operation of the user. In the embodiment of the invention, the display device displays the target page after the final operation event is executed and displays the guide language on the target page, so that a user does not need to strenuously rummage the guide language, and the instantly displayed guide language and the corresponding user operation have relatively strong relevance, so that the interaction feeling of the user can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a method for guiding a user to use voice commands. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, mobile terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] The display device can be configured with a far-field voice function, such as a "voice assistant". Based on the far-field voice function, the display device can respond to the user's voice instructions and perform corresponding tasks, so that the user can interact with the display device more conveniently by voice. The standardization of the voice instructions input by the user is one of the important factors that affect the accuracy of triggering the display device to perform tasks and the display device to determine the tasks to be performed. Therefore, in order to improve the standardization of the voice instructions input by the user, the display device can be configured with a guidance function. Based on the guidance function, the display device can display guidance words to the user to guide the user to learn standard voice instructions through the guidance words.

[0004] After the display device is turned on for the first time, it usually provides the user with guidance words related to the far-field voice function during the operation guidance process of the display device. Alternatively, the display device will display guidance words related to the far-field voice function on the homepage. Alternatively, when the user triggers a specified control, the display device will display guidance words related to the voice command corresponding to the specified control. It can be seen that the scenario of triggering the display device to guide the user to use voice commands is relatively fixed, and the guidance words displayed are also relatively fixed, which cannot adapt well to the scenario where the user actually uses the display device. Summary of the invention

[0005] The present application provides a display device and a method for guiding a user to use voice commands, which can dynamically display matching guiding words based on user operations, so as to guide the user to use the voice commands corresponding to the user operations through the guiding words.

[0006] In a first aspect, the present application provides a display device, including:

[0007] A display configured to: display a user interface;

[0008] The controller is configured to: when a trigger condition for generating a guide statement is met, obtain at least one operation sequence, each of the operation sequences includes at least one operation event, and the operation event corresponds to a non-voice instruction of a user; from a plurality of subsequences included in the at least one operation sequence, screen out at least one target subsequence whose frequency of occurrence in the at least one operation sequence is greater than or equal to a frequency threshold, wherein the subsequence includes at least one operation event; based on the correlation between the operation events in the at least one target subsequence, screen out at least one target operation event; based on the semantic instructions corresponding to each of the target operation events, generate a guide statement corresponding to the at least one target operation event, the guide statement is used to guide the user to use the voice instruction corresponding to the at least one target operation event; control the display to display a target page after executing the final operation event, and display the guide statement on the target page, the final operation event being the last operation event in the last operation sequence in the at least one operation sequence.

[0009] The above technical solution has the following beneficial effects or advantages:

[0010] The display device can dynamically generate corresponding guide words according to the operations performed by the user through non-voice instructions. The guide words are used to guide the user to use the voice instructions corresponding to these operations, and can accurately match the user's actual operations. In addition, the display device can instantly display the guide words on the page displayed after the user's last operation is executed, without the user having to laboriously search for the guide words. Moreover, by instantly displaying the guide words, the display device can enhance the correlation between the displayed guide words and the corresponding user operations, which can effectively improve the user's sense of interaction.

[0011] In some embodiments of the present application, the controller obtains at least one operation sequence when the trigger condition for generating the guide words is met, and is configured to: start the task of generating the guide words when the trigger condition for generating the guide words is met; after receiving the non-voice instruction input by the user, record the operation event corresponding to the non-voice instruction; when the operation event of the specified type is detected, determine the operation event of the specified type as the final operation event; obtain the at least one operation sequence from the recorded operation events, wherein each operation sequence corresponds to a preset time unit, and the at least one operation event included in each operation sequence is arranged in a corresponding recording order, and a preset offset distance is separated between two adjacent operation sequences.

[0012] The above technical solution has the following advantages or beneficial effects:

[0013] The display device can more flexibly obtain an operation sequence that meets the requirements of subsequent generation of guide words by configuring the time unit and the offset distance. The display device can limit the time length of the operation sequence by configuring the time unit and limit the coverage between adjacent operation sequences by configuring the offset distance.

[0014] In some embodiments of the present application, the controller filters out at least one target subsequence whose frequency of appearance in the at least one operation sequence is greater than or equal to a frequency threshold from the multiple subsequences included in the at least one operation sequence, and is configured to: obtain the multiple subsequences included in the at least one operation sequence; calculate the frequency of appearance of each of the multiple subsequences in the at least one operation sequence; and filter out the target subsequence whose frequency of appearance in the at least one operation sequence is greater than or equal to the frequency threshold.

[0015] The above technical solution has the following advantages or beneficial effects:

[0016] The display device can accurately filter out a target subsequence whose frequency meets a frequency threshold from multiple subsequences based on the frequency of each subsequence appearing in each operation sequence, thereby ensuring that the guide language subsequently generated based on the target subsequence can accurately guide the user to use voice instructions corresponding to the series of operations performed.

[0017] In some embodiments of the present application, the controller filters out at least one target operation event based on the correlation between each of the operation events in the at least one target subsequence, and is configured to: obtain semantic features corresponding to each of the operation events in the at least one target subsequence; generate semantic instructions corresponding to the corresponding operation event based on the semantic features corresponding to each of the operation events in the at least one target subsequence; calculate the semantic correlation between the semantic instructions corresponding to the corresponding operation events; and filter out the at least one target operation event based on the semantic correlation between the semantic instructions.

[0018] The above technical solution has the following advantages or beneficial effects:

[0019] The display device can accurately calculate the semantic relevance between the semantic instructions corresponding to each operation event based on the semantic features corresponding to each operation event in the target subsequence, and screen out the operation events corresponding to the semantic instructions whose semantic relevance meets the relevance criteria, namely, the target operation events. These target operation events can reflect the true intention of a series of operations performed by the user, and can enable the subsequent guiding words generated based on the target operation events to accurately guide the user to use the voice instructions corresponding to the series of operations performed.

[0020] In some embodiments of the present application, the controller executes, based on the semantic features corresponding to each of the operation events in the at least one target subsequence, to generate semantic instructions corresponding to the corresponding operation events, and is specifically configured to: obtain a mapping table, the mapping table including a mapping relationship between preset controls and guide words; determine the corresponding target guide word in the mapping table based on the control name in the semantic features corresponding to each of the operation events; determine the user intent based on the semantic features; and process the target guide word based on the user intent to obtain the semantic instructions corresponding to the corresponding operation events.

[0021] The above technical solution has the following advantages or beneficial effects:

[0022] The display device is based on the pre-stored guide words. By processing the guide words, it can not only improve the standardization of the generated semantic instructions, but also effectively improve the efficiency of generating semantic instructions.

[0023] In some embodiments of the present application, the controller generates a guide language corresponding to at least one target operation event based on the semantic instructions corresponding to each target operation event, and is configured to: generate at least one candidate path based on the at least one target operation event, each candidate path including at least one target operation event; determine a target path from the at least one candidate path, the target path containing the least number of target operation events; obtain the semantic instructions corresponding to each target operation event in the target path; and generate the guide language based on the semantic instructions corresponding to each target operation event in the target path.

[0024] The above technical solution has the following advantages or beneficial effects:

[0025] The display device can effectively simplify the number of semantic instructions used to generate guide words by screening out the target path containing the least number of target operation events, and generating guide words according to the semantic instructions corresponding to each target operation event in the target path, thereby effectively simplifying the generated guide words and effectively improving the efficiency of generating guide words.

[0026] In some embodiments of the present application, the controller controls the display to display the target page after executing the final operation event, and displays the guide words on the target page, and is configured to: detect the state of the far-field voice function; if the state of the far-field voice function is on, display the guide words on the target page; if the state of the far-field voice function is off, do not display the guide words on the target page.

[0027] The above technical solution has the following advantages or beneficial effects:

[0028] The display device can determine whether to display the guide words according to the state of the far-field voice function. In this way, the display device can display the guide words when the state of the far-field voice function is turned on, that is, when the far-field voice function is available, so that when the user follows the guide words to learn the corresponding voice instructions, the display device can respond based on the far-field voice function. When the state of the far-field voice function is turned off, that is, when the far-field voice function is not available, the guide words are not displayed to avoid the problem that the display device cannot respond based on the far-field voice function when the user follows the guide words to learn the corresponding voice instructions.

[0029] In some embodiments of the present application, if the state of the far-field voice function is on, the controller executes displaying the guide words on the target page, and is specifically configured as follows: if the state of the far-field voice function is on, obtain a first duration from the moment corresponding to a preset node to the current moment; if the first duration is less than or equal to a duration threshold, display the guide words on the target page; if the first duration is greater than the duration threshold, do not display the guide words on the target page.

[0030] The above technical solution has the following advantages or beneficial effects:

[0031] The display device may no longer display the dynamically generated guide words to the user after a certain period of time, so as to avoid displaying the dynamically generated guide words to the user for a long time, which may affect the user's normal use of the display device.

[0032] In some embodiments of the present application, the controller generates the guide language based on the semantic instructions corresponding to each of the target operation events in the target path, and is also configured to: determine the target control corresponding to the final operation event; create a mapping relationship between the target control and the guide language; and update the mapping relationship to the mapping table.

[0033] The above technical solution has the following advantages or beneficial effects:

[0034] The display device can create a mapping relationship between the guide words and the target control and expand the mapping table so that the guide words can be directly displayed when the user triggers the target control again. In addition, a richer guide word basis can be provided when the guide words of other operation events are dynamically generated.

[0035] In a second aspect, the present application also provides a method for guiding a user to use voice instructions, the method comprising: when a trigger condition for generating a guide statement is met, obtaining at least one operation sequence, each of the operation sequence comprising at least one operation event, and the operation event corresponding to a non-voice instruction of the user; from a plurality of subsequences included in the at least one operation sequence, screening out at least one target subsequence whose frequency of appearance in the at least one operation sequence is greater than or equal to a frequency threshold, wherein the subsequence comprises at least one operation event; based on the correlation between the operation events in the at least one target subsequence, screening out at least one target operation event; based on the semantic instructions corresponding to each of the target operation events, generating a guide statement corresponding to the at least one target operation event, the guide statement being used to guide the user to use the voice instruction corresponding to the at least one target operation event; displaying a target page after executing the final operation event, and displaying the guide statement on the target page, the final operation event being the last operation event in the last operation sequence in the at least one operation sequence.

[0036] The above technical solution has the following advantages or beneficial effects:

[0037] The display device can dynamically generate corresponding guide words according to the operations performed by the user through non-voice instructions. The guide words are used to guide the user to use the voice instructions corresponding to these operations, and can accurately match the user's actual operations. In addition, the display device can instantly display the guide words on the page displayed after the user's last operation is executed, without the user having to laboriously search for the guide words. Moreover, by instantly displaying the guide words, the display device can enhance the correlation between the displayed guide words and the corresponding user operations, which can effectively improve the user's sense of interaction.

[0038] In the technical solution provided by the embodiment of the present application, when the trigger condition for generating the guide language is met, the display device obtains at least one operation sequence, each operation sequence includes at least one operation event, and the operation event corresponds to the non-voice instruction of the user. The display device screens out at least one target subsequence whose frequency of appearance in at least one operation sequence is greater than or equal to the frequency threshold from the multiple subsequences included in the at least one operation sequence, and screens out at least one target operation event according to the correlation between the operation events in the at least one target subsequence. The display device generates the guide language corresponding to the at least one target operation event according to the semantic instructions corresponding to each target operation event, and the guide language can be accurately matched with the user's real operation. The display device displays the target page after executing the final operation event, and displays the guide language on the target page, without the user having to laboriously search for the guide language, and the instantly displayed guide language has a strong correlation with the corresponding user operation, which can effectively improve the user's sense of interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0040] Figure 1 A schematic diagram of an operation scenario between a display device 200 and a control device 100 in an embodiment of the present application;

[0041] Figure 2 This is a hardware configuration block diagram of the control device 100 in an embodiment of the present application;

[0042] Figure 3 It is a hardware configuration block diagram of the display device 200 in the embodiment of the present application;

[0043] Figure 4 This is a configuration diagram of an operating system of a display device 200 in an embodiment of the present application;

[0044] Figure 5 A flow chart of a display device 200 provided in an embodiment of the present application guiding a user to use voice commands;

[0045] Figure 6 A flowchart of obtaining an operation sequence for the display device 200 provided in an embodiment of the present application;

[0046] Figure 7 A timing diagram of the display device 200 provided in an embodiment of the present application guiding a user to use voice commands;

[0047] Figure 8 A flowchart of the display device 200 screening target operation events provided by an embodiment of the present application;

[0048] Fig. 9 A flow chart of generating a guide message for the display device 200 provided in an embodiment of the present application;

[0049] Fig.10 A schematic diagram of a path to be selected provided in an embodiment of the present application;

[0050] Fig.11 A flow chart of displaying a guide message on a display device 200 provided in an embodiment of the present application;

[0051] Fig.12 Another flow chart of the display device 200 displaying the guide words provided in the embodiment of the present application;

[0052] Fig.13 The user operation process in scenario 1 provided in the embodiment of the present application;

[0053] Fig.14 A schematic diagram of an interface for displaying a guide message on a target page provided in an embodiment of the present application;

[0054] Fig.15 The user operation process in scenario 2 provided in the embodiment of the present application;

[0055] Fig.16 The user operation process in scenario 3 provided in the embodiment of the present application;

[0056] Fig.17 The user operation process in scenario 4 provided in the embodiment of the present application;

[0057] Fig.18 This is the user operation process under scenario 5 provided in the embodiment of the present application. DETAILED DESCRIPTION

[0058] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0059] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0060] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0061] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0062] In the embodiments of the present application, the display device generally refers to a device with image display and data processing capabilities. For example, the display device includes but is not limited to smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0063] Figure 1This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, voice, mobile terminal 300 and control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0064] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0065] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0066] Figure 2 Some embodiments of the present application provide Figure 1 1 is a block diagram of the hardware configuration of the control device 100.

[0067] like Figure 2 As shown, the control device 100 may include: a controller 110 , a communication interface 120 , a user input / output interface 130 , a memory 140 , and a power supply 150 .

[0068] The controller 110 may include a processor 111 , a RAM (Random Access Memory) 112 , a ROM (Read-Only Memory) 113 , and a communication bus (Bus).

[0069] The communication interface 120 may include at least one of a WiFi chip 121 , a Bluetooth module 122 , an NFC module 123 and other near field communication modules.

[0070] The user input / output interface 130 , wherein the input interface may include at least one of other input interfaces such as a microphone 131 , a touch panel 132 , a sensor 133 , and a button 134 .

[0071] In some embodiments, the control device 100 may include at least one of a communication interface 120 and an input / output interface 130 .

[0072] The memory 140 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller 110. The memory 140 can store various control signal instructions input by the user.

[0073] The power supply 150 is used to provide operating power support for various components of the control device 100 under the control of the controller 110 .

[0074] Figure 3 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0075] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a user input interface 280, a memory, and a power supply.

[0076] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. The communication device 220 may enable the display device 200 to communicate with an external device or server 400 by wireless or wired connection.

[0077] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0078] In some embodiments, the device interface 240 is used to access an external device.

[0079] In some embodiments, the controller 250 is used to control the overall operation of the display device 200. The controller 250 may include at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), and a power processor, and first to nth interfaces for input / output. The controller 250 controls the operation of the display device 200 and responds to user operations through various software control programs stored in the memory.

[0080] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0081] In some embodiments, the display 260 is used to receive and display image signals outputted from the controller 250. The display 260 may include a display function component for presenting images, and a driving component for driving image display.

[0082] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface 280 may receive the user command through the GUI.

[0083] In some embodiments, the audio output device 270 may be a local speaker of the display device 200 , or may be an external audio output device of the display device 200 .

[0084] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0085] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200.

[0086] The operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device 200 .

[0087] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system can be divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system runtime layer and the kernel layer.

[0088] In some embodiments, the application layer is used to provide services and interfaces for the application so that the display device 200 can run the application and interact with the user based on the application. For example, the application layer may include a voice assistant, which provides a voice interaction function, and the user can interact with the display device through the voice assistant in a voice manner.

[0089] The framework layer can provide application programming interfaces (API) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center, which determines the actions of applications in the application layer. Applications can access system resources and obtain system services during execution through the API interface.

[0090] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0091] In some embodiments, the kernel layer is a functional level between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management.

[0092] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0093] The display device 200 may be configured with a far-field voice function, such as a "voice assistant". Based on the far-field voice function, the display device 200 may perform corresponding tasks in response to the user's voice instructions, so that the user can interact with the display device 200 more conveniently by voice. The standardization of the voice instructions input by the user is one of the important factors that affect the accuracy of triggering the display device 200 to perform tasks and the display device 200 determining the tasks to be performed. Therefore, in order to improve the standardization of the voice instructions input by the user, the display device 200 may be configured with a guidance function. Based on the guidance function, the display device 200 may display a guidance statement to the user to guide the user to learn standard voice instructions through the guidance statement.

[0094] The display device 200 usually provides the user with guidance words related to the far-field voice function during the operation guidance process of the display device 200 after the first power-on. Alternatively, the display device 200 may display guidance words related to the far-field voice function on the homepage. Alternatively, when the user triggers a specified control, the display device 200 may display guidance words related to the voice instruction corresponding to the specified control, wherein the specified control is a page control pre-specified by the display device 200 before leaving the factory. It can be seen that the scenario of triggering the display device 200 to guide the user to use voice instructions is relatively fixed, and the guidance words displayed are also relatively fixed, which cannot adapt well to the scenario where the user actually uses the display device 200.

[0095] In order to solve the above problems, the display device 200 provided by the present application is configured to dynamically generate matching guide words according to the actual operation performed by the user, so as to improve the matching degree between the guide words and the scene in which the user actually uses the display device 200. In addition, the guide words can be displayed to the user in real time, without the user having to search for the guide words on his own, and the guide words displayed in real time have a strong correlation with the corresponding operation of the user, which can effectively improve the interactive feeling between the user and the display device 200.

[0096] The display device 200 provided in the present application is schematically described in conjunction with the following embodiments:

[0097] Figure 5 The flowchart of the display device guiding the user to use voice commands provided in the embodiment of the present application, the specific steps are as follows:

[0098] Step S501: when a trigger condition for generating a guide word is met, at least one operation sequence is obtained.

[0099] The display device 200 may be configured with an introductory language generation function. If the introductory language generation function is in an on state, the display device 200 starts the task of generating introductory language; if the introductory language generation function is in an off state, the display device 200 does not start the task of generating introductory language. In some embodiments, the display device 200 may be configured to set the introductory language generation function to an on state by default.

[0100] When the state of the guide word generation function is turned on, the display device 200 monitors whether the trigger condition for generating the guide word is met. For example, the trigger condition may include monitoring a power-on instruction, the completion of the task of generating the guide word last time, detecting that the current page is a specific page (such as the homepage of the display device 200), and monitoring a specific instruction (such as an instruction to return to the homepage) in the power-on state, or a combination of several of them. Of course, the trigger condition is not limited to the specific examples disclosed above.

[0101] If the display device 200 does not monitor that the trigger condition for generating the guide word is met, the display device 200 does not start the task of generating the guide word. If the display device 200 monitors that the trigger condition for generating the guide word is met, the display device 200 starts the task of generating the guide word.

[0102] When the display device 200 monitors that the trigger condition for generating the guide language is met and starts the task of generating the guide language, the display device 200 obtains at least one operation sequence. Each operation sequence includes at least one operation event (represented by Do in the embodiment of the present application), and the operation event corresponds to the user's non-voice instruction, that is, the operation event corresponds to the operation performed by the user in a non-voice manner. For example, the operation performed by the user may include an operation of controlling the display device 200 through the control device 100, such as controlling the display device 200 by operating (for example, long pressing, short pressing, double-clicking, etc.) buttons (for example, direction keys, confirmation keys, back keys, etc.) on the control device 100 (for example, moving the focus on the user interface, selecting a page control in the user interface, adjusting system parameters, etc.).

[0103] In some examples, the operation sequence D can be expressed as D = {Do1, Do2, ... Do n}, the operation sequence includes n operation events, where n is an integer greater than or equal to 1.

[0104] In some examples, at least one operation sequence acquired by the display device 200 may also be referred to as an operation sequence set DT, which may be expressed as DT={D1, D2, ... D m}, the operation sequence set includes m operation sequences, where m is an integer greater than or equal to 1.

[0105] Figure 6 The flowchart of the display device acquisition operation sequence provided in the embodiment of the present application, the specific steps are as follows:

[0106] Step S601: When a trigger condition for generating an introductory statement is met, a task for generating an introductory statement is started.

[0107] Figure 7 A timing diagram of a display device guiding a user to use voice commands provided in an embodiment of the present application.

[0108] Combination Figure 7 The display device 200 may be configured with a behavior pattern mining module, and when the behavior pattern mining module monitors that a trigger condition for generating a guide word is met, the task of generating a guide word is controlled to be started.

[0109] Step S602: After receiving the non-voice instruction input by the user, record the operation event corresponding to the non-voice instruction.

[0110] Combination Figure 7 When starting the task of generating a guide statement, if the display device 200 receives a non-voice instruction input by the user, for example, a non-voice instruction input by the user through the control device 100 or the mobile terminal 300, the operation events corresponding to these non-voice instructions can be recorded through the behavior pattern mining module.

[0111] An operation event may include elements related to the corresponding operation. In some embodiments, an operation event may include three elements, such as an event (indicated by o in the embodiment of the present application), and a device state of the display device 200 before and after responding to the corresponding operation (indicated by s in the embodiment of the present application).

[0112] In one example, any action event Do i The three elements included i , o i ,s i+1 ), where Do i Indicates that the operation event is the i-th operation event in the operation sequence to which it belongs. i Represents the event elements included in the i-th operation event, s i represents the device state of the display device 200 before executing the i-th operation event, s i+1 Represents the device state of the display device 200 after executing the i-th operation event.

[0113] The event may include an event type and an attribute (attr) of a control that interacts with the user.

[0114] Event types correspond to actions performed by users, such as click, exit, and so on.

[0115] The properties of a control may include the control ID, the text corresponding to the control, the content description, the hint, and the image file name corresponding to the control. The ID of the control may indicate the function corresponding to the control; the text corresponding to the control is the name of the control displayed on the corresponding page; the content description is the remark of the control, which is used to explain the function of the control; the hint is the information displayed on the page when the hint function of the corresponding control is triggered; the image file name corresponds to the page control that displays the image content, and is the file name of the image content to be displayed by the page control.

[0116] The device status may include the activity component name and UI interface hierarchy result of the page. The activity component name may indicate the main function provided by the corresponding page. The UI interface hierarchy result indicates the interface hierarchy to which the activity component belongs.

[0117] Step S603: when an operation event of a specified type is detected, determining the operation event of the specified type as a final operation event.

[0118] The user will achieve an operation goal by performing a series of operations. For example, the user controls the display device 200 to play media A by performing a series of operations, so playing media A is the user's operation goal. The display device 200 needs to generate corresponding guide words for a series of operations corresponding to each operation goal, so that the guide words can accurately match the operations performed by the user when achieving different operation goals, thereby accurately guiding the user to use the corresponding voice instructions based on the user's operation purpose. Based on this, the display device 200 needs to accurately obtain the operation events corresponding to the same operation goal.

[0119] In some embodiments, at least one type of operation event can be preset, and the at least one type of operation event has a high probability of corresponding to the operation target. The at least one type of operation event can also be called a specified type of operation event, wherein the specified type of operation event can include switching (e.g., opening, returning, etc.) to a specified page, setting system parameters, etc.

[0120] Combination Figure 7 When the behavior pattern mining module obtains an operation event corresponding to the same operation target, it can determine whether the type of the obtained operation event is the specified type based on the matching result between the type of the obtained operation event and the specified type. For example, the behavior pattern mining module can obtain the device state after the operation event is executed, such as the name of the activity component after the operation event is executed, and match the activity component name with the pre-set activity component name. If the matching result is a match, it can be determined that the specified type of operation event is detected; if the matching result is a mismatch, it can be determined that the specified type of operation event is not detected.

[0121] When the behavior pattern mining module detects an operation event of a specified type, it can determine that after executing the operation event of the specified type, the user's operation goal can be achieved, that is, the operation event of the specified type is the last operation event used to achieve the operation goal, and can also be called the final operation event corresponding to the operation goal.

[0122] In some embodiments, the behavior pattern mining module may mark the detected final operation events, so that in subsequent processing, the operation events corresponding to different operation targets may be accurately distinguished based on these marks.

[0123] Step S604: obtaining at least one operation sequence from the recorded operation events, wherein each operation sequence corresponds to a preset time unit, and at least one operation event included in each operation sequence is arranged in a corresponding recording order, and a preset offset distance is spaced between two adjacent operation sequences.

[0124] In step S604, the recorded operation events refer to operation events corresponding to the same operation target. The first operation event in the recorded operation events may be the first operation event recorded after the task of generating the guide words is started, or, if there is an operation event corresponding to the previous operation target, the first operation event in the recorded operation events may be the first operation event recorded after the final operation event corresponding to the previous operation target.

[0125] A time window mechanism can be used to obtain at least one operation sequence from the recorded operation events. In step S602, when recording the operation events, the behavior pattern mining module will obtain the corresponding recording order of each operation event. Each operation sequence includes at least one recorded operation event, which are continuously recorded operation events and are sorted in the operation sequence according to the corresponding recording order, so that the correctness of the execution order of the corresponding operation can be guaranteed. The last operation event in the last operation sequence is the final operation event corresponding to the current operation target.

[0126] Each operation sequence corresponds to a preset time unit, and the duration of the preset time unit can be adjusted by the user. In this way, the time length of the operation sequence can be limited by the time unit, thereby controlling the number of operation events included in the operation sequence.

[0127] There is a preset offset distance between two adjacent operation sequences. If the offset distance is small, the coverage between the two adjacent operation sequences is high, that is, the two adjacent operation sequences will include more identical operation events; if the offset distance is large, the coverage between the two adjacent operation sequences is low, that is, the two adjacent operation sequences will include fewer identical operation events, or even no identical operation events. Therefore, the behavior pattern mining module can limit the coverage between adjacent operation sequences by flexibly configuring the offset distance.

[0128] In some embodiments, the offset distance may be a time distance, i.e., a time length, that is, in two adjacent operation sequences, a preset time distance is defined between the start time of a time unit corresponding to a previous operation sequence and the start time of a time unit corresponding to a subsequent operation sequence.

[0129] In some embodiments, the offset distance may be a numerical distance, that is, a number. That is, in two adjacent operation sequences, the first operation event included in the previous operation sequence and the first operation event included in the next operation sequence are separated by a preset numerical distance.

[0130] Step S502: Filter out, from a plurality of subsequences included in at least one operation sequence, at least one target subsequence whose frequency of appearance in at least one operation sequence is greater than or equal to a frequency threshold.

[0131] Each operation sequence includes at least one subsequence α, which can also be called a behavior sequence. Each subsequence includes at least one operation event belonging to the same operation sequence. The at least one operation event included in the subsequence can be an operation event recorded continuously or discontinuously.

[0132] In one example, a subsequence can be represented as α={Do i ,Do i+1 ,…Do k}, where i≥1 and k≤n.

[0133] The frequency of a subsequence appearing in the entire operation sequence can be used to represent the frequency with which a user executes the operations corresponding to the operation events in the subsequence. Therefore, the higher the frequency of a subsequence, the more frequently a user executes the operations corresponding to the operation events in the subsequence; the lower the frequency of a subsequence, the less frequently a user executes the operations corresponding to the operation events in the subsequence.

[0134] In some embodiments, the frequency of occurrence may be represented by the support of a subsequence, where the support of a subsequence refers to the total number of the subsequences included in the operation sequence set.

[0135] In an example, the support of a subsequence can be expressed as: Here, |DT| represents the number of operation sequences in DT.

[0136] The frequency of user operations can reflect the user's operation characteristics, preferences and other characteristics, and can also be called the user's behavior pattern. It can be understood that the higher the frequency of user operations, the more consistent with the user's behavior pattern; the lower the frequency of user operations, the more deviated from the user's behavior pattern. By setting a frequency threshold, subsequences that are more consistent with the user's behavior pattern can be screened out. Among them, the subsequences that appear in each subsequence with a frequency greater than or equal to the frequency threshold can be determined as target subsequences, and the target subsequences can also be called frequent subsequences. The set formed by each target subsequence can be called a frequent subsequence set, and the operations corresponding to each operation event in the target subsequence are consistent with the user's behavior pattern.

[0137] In an example, the frequent subsequence set FS can be expressed as: FS={α|support(α)≥min_sup}, where min_sup represents a frequency threshold.

[0138] The frequent subsequence set can be processed to ensure that there is no inclusion relationship between the target subsequences. The processed frequent subsequence set can be called a behavior pattern set.

[0139] In one example, the behavior pattern set PS can be expressed as:

[0140] The guide words generated based on the target subsequence in the behavior pattern set can effectively improve the matching degree between the guide words and the user's behavior pattern.

[0141] Step S503: Screening out at least one target operation event according to the correlation between the operation events in at least one target subsequence.

[0142] Among the multiple operations performed by the user, there may be operations that are less relevant to the operation target, and these operations that are less relevant to the operation target may affect the accuracy of the subsequently generated guide words. In order to improve the accuracy of the generated guide words, the display device 200 may filter out target operation events that are more relevant to the operation target (i.e., the final operation event) from each target subsequence, so as to subsequently generate guide words based on these target operation events.

[0143] Figure 8 The flowchart of the display device screening target operation events provided in the embodiment of the present application has the following specific steps:

[0144] Step S801: Acquire semantic features corresponding to each operation event in at least one target subsequence.

[0145] Combination Figure 7 The display device 200 may be configured with a semantic event matching module, through which a target operation event with a strong correlation with the operation target (ie, the final operation event) may be screened out from each target subsequence.

[0146] The behavior pattern mining module can pass the filtered target subsequence to the semantic event matching module, and the semantic event matching module obtains the semantic features corresponding to each operation event in the target subsequence.

[0147] The semantic features corresponding to the operation event may include the name of the active component included in the operation event and the attributes of the controls in the event elements included in the operation event. That is to say, the semantic features corresponding to the operation event are associated with the pages displayed by the display device 200 in response to the user's operation, the controls that the user interacts with when performing the operation, etc.

[0148] Step S802: Generate a semantic instruction corresponding to each operation event in at least one target subsequence according to the semantic feature corresponding to each operation event.

[0149] Combination Figure 7 The semantic event matching module can determine the user intent corresponding to the operation event according to the semantic features, and then according to the user intent. In some examples, the semantic event matching module can determine the user intent according to the semantic features based on the natural language processing (Natural Language Processing, NLP) algorithm.

[0150] In some embodiments, the memory 140 of the display device 200 may store a mapping table (Map), which includes a mapping relationship between the name of the page control and the guide word. The mapping relationship includes a first mapping relationship between the name of the page control pre-configured by the display device 200 and the corresponding guide word. The mapping relationship may also include a second mapping relationship between the guide word and the corresponding page control obtained by executing the task of generating the guide word.

[0151] The semantic event matching module can determine the corresponding target guide word from the mapping table according to the name of the control in the semantic feature. The semantic event matching module can process the target guide word according to the user's intention and obtain a semantic instruction that can meet the user's intention. For example, some words in the target guide word can be replaced with words that meet the user's intention, or words that meet the user's intention can be added to the target guide word.

[0152] In this way, based on the pre-stored guide words, by processing the guide words, not only the standardization of the generated semantic instructions can be improved, but also the efficiency of generating semantic instructions can be effectively improved.

[0153] Step S803: Calculate the semantic correlation between the semantic instructions corresponding to the corresponding operation events.

[0154] All operation events other than the final operation event may also be referred to as non-final operation events. The non-final operation event corresponds to the operation process of the user achieving the operation goal.

[0155] The semantic event matching module can determine the correlation between each operation and the operation target in the operation process of the user achieving the operation target by calculating the semantic correlation between the semantic instructions corresponding to each non-final operation event and the semantic instructions corresponding to the final operation event.

[0156] The semantic event matching module can determine the correlation between the operations in the operation process of the user achieving the operation goal by calculating the semantic correlation between the semantic instructions corresponding to the non-final operation events.

[0157] Combination Figure 7 , the semantic correlation between the semantic instructions is calculated by the semantic event matching module, and the semantic correlation between the semantic instructions can reflect the correlation between the corresponding operation events. In some examples, the semantic event matching module can calculate the semantic distance between the semantic instructions corresponding to the corresponding operation events based on the semantic features, and the semantic distance is used to characterize the semantic correlation between the semantic instructions. Among them, the closer the semantic distance, the higher the semantic correlation between the semantic instructions; the farther the semantic distance, the lower the semantic correlation between the semantic instructions.

[0158] Step S804: Filter out at least one target operation event according to the semantic relevance between the semantic instructions.

[0159] A correlation threshold is set in advance to filter target operation events by the correlation threshold. By adjusting the correlation threshold, the quantity and quality of the filtered target operation events can be controlled. The higher the correlation threshold, the smaller the number of filtered target operation events and the higher the quality; the lower the correlation threshold, the larger the number of filtered target operation events and the lower the quality.

[0160] Combination Figure 7 , the target operation event can be screened out by the semantic event matching module according to the correlation threshold and the semantic correlation. Among them, the operation event corresponding to the semantic instruction whose semantic correlation is greater than or equal to the correlation threshold is determined as the target operation event.

[0161] Step S504: generating at least one guide corresponding to the target operation event according to the semantic instruction corresponding to each target operation event.

[0162] The guiding words are used to guide the user to use the voice instruction corresponding to the at least one target operation event.

[0163] In some embodiments, before generating the guide language, the display device 200 may process the at least one target operation event to reduce the number of target operation events and further simplify the corresponding guide language.

[0164] In one example, the display device 200 can determine meaningless operation events in the at least one target operation event and remove these meaningless operation events to reduce the number of target operation events. In some embodiments, the display device 200 can splice the semantic instructions corresponding to the at least one target operation event according to the record order corresponding to the at least one target operation event to obtain a guide.

[0165] In some embodiments, the display device 200 may process the at least one target operation event, and splice the semantic instructions corresponding to the processed target operation event according to the record order corresponding to the processed target operation event to obtain the guide language. In this way, the guide language can be effectively simplified and the readability of the guide language can be improved.

[0166] Fig. 9 The flowchart of generating a guide message for a display device provided in an embodiment of the present application, the specific steps are as follows:

[0167] Step S901: generating at least one candidate path according to at least one target operation event, each candidate path including at least one target operation event.

[0168] The target operation events included in each of the candidate paths are not completely the same, so the guide words corresponding to each of the candidate paths are different. Wherein, at least one target operation event included in each of the candidate paths is arranged in the order of recording.

[0169] Combination Figure 7 The display device 200 may be configured with a guide language generation module, which may generate a candidate path according to the target operation event screened by the semantic event matching module.

[0170] Executing the operation events included in each of the selected paths can switch the display device 200 from the same initial device state to the same final device state. The initial device state is the device state of the display device 200 before executing the first operation event in the task of generating the guide words, and the final device state is the device state of the display device 200 after executing the final operation event.

[0171] In some embodiments, the guide generation module can generate a candidate path through a corresponding path model, wherein the starting device state and the final device state can be set for the path model, and each target operation event can be used as input data to output the candidate path through the path model.

[0172] Fig.10A schematic diagram of the paths to be selected is provided in an embodiment of the present application. Taking the target operation events including O0~O7, the initial device state is S0, and the final device state is S6 as an example, according to the correlation between the target operation events, three paths to be selected can be generated, namely O0→O1→O2→O3→O4, O0→O5→O4, and O6→O7.

[0173] Step S902: determining a target path from at least one candidate path, where the target path contains the least number of target operation events.

[0174] The number of target operation events included in the selected path can represent the degree of simplicity of the guide language corresponding to the selected path, wherein the fewer the number of target operation events included in the selected path, the more simplified the corresponding guide language. In order to improve the readability of the guide language, the guide language with the highest degree of simplicity is selected to be displayed to the user, that is, the selected path with the least number of target operation events is selected as the target path, and the guide language is generated based on the target path.

[0175] Combination Fig.10 ,The candidate path O6→O7 contains the least number of target operation events and can be determined as the target path.

[0176] Step S903: Acquire the semantic instructions corresponding to each target operation event in the target path.

[0177] The semantic instructions corresponding to each target operation event in the target path are used to generate the guide words.

[0178] Step S904: Generate guiding words according to the semantic instructions corresponding to each target operation event in the target path.

[0179] In some embodiments, the display device 200 may splice corresponding semantic instructions according to the order of records corresponding to each target operation event in the target path to obtain a guide statement.

[0180] Step S505: displaying the target page after the final operation event is executed, and displaying a guide on the target page.

[0181] Combination Figure 7 The display device 200 may be configured with a voice terminal display module, through which the guide words generated by the guide word generation module are displayed.

[0182] In some embodiments, after generating the guide words, the guide word generation module writes the guide words into the guide word database, that is, updates the guide word database. The voice terminal display module can monitor the data status of the guide word database. If the voice terminal display module monitors that the data status of the guide word database is updated data, the updated data, that is, the guide words, are read from the guide word database. The voice terminal display module determines that the updated data is the guide words that need to be displayed, that is, determines to display the guide words.

[0183] Fig.11 The flowchart of the display device displaying the guide message provided in the embodiment of the present application has the following specific steps:

[0184] Step S1101: Detect the status of the far-field voice function.

[0185] Combination Figure 7 Before displaying the guide words, the display device 200 detects the state of the far-field voice function through the voice terminal display module. The far-field voice function includes an on state and an off state. In the on state, the far-field voice function can be used normally, that is, the user can perform voice interaction with the display device 200 based on the far-field voice function; in the off state, the far-field voice function cannot be used, that is, the user cannot perform voice interaction with the display device 200 based on the far-field voice function.

[0186] Step S1102: If the far-field voice function is in an on state, a guide message is displayed on the target page.

[0187] When the voice terminal display module detects that the far-field voice function is turned on, it adds a display guide on the target page.

[0188] In this way, it can be ensured that when the user follows the guidance language to learn the corresponding voice instructions, the display device 200 can respond based on the far-field voice function in the turned-on state and interact with the user.

[0189] Step S1103: If the far-field voice function is in an off state, no guide words are displayed on the target page.

[0190] When the voice terminal display module detects that the far-field voice function is in a disabled state, the guide message is not displayed on the target page.

[0191] In this way, it is possible to avoid the problem that the display device 200 is unable to respond when the user follows the guidance language to learn the corresponding voice instructions due to the unavailability of the far-field voice function.

[0192] Fig.12 Another flowchart of a display device displaying a guide message provided in an embodiment of the present application, the specific steps are as follows:

[0193] Step S1201: If the far-field voice function is in the on state, obtain the first duration from the time corresponding to the preset node to the current time.

[0194] In some embodiments, the preset node may be a task for starting or triggering the generation of the guide words. The current moment may be the moment of generating the guide words, or the moment when the task of generating the guide words is completed.

[0195] After the preset node is detected, a timer can be started and timed by the timer. After the generation of the introductory words is detected or the task of generating the introductory words is detected to be completed, the duration currently counted by the timer is read, and the duration is the first duration.

[0196] The display device 200 may be set with a duration threshold to control the task of displaying the guide words through the duration threshold. In this way, the dynamically generated guide words may no longer be displayed to the user after a certain time has passed, so as to avoid displaying the dynamically generated guide words to the user at any time for a long time, which may affect the user's normal use of the display device 200.

[0197] Step S1202: If the first duration is less than or equal to the duration threshold, a guide message is displayed on the target page.

[0198] Combination Figure 7 If the voice terminal display module detects that the first duration is less than or equal to the duration threshold, it means that the duration of displaying the dynamically generated introductory words to the user is within a reasonable duration range, and the introductory words can be displayed to the user after the introductory words are dynamically generated. Therefore, the voice terminal display module confirms the task of triggering the display of the introductory words, and displays the introductory words on the target page after the task of triggering the display of the introductory words is triggered.

[0199] Step S1203: If the first duration is greater than the duration threshold, the guide statement is not displayed on the target page.

[0200] Combination Figure 7 If the voice terminal display module detects that the first duration is greater than the duration threshold, it means that the duration of displaying the dynamically generated guide words to the user has exceeded the reasonable duration range. At this time, even if the guide words have been dynamically generated, the guide words cannot be displayed to the user to avoid affecting the user's normal use of the display device 200. Therefore, the voice terminal display module confirms that the task of displaying the guide words is not triggered, that is, the guide words are not displayed on the target page.

[0201] Taking the control device 100 as a remote control and the display device 200 as a television as an example, examples of the display device 200 guiding the user to use voice commands in different scenarios are given.

[0202] In scenario 1, the guidance words generated by the TV can guide the user to use multiple voice commands at one time to control the TV to perform multiple tasks.

[0203] Combination Fig.13 The operation events corresponding to the user operation may include: when the TV is playing media A, in response to the instruction input by the user based on the back key of the remote control, the TV exits the playing of media A and displays the TV homepage; the TV plays media B in response to the instruction input by the user based on the control of media B in the TV homepage through the confirmation key of the remote control; the TV displays the setting menu in response to the instruction input by the user based on the setting key of the remote control; the TV turns on the eye protection mode in response to the instruction input by the user based on the control of the eye protection mode in the setting menu through the confirmation key of the remote control.

[0204] The TV can generate corresponding guidance words according to the above operation events. The process can be referred to in the previous text and will not be repeated here. The generated guidance words can be: The remote control operation is too slow, you can say to me: "××, turn off media A; then play media B; turn on eye protection mode". The final operation event in the above operation events is to turn on the eye protection mode. After turning on the eye protection mode, the TV can display the following Fig.14 When the TV detects that the conditions for displaying the introductory message are met, it can Fig.14 As shown, the guide words are displayed on the target page, for example, a guide frame 1401 is displayed on the upper layer of the target page, and the guide words are displayed in the guide frame 1401. In this way, the user can follow the guide words to learn the voice instructions of the corresponding user operation.

[0205] In scenario 2, the guide words generated by the TV can guide the user to use voice commands to control the TV to quickly start playing media resources.

[0206] Combination Fig.15 The operation events corresponding to the user operation may include: when the TV homepage is displayed, the TV displays a TV series list in response to an instruction input by the user through the confirmation key of the remote control based on the TV series control; the TV displays the details page of TV series A in response to an instruction input by the user through the confirmation key of the remote control based on the control of TV series A in the TV series list; the display device 200 plays the 6th episode of TV series A in response to an instruction input by the user through the confirmation key of the remote control based on the control of the 6th episode in the details page of TV series A; the display device 200 fast-forwards the 6th episode of TV series A to the 3rd minute in response to an instruction input by the user through the right key of the remote control.

[0207] The TV can generate corresponding guide words according to the above operation events. The process can be referred to in the previous text and will not be described here. The generated guide words can be: Start playing the film and television content directly, try to say to me: "××, TV series A, episode 6, minute 3". The final operation event in the above operation events is to fast forward the 6th episode of TV series A to the 3rd minute. After fast forwarding the 6th episode of TV series A to the 3rd minute, the TV displays the target page of the 6th episode of TV series A, minute 3. When the TV detects that the conditions for displaying the guide words are met, the guide words can be displayed on the target page. The display format can refer to Fig.14 , which will not be elaborated here.

[0208] In scenario 3, the guidance words generated by the TV can guide the user to use voice commands to control the TV to play music or sing through the TV.

[0209] Combination Fig.16 The operation events corresponding to the user operation may include: when the TV displays the TV homepage, in response to the instruction input by the user through the confirmation key of the remote control based on the music control, the TV displays the music search page; in response to the instruction input by the user through the confirmation key of the remote control based on the search control in the music search page, the TV displays the search result page of song A according to the search term input by the user; the display device 200 plays song A in response to the instruction input by the user through the confirmation key of the remote control based on the play option in the search result page; the display device 200 starts the karaoke function in response to the instruction input by the user through the down key of the remote control.

[0210] The TV can generate corresponding guide words according to the above operation events. The process can be referred to in the previous text and will not be repeated here. The generated guide words can be: To experience the karaoke function, you can say to me: "××, I want to sing song A". The final operation event in the above operation events is to start the karaoke function. After the TV starts the karaoke function, it displays the song playback page in the karaoke state, which is the target page. When the TV detects that the conditions for displaying the guide words are met, it can display the guide words on the target page. The display format can refer to Fig.14 , which will not be elaborated here.

[0211] In scenario 4, the guide words generated by the TV can guide the user to use voice commands to control the playback of live programs.

[0212] Combination Fig.17, the operation events corresponding to the user operation may include: when the TV displays the signal source page, in response to the instruction input by the user through the confirmation key of the remote control based on the High Definition Multimedia Interface (HDMI) control, the TV displays the set-top box live page; in response to the instruction input by the user through the menu key of the remote control, the TV displays the channel list in the set-top box live page; the display device 200 switches to playing channel A in response to the instruction input by the user through the confirmation key of the remote control based on the control of channel A in the channel list.

[0213] The TV can generate corresponding guide words according to the above operation events. The process can be referred to in the previous text and will not be repeated here. The generated guide words can be: Watch live programs, and next time say to me: "××, watch channel A". The final operation event in the above operation events is switching to playback channel A. After switching to playback channel A, the TV displays the target page of playback channel A. When the TV detects that the conditions for displaying the guide words are met, the guide words can be displayed on the target page. The display format can refer to Fig.14 , which will not be elaborated here.

[0214] In scenario 5, the guide language generated by the TV can guide the user to use voice commands to control content search in third-party applications.

[0215] Combination Fig.18 , the operation events corresponding to the user operation may include: when the TV displays the "My Applications" page, in response to the instruction input by the user through the confirmation key of the remote control based on the control of application A, the TV displays the homepage of application A; in response to the instruction of the user searching for TV series B through the remote control in the homepage of application A, the TV displays the details page of TV series B; the display device 200 plays the 12th episode of TV series B in response to the instruction input by the user through the confirmation key of the remote control based on the control of the 12th episode in the details page of TV series B.

[0216] The TV can generate corresponding guide words according to the above operation events. The process can be referred to in the previous text and will not be repeated here. The generated guide words can be: Play the video of application A, you can say to me: "××, application A plays the 12th episode of TV series B". The final operation event in the above operation events is to play the 12th episode of TV series B. After the TV plays the 12th episode of TV series B, it displays the target page of the 12th episode of TV series B. When the TV detects that the conditions for displaying the guide words are met, it can display the guide words on the target page. The display format can refer to Fig.14 , which will not be elaborated here.

[0217] In some embodiments, a method for guiding a user to use voice instructions is also provided, which can be applied to the display device 200 shown above. The method includes:

[0218] When a trigger condition for generating a guide statement is met, at least one operation sequence is obtained, each of the operation sequences includes at least one operation event, and the operation event corresponds to a non-voice instruction of the user; from a plurality of subsequences included in the at least one operation sequence, at least one target subsequence whose frequency of occurrence in the at least one operation sequence is greater than or equal to a frequency threshold is screened out, wherein the subsequence includes at least one operation event; at least one target operation event is screened out based on the correlation between the operation events in the at least one target subsequence; based on the semantic instructions corresponding to each of the target operation events, a guide statement corresponding to the at least one target operation event is generated, and the guide statement is used to guide the user to use the voice instruction corresponding to the at least one target operation event; a target page after executing the final operation event is displayed, and the guide statement is displayed on the target page, wherein the final operation event is the last operation event in the last operation sequence in the at least one operation sequence.

[0219] The display device can dynamically generate corresponding guide words according to the operations performed by the user through non-voice instructions. The guide words are used to guide the user to use the voice instructions corresponding to these operations, and can accurately match the user's actual operations. In addition, the display device can instantly display the guide words on the page displayed after the user's last operation is executed, without the user having to laboriously search for the guide words. Moreover, by instantly displaying the guide words, the display device can enhance the correlation between the displayed guide words and the corresponding user operations, which can effectively improve the user's sense of interaction.

[0220] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion in some embodiments is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the content of this disclosure, so that those skilled in the art can better use the embodiments.

Claims

1. A display device, characterized in that: include: A display configured to: display a user interface; The controller is configured as: When a trigger condition for generating a guide word is met, at least one operation sequence is acquired, each of the operation sequences includes at least one operation event, and the operation event corresponds to a non-voice instruction of the user; Filtering out, from a plurality of subsequences included in the at least one operation sequence, at least one target subsequence whose frequency of appearance in the at least one operation sequence is greater than or equal to a frequency threshold, wherein the subsequence includes at least one of the operation events; Screening out at least one target operation event according to the correlation between the operation events in the at least one target subsequence; generating, according to the semantic instructions corresponding to each of the target operation events, a guide corresponding to the at least one target operation event, the guide being used to guide the user to use the voice instructions corresponding to the at least one target operation event; The display is controlled to display a target page after executing a final operation event, and the guide words are displayed on the target page, wherein the final operation event is the last operation event in the last operation sequence in the at least one operation sequence.

2. The display device according to claim 1, characterized in that The controller executes, when a trigger condition for generating a guide word is met, obtaining at least one operation sequence, and is specifically configured as follows: When the triggering condition for generating the guide words is met, the task of generating the guide words is started; After receiving the non-voice instruction input by the user, recording the operation event corresponding to the non-voice instruction; When the operation event of a specified type is detected, determining the operation event of the specified type as the final operation event; The at least one operation sequence is obtained from the recorded operation events, wherein each operation sequence corresponds to a preset time unit, the at least one operation event included in each operation sequence is arranged in a corresponding recording order, and a preset offset distance is spaced between two adjacent operation sequences.

3. The display device according to claim 1, characterized in that The controller performs, from a plurality of subsequences included in the at least one operation sequence, screening out at least one target subsequence whose frequency of occurrence in the at least one operation sequence is greater than or equal to a frequency threshold, and is specifically configured as follows: Acquire the multiple subsequences included in the at least one operation sequence; Calculate the frequency of occurrence of each of the multiple subsequences in the at least one operation sequence; The target subsequence that appears in the at least one operation sequence with a frequency greater than or equal to the frequency threshold is screened out.

4. The display device according to claim 1, characterized in that The controller performs screening out at least one target operation event according to the correlation between the operation events in the at least one target subsequence, and is specifically configured to: Acquire a semantic feature corresponding to each of the operation events in the at least one target subsequence; generating a semantic instruction corresponding to each of the operation events in the at least one target subsequence according to the semantic feature corresponding to each of the operation events; Calculating the semantic relevance between the semantic instructions corresponding to the operation events; The at least one target operation event is screened out according to the semantic relevance between the semantic instructions.

5. The display device according to claim 4, characterized in that The controller generates a semantic instruction corresponding to each operation event according to the semantic feature corresponding to each operation event in the at least one target subsequence, and is specifically configured to: Obtaining a mapping table, wherein the mapping table includes a mapping relationship between a preset control and a guide word; According to the control name in the semantic feature corresponding to each of the operation events, determining the corresponding target guide word in the mapping table; Determining user intention according to the semantic features; The target guiding word is processed according to the user intention to obtain a semantic instruction corresponding to the corresponding operation event.

6. The display device according to claim 1, characterized in that The controller executes the semantic instruction corresponding to each of the target operation events to generate a guide corresponding to the at least one target operation event, and is specifically configured as follows: Generate at least one to-be-selected path according to the at least one target operation event, each of the to-be-selected paths including at least one of the target operation events; Determine a target path from the at least one candidate path, wherein the target path contains the least number of the target operation events; Obtaining semantic instructions corresponding to each target operation event in the target path; The guide language is generated according to the semantic instructions corresponding to each of the target operation events in the target path.

7. The display device according to any one of claims 1 to 5, characterized in that: The controller controls the display to display a target page after the final operation event is executed, and displays the guide words on the target page, and is specifically configured as follows: Detect the status of the far-field voice function; If the far-field voice function is in an on state, displaying the guide words on the target page; If the far-field voice function is in an off state, the guide words are not displayed on the target page.

8. The display device according to claim 7, characterized in that If the state of the far-field voice function is turned on, the controller executes displaying the guide on the target page, which is specifically configured as follows: If the state of the far-field voice function is turned on, obtaining a first duration from the time corresponding to the preset node to the current time; If the first duration is less than or equal to a duration threshold, displaying the guide statement on the target page; If the first duration is greater than the duration threshold, the guide statement is not displayed on the target page.

9. The display device according to claim 5, characterized in that The controller executes the semantic instructions corresponding to each of the target operation events in the target path to generate the guide language, and is specifically configured as follows: Determine the target control corresponding to the final operation event; Creating a mapping relationship between the target control and the guide language; The mapping relationship is updated to the mapping table.

10. A method for guiding a user to use voice instructions, characterized in that: The method comprises: When a trigger condition for generating a guide word is met, at least one operation sequence is acquired, each of the operation sequences includes at least one operation event, and the operation event corresponds to a non-voice instruction of the user; Filtering out, from a plurality of subsequences included in the at least one operation sequence, at least one target subsequence whose frequency of appearance in the at least one operation sequence is greater than or equal to a frequency threshold, wherein the subsequence includes at least one of the operation events; Screening out at least one target operation event according to the correlation between the operation events in the at least one target subsequence; generating, according to the semantic instructions corresponding to each of the target operation events, a guide corresponding to the at least one target operation event, the guide being used to guide the user to use the voice instructions corresponding to the at least one target operation event; A target page after executing a final operation event is displayed, and the guide words are displayed on the target page, wherein the final operation event is the last operation event in the last operation sequence in the at least one operation sequence.

Images