Display apparatus, control method of display apparatus, and voice text processing method

Abstract

The invention relates to a display device, a control method of the display device and a voice text processing method, and relates to the technical field of display devices. And at least one processor configured to execute the instruction to enable the display device to: after receiving a voice instruction containing characterization of an added item, identify information of at least one to-do item contained in the voice instruction, the information of the to-do item including item content and prompt time of the item content, or send the information of the to-do item to the display device; the information of the to-do list comprises the item content, the prompt time of the item content and the item type to which the item content belongs; adding the information of the to-do list to a database of a target application, wherein the target application is an application capable of storing the information of the to-do list; and displaying a first prompt page added to the to-do list through the display, wherein the first prompt page comprises information of at least one to-do list. According to the scheme of the embodiment, the efficiency of adding the to-do list through the display equipment is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device, a control method of a display device, and a voice text processing method. Background Art

[0002] With the development of display technology, the number of applications that can be realized through display devices such as smart TVs is also increasing. For example, relevant to-do items can be recorded through display devices, and reminders of to-do items can be issued at specified times.

[0003] In the related art, when recording to-do items through a display device, it is necessary to open the to-do application, create a new page for adding to-do items, and enter the specific item content and the corresponding reminder time. This method of adding to-do items is inefficient. Summary of the invention

[0004] The present application provides a display device and a method for controlling the display device to improve the efficiency of adding to-do items through the display device.

[0005] In a first aspect, some embodiments provide a display device, the display device comprising:

[0006] monitor;

[0007] and at least one processor configured to execute instructions to cause the display device to:

[0008] After receiving a voice command containing a character indicating adding an item, identifying information of at least one to-do item contained in the voice command, the to-do item information including the item content and a reminder time of the item content, or the to-do item information including the item content, a reminder time of the item content and an item type to which the item content belongs;

[0009] Adding the information of the to-do items to a database of a target application, where the target application is an application capable of storing information of the to-do items;

[0010] A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

[0011] Based on the display device of this embodiment, after receiving a voice instruction containing a representation of adding an item, it can identify the item content and the reminder time of at least one to-do item in the voice instruction, and can also determine the item type to which the item content belongs, and accordingly add the to-do item information to a database of an application that can store to-do item information, and prompt through a first prompt page, so that the user can conveniently add at least one to-do item information by inputting semantic instructions, which is convenient and can efficiently and conveniently realize the recording of at least one to-do item, thereby improving the efficiency of adding to-do items through the display device.

[0012] In a second aspect, some embodiments provide a display device, the display device comprising:

[0013] monitor;

[0014] and at least one processor configured to execute instructions to cause the display device to:

[0015] After receiving the voice instruction including the representation of the added item, sending the voice instruction to the cloud server;

[0016] Receiving a recognition processing result returned by the cloud server, the recognition processing result including: a matter instruction type included in the voice instruction;

[0017] If the item instruction type in the recognition processing result includes an item adding instruction, obtaining the item content of at least one to-do item corresponding to the item adding instruction in the recognition processing result, or the item content of at least one to-do item and the item type to which the item content belongs;

[0018] Adding the information of the to-do items to a database of a target application, wherein the target application is an application capable of storing the information of the to-do items, wherein the information of the to-do items includes: the item content of the to-do items, the item content of the to-do items, and the item type to which the item content belongs;

[0019] A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

[0020] Based on the display device of this embodiment, after receiving a voice instruction containing a representation of adding an item, the display device can obtain the item content and the reminder time of at least one to-do item in the voice instruction recognized by the cloud server by sending the voice instruction to the cloud server, and can also determine the item type to which the item content belongs, and accordingly add the to-do item information to the database of an application that can store to-do item information, and provide a prompt through a first prompt page. Thus, the user can conveniently add the information of at least one to-do item by inputting semantic instructions, which is convenient and can efficiently and conveniently realize the recording of at least one to-do item, thereby improving the efficiency of adding to-do items through the display device.

[0021] In a third aspect, some embodiments of the present application provide a method for controlling a display device, wherein the method includes:

[0022] receiving a voice command including a representation of an added item;

[0023] Identify at least one to-do item information contained in the voice command, wherein the to-do item information includes item content and a reminder time of the item content, or the to-do item information includes item content, a reminder time of the item content, and an item type to which the item content belongs;

[0024] Adding the information of the to-do items to a database of a target application, where the target application is an application capable of storing information of the to-do items;

[0025] A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

[0026] In a fourth aspect, some embodiments of the present application provide a method for controlling a display device, wherein the method includes:

[0027] Receiving a voice command including a representation of an added item, and sending the voice command to a cloud server;

[0028] Receiving a recognition processing result returned by the cloud server, the recognition processing result including: a matter instruction type included in the voice instruction;

[0029] If the item instruction type in the recognition processing result includes an item adding instruction, obtaining the item content of at least one to-do item corresponding to the item adding instruction in the recognition processing result, or the item content of at least one to-do item and the item type to which the item content belongs;

[0030] Adding the information of the to-do items to a database of a target application, wherein the target application is an application capable of storing the information of the to-do items, wherein the information of the to-do items includes: the item content of the to-do items, the item content of the to-do items, and the item type to which the item content belongs;

[0031] A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

[0032] In a fifth aspect, some embodiments of the present application provide a method for processing speech and text, wherein the method comprises:

[0033] Receive the voice text content to be processed;

[0034] Performing sentence segmentation processing on the speech text content to be processed to obtain at least one sentence;

[0035] Item recognition is performed on each of the sentences respectively to obtain the item instruction type, item content and corresponding to-do item type of the to-do item corresponding to each of the sentences.

[0036] Based on the speech-text processing method of this embodiment, after obtaining the speech-text content to be processed, it can segment the speech-text content to be processed to obtain at least one sentence, and then analyze each sentence to obtain the matter instruction type, matter content and corresponding matter type corresponding to each sentence, so that information of multiple different to-do matters can be identified through one speech-text content to be processed, which can help improve the efficiency and convenience of information processing, and on this basis help improve the experience of information processing based on the speech-text processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0039] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0040] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0041] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0042] Figure 5 A schematic diagram of an interface for displaying voice text on a display of some embodiments;

[0043] Figure 6 A schematic diagram of an interface showing a first prompt page in some embodiments;

[0044] Figure 7 Schematic diagram of an interface showing a first prompt page in some other embodiments;

[0045] Figure 8 This is a schematic diagram of the interface of the first page displayed in some embodiments;

[0046] Fig. 9 A schematic diagram of an interface of a target application guide prompt page in some embodiments;

[0047] Fig.10 A schematic diagram of an interface showing a third prompt page in some embodiments;

[0048] Fig.11 It is a schematic diagram of an interface showing a third prompt page in some other embodiments;

[0049] Fig.12 A schematic diagram of an interface showing a fifth prompt page in some embodiments;

[0050] Fig.13 A schematic diagram of an interface showing a fifth prompt page in some other embodiments;

[0051] Fig.14 A schematic diagram of an interface showing a prompt message that a to-do item has been deleted in some embodiments;

[0052] Fig.15 A schematic diagram of an interface showing a prompt message that a to-do item has been deleted in some other embodiments;

[0053] Fig.16 A schematic diagram of the principle architecture of performing scene recognition and event recognition on text in some embodiments;

[0054] Fig.17 A schematic diagram of the interaction flow between the display device and the relevant modules of the server during the processing of some embodiments;

[0055] Fig.18 A schematic flow chart of a method for controlling a display device according to some embodiments of the present application;

[0056] Fig.19 A schematic flow chart of a method for controlling a display device according to some other embodiments of the present application;

[0057] Fig. 20 This is a flowchart of the information method of some embodiments of the present application. DETAILED DESCRIPTION

[0058] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0059] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0060] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0061] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0062] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0063] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc. In the following related embodiments of the present application, the display device 200 is taken as a smart TV as an example for illustration.

[0064] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0065] The mobile terminal 300 can be used as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0066] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0067] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0068] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0069] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0070] In some embodiments, the detector 230 is used to collect signals from the external environment or from the external environment. For example, the detector 230 includes a light receiver ( Figure 2 ), a sensor for collecting ambient light intensity; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes or user interaction gestures, or, the detector 230 includes a sound collector, such as a microphone, for receiving external sounds.

[0071] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0072] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0073] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0074] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0075] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0076] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0077] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0078] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0079] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0080] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0081] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0082] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0083] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130 and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0084] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0085] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, the Bluetooth protocol, or the NFC protocol and send it to the display device 200. In the user input / output interface 140, the input interface includes at least one of other input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144.

[0086] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0087] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0088] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0089] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0090] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0091] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0092] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0093] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0094] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0095] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0096] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0097] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0098] Usually, when adding a new to-do item to a display device, it is usually necessary to open the to-do item application, create a new to-do item page, and enter the specific item content and the corresponding reminder time. Based on this method, if multiple to-do items need to be added, the above operation needs to be repeated multiple times, which affects the efficiency of adding to-do items.

[0099] After research, it was found that current display devices usually have the function of voice interaction with users and networking. Taking the display device as a television device as an example, the television device can have a sound collector to receive external sounds to obtain voice commands. Therefore, by receiving voice commands, to-do items can be added and viewed, and by analyzing the voice commands, more than two to-do items can be added and viewed at the same time, thereby improving the convenience and efficiency of adding to-do items.

[0100] The display device in some embodiments of the present application includes:

[0101] monitor;

[0102] and at least one processor configured to execute instructions to cause the display device to:

[0103] After receiving a voice command containing a character indicating adding an item, identifying information of at least one to-do item contained in the voice command, the to-do item information including the item content and a reminder time of the item content, or the to-do item information including the item content, a reminder time of the item content and an item type to which the item content belongs;

[0104] Adding the information of the to-do items to a database of a target application, where the target application is an application capable of storing information of the to-do items;

[0105] A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

[0106] Among them, the method of receiving voice instructions is not limited. For example, the sound collector collects external sound signals, converts the collected analog signals into digital signals after analog-to-digital conversion, and provides them to the processor. The processor performs ASR (Automatic Speech Recognition) on the digital signals to convert the digital signals into voice text, obtain the voice text corresponding to the external sound, and recognize the voice text to obtain the instruction category of the voice instruction, such as a voice instruction or other type of instruction containing a characterization of adding items. For another example, the sound collector collects external sound signals, converts the collected analog signals into digital signals after analog-to-digital conversion, and performs ASR recognition on the digital signals to convert the digital signals into voice text, obtain the voice text corresponding to the external sound, and provide the voice text to the processor, and the processor recognizes the voice text to obtain the instruction category of the voice instruction. It can be understood that in other embodiments, the processor can also obtain voice instructions in other ways.

[0107] Among them, after obtaining the voice instruction, the processor can also display the text corresponding to the voice instruction, that is, display the semantic text used to identify the voice instruction, so as to facilitate the user to view the input voice content.

[0108] The display may display the text corresponding to the voice command in any manner, for example, the voice text may be displayed below the display area of ​​the display, such as Figure 5 As shown, it is understood that in other embodiments, the text corresponding to the voice command may also be displayed in other areas of the display, such as above, in the middle or on the side of the display area of ​​the display. It is understood that the display displays the text corresponding to the voice command in any manner, such as the color, font, size, etc. of the voice text, as long as the user can view the voice text.

[0109] The first prompt page is a page for prompting the to-do items added based on the voice command. If only one to-do item is recognized based on the voice command, the first prompt page contains information about the one to-do item; if more than two to-do items are recognized based on the voice command, the first prompt page contains information about more than two to-do items.

[0110] Adding an item is used to indicate that a to-do item needs to be added, that is, information of the to-do item corresponding to the voice command needs to be added and stored in the target application.

[0111] To-do items can be items that users plan to do at a certain time or period. The certain time or period can be a target time point of a target date, or an event determined by a time period in weeks, months, or years. For example, X o'clock on a certain day of a certain month of a certain year, such as 9 o'clock, or it can be a certain number of days, weeks, months, or years later. In the case of combining a certain number of days, weeks, months, or years, a certain time point a few days, weeks, months, or years later can be further determined as the target time point. At this time, the time point can be determined by default, or it can be a time point determined based on a certain rule combined with a reference time.

[0112] The information of the to-do items is information related to the to-do items, such as the item content and reminder time of the to-do items. In some embodiments, the information of the to-do items may also include the item type of the to-do items.

[0113] The content of the to-do list is the specific content of the to-do list, which usually refers to the specific content of the things that need to be done, such as ordering a birthday cake, watching a TV show, watching a game, booking a restaurant, preparing a birthday gift, starting a job, calling someone, etc., but not limited to these.

[0114] The reminder time of a to-do item is information about the time when the to-do item needs to be reminded. Taking the to-do item of ordering a birthday cake as an example, assuming that the corresponding reminder time is 10 o'clock on ** month ** day, ** year, then a reminder to order a birthday cake will be given at 10 o'clock on ** month ** day, ** year.

[0115] The item type of a to-do item is information related to the classification of the to-do item. The item type of a to-do item represents the category of the to-do item, wherein the classification method of the to-do item category can be determined in combination with specific actual conditions. Taking the to-do item "order birthday cake" as an example, its corresponding category can be "holidays" or "important days", etc. Taking the to-do item "watch a game" as an example, its corresponding category can be "watching game schedule", taking the to-do item "watching a TV series" as an example, its corresponding category can be "TV series calendar", etc. It can be understood that in other embodiments, other settings can also be made.

[0116] The target application is an application that can store to-do information. The specific type of the target application is not limited, such as memos, notes, sticky notes, calendars, etc., as long as the to-do information is stored and the corresponding to-do reminders can be made based on the set reminder time.

[0117] Based on the display device of this embodiment, after receiving a voice instruction containing a representation of adding an item, it can identify the item content and the reminder time of at least one to-do item in the voice instruction, and can also determine the item type to which the item content belongs, and accordingly add the to-do item information to a database of an application that can store to-do item information, and prompt through a first prompt page, so that the user can conveniently add at least one to-do item information by inputting semantic instructions, which is convenient and can efficiently and conveniently realize the recording of at least one to-do item, thereby improving the efficiency of adding to-do items through the display device.

[0118] The display device in some embodiments of the present application includes:

[0119] monitor;

[0120] and at least one processor configured to execute instructions to cause the display device to:

[0121] After receiving the voice instruction including the representation of the added item, sending the voice instruction to the cloud server;

[0122] Receiving a recognition processing result returned by the cloud server, the recognition processing result including: a matter instruction type included in the voice instruction;

[0123] If the item instruction type in the recognition processing result includes an item adding instruction, obtaining the item content of at least one to-do item corresponding to the item adding instruction in the recognition processing result, or the item content of at least one to-do item and the item type to which the item content belongs;

[0124] Adding the information of the to-do items to a database of a target application, wherein the target application is an application capable of storing the information of the to-do items, wherein the information of the to-do items includes: the item content of the to-do items, the item content of the to-do items, and the item type to which the item content belongs;

[0125] A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

[0126] The method of obtaining the voice command and the type of information of the to-do item may be the same as those in the above embodiment and will not be described in detail here.

[0127] Based on the display device of this embodiment, after receiving a voice instruction containing a representation of adding an item, the display device can obtain the item content and the reminder time of at least one to-do item in the voice instruction recognized by the cloud server by sending the voice instruction to the cloud server, and can also determine the item type to which the item content belongs, and accordingly add the to-do item information to the database of an application that can store to-do item information, and provide a prompt through a first prompt page. Thus, the user can conveniently add the information of at least one to-do item by inputting semantic instructions, which is convenient and can efficiently and conveniently realize the recording of at least one to-do item, thereby improving the efficiency of adding to-do items through the display device.

[0128] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0129] If the scenario corresponding to the voice command is a to-do application scenario, the voice command is identified by the to-do application to obtain information of the to-do, and the target application includes the to-do application.

[0130] Based on this embodiment, the to-do application is used as the target application. The to-do application can recognize voice instructions to obtain to-do information and store it in the database of the to-do application, so that the to-do item recognition, addition and storage can be realized through the same application.

[0131] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0132] If the scenario corresponding to the voice command is a to-do list application scenario, the voice command is identified by the to-do list application to obtain information of the to-do list;

[0133] The information of the to-do item is sent to the target application, so that the information of the to-do item is added to a database of the target application through the target application.

[0134] Based on this embodiment, the to-do application and the target application are different applications. The to-do application can recognize voice instructions to obtain to-do information and store it in the database of the target application, thereby performing to-do item identification, addition and storage through different applications.

[0135] When the display displays the first prompt page, the first prompt page may be displayed in full screen on the display, with the voice text being "Please remember to remind me of the Champions League group match at 9 pm next Wednesday. In addition, the drama I am following will be updated the day after tomorrow. Next Tuesday is my son's birthday. Remind me to order a cake in advance." The schematic diagram of the interface for displaying the first prompt page in full screen is as follows Figure 6 shown.

[0136] By displaying the first prompt page in full screen, the information of the added to-do items can be displayed in a larger display area, and the information of the added to-do items can be prompted more intuitively.

[0137] When the display displays the first prompt page, the first prompt page may also be displayed in a partial area of ​​the display.

[0138] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0139] While the target content is being played through the display, the first prompt page is displayed in a first display area of ​​the display superimposed on the target content being played on the display.

[0140] The first display area of ​​the display is a partial area of ​​the display. The specific location of the first display area is not limited, for example, the upper left, lower left, upper right, lower right or other partial areas of the display area of ​​the display. Taking the first display area as the lower right of the display area of ​​the display as an example, the interface schematic diagram of full-screen display of the first prompt page is as follows Figure 7 shown.

[0141] When the first prompt page is displayed superimposed on the target content played on the display, the specific method is not limited, for example, it can be displayed in a floating window, by adding a new layer, etc., but it is not limited thereto.

[0142] Based on this embodiment, the first prompt page is superimposed on the content displayed on the display and is displayed in the first display area of ​​the display, so that while the content is displayed on the display, the information of the to-do items added based on the voice command can be viewed through the first prompt page, and the information of the to-do items added based on the current voice command can be conveniently and quickly viewed and verified.

[0143] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0144] In response to receiving an instruction to open a first page, the display displays the first page, the first page including a to-do application selection item;

[0145] The control command to open a to-do application is received and responded to, the to-do application is opened, and an application guide prompt page is displayed. The to-do application is an application that can recognize the type of item instruction in the voice instruction and the corresponding to-do item information.

[0146] The type of the first page is not limited, as long as the target application can be opened and entered through the first page. Figure 8 As shown, the first page may have multiple tabs, such as A*, B*, C*, and D*, and the page corresponding to the tab D* includes multiple application options, such as 1*, memo, 2*, 3*, etc. It is understandable that in other embodiments, the first page may also have other numbers of tabs, or no tabs, and only include options for related applications. The number of applications under each option is not limited. In the case of multiple applications, only some application options may be displayed in the displayed interface, and the application options may be scrolled by means of a scroll bar or the like. It is understandable that in other embodiments, the first page may also have other settings.

[0147] The to-do application is an application that can recognize the type of task instruction in the voice instruction and the corresponding to-do information. The specific type of to-do application is not limited, such as memos, notes, sticky notes, calendars, etc., as long as it can recognize the to-do information.

[0148] The to-do application and the target application may be the same application or different applications.

[0149] The guidance prompt page is a page for prompting the use of the to-do application. In some embodiments, it can prompt how to use the to-do application through voice, for example, it can guide the user to perform what operations. In some embodiments, the application guidance prompt page is as follows: Fig. 9 shown. Fig. 9 As shown, the user is prompted to add to-do items by voice input. Fig. 9 In the figure, the voice content that needs user input for reminder can be repeatedly displayed multiple times. In other embodiments, it can also be displayed only once, or displayed other times. In addition, the voice content that needs user input for reminder can also have multiple contents and be scrolled or switched in other ways to remind.

[0150] In some embodiments, the application guidance prompt page may also display a scene screen for adding a to-do item, for example Fig. 9 The left side of the application guide prompt page shown shows a scene screen of a user performing voice input on a display device. It is understandable that in other embodiments, the application guide prompt page can also be configured in other ways, as long as it can provide guidance prompts to the user.

[0151] In the case of displaying the first page, the method of obtaining and receiving the control instruction to open the to-do list application is not limited. Figure 8As shown, taking the to-do application as a memo as an example, when the control focus of the controller of the display device is moved to the memo application, a control instruction to open the to-do application is issued by pressing a corresponding button of a controller such as a remote controller. When the display device displays an option of the memo application with the control focus and receives an instruction from a corresponding button of a controller such as a remote controller, it can be considered that a control instruction to open the to-do application has been received.

[0152] In other embodiments, a voice instruction for opening a to-do application may be acquired through a sound collector to open the to-do application.

[0153] Based on this embodiment, after opening the first page containing the to-do application selection item, the to-do application can be opened by responding to the control instruction to open the to-do application, and after opening the to-do application, the application guidance prompt page is displayed. The application guidance prompt page can display guidance prompt information for the to-do application, thereby being able to conveniently prompt the user on how to use the to-do application to add to-do items. The operation is convenient and helps to improve the user experience.

[0154] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0155] During the process of displaying the first prompt page, a first item viewing instruction is received and responded to, and a third prompt page is displayed through the display, wherein the third prompt page includes information of to-do items stored in a database of the target application.

[0156] The first item viewing instruction is an instruction for instructing to view all to-do items stored in the database of the target application. For example, when the information of the to-do items of the display device is stored in the server, it refers to an instruction for viewing all to-do items related to the display device.

[0157] The database of the target application refers to a database for storing the information of the to-do items of the target application, which may be a sub-database in the database of the display device, and other databases of the display device may be used to store other related data. The database of the target application may also be a database including a database of the server for storing the information of the to-do items of the display device.

[0158] When the display screen displays the third prompt page, the specific display method is not limited. In some embodiments, the third prompt page may be displayed in full screen.

[0159] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to: display the third prompt page in full screen on the display. An example of an interface diagram for displaying the third prompt page in full screen is shown in FIG. Fig.10 shown.

[0160] Based on this embodiment, when displaying the third prompt page, the third prompt page can be displayed in full screen. Since the full screen display has a larger display range, more to-do information can be viewed on the same interface, which can help to further improve the experience.

[0161] In other embodiments, when the third prompt page is displayed, it can also be displayed in a partial area of ​​the display area of ​​the display. The specific location of the partial area is not limited, such as the upper left, lower left, upper right, lower right or other partial areas of the display area of ​​the display. Taking the third prompt page being displayed in the lower right of the display area of ​​the display as an example, the interface schematic diagram for displaying the third prompt page is as follows: Fig.11 shown.

[0162] In some embodiments, when the first item viewing instruction is received, it can be determined whether to display the third prompt page in full screen or in a partial display area of ​​the display based on the number of stored to-do items. For example, when the number of stored to-do items is greater than or equal to a preset number threshold, the third prompt page is displayed in full screen, and when the number of stored to-do items is less than or equal to the preset threshold, the third prompt page is displayed in a partial display area of ​​the display.

[0163] Based on this embodiment, when displaying the information of the new to-do item added based on the current voice command, a first item viewing instruction can also be received to view the information of the to-do items stored in the database of the target application, so that after viewing the newly added to-do items, the stored to-do items can be further quickly and conveniently viewed.

[0164] There is no limitation on the method of receiving the first item viewing instruction, and several of the methods are described below with examples.

[0165] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0166] Controlling the first prompt page displayed on the display to include prompt information of pressing a preset button to view stored to-do items;

[0167] The third prompt page is displayed through the display in response to a key instruction of the preset key received.

[0168] The specific content of the prompt information is not limited, as long as it can prompt the user which button to press to view the stored to-do information, for example Figure 6 , Figure 7Based on the prompt information on the first prompt page, if the user presses the preset button, such as Figure 6 , Figure 7 , the display device receives the first item viewing instruction.

[0169] Based on this embodiment, a prompt message can be displayed on the first prompt page, indicating that the stored to-do items can be viewed by pressing a preset button. Based on the prompt of the prompt message, the user can issue an item viewing instruction by pressing a preset button to find the prompt of the to-do items and display it through a third prompt page. The operation is convenient and simple, which helps to improve the efficiency of viewing stored to-do items.

[0170] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0171] Controlling the first prompt page displayed on the display to include a to-do list viewing control;

[0172] Receive and respond to the operation instruction of the to-do list viewing control, and display the third prompt page through the display.

[0173] The specific type of the to-do list viewing control is not limited, as long as the control can be used to issue an instruction to view the stored to-do list. In some embodiments, the to-do list viewing control may also display a prompt word to prompt that the stored to-do list can be viewed by clicking the control. For example, some possible control prompt words may be "view all", or Figure 6 , 7 When the focus of the display device is moved to the control, the first item viewing instruction can be issued by pressing a control device such as a remote controller of the display device.

[0174] Based on this embodiment, the first prompt page includes a to-do list viewing control, so that by directly moving the control focus of the display device to the to-do list viewing control, the first item viewing instruction can be issued by operating the to-do list viewing control to find the to-do list information and display it through the third prompt page. The operation is convenient and simple, which helps to improve the efficiency of viewing stored to-do lists.

[0175] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0176] During the process of displaying the first prompt page, if a voice instruction including a first matter viewing instruction is received, the display is controlled to display the third prompt page.

[0177] Based on this embodiment, when the first prompt page is displayed, the user can issue a voice command containing the first item viewing instruction through voice, so that the display can be controlled by voice to search for the to-do item information and display it through the third prompt page. That is, without the need to perform related operations on the first prompt page, the search for the to-do item information and display it through the third prompt page can be controlled by voice. The operation is convenient and simple, which helps to further improve the efficiency of viewing stored to-do items.

[0178] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0179] The third prompt page that controls the display includes at least one prompt sub-page, each prompt sub-page corresponds to one item type, and each prompt sub-page includes: information on to-do items whose item type is the item type corresponding to the prompt sub-page in the to-do items stored in the database of the target application.

[0180] The reminder subpage is a page for displaying to-do items of a certain item type. Each reminder subpage includes the item type corresponding to the reminder subpage, as well as the item content and reminder time of each to-do item under the item type. In some embodiments, when the information of the to-do items also includes the item type of the item content, each reminder subpage may also include a pattern of the item type corresponding to the reminder subpage to visually distinguish different item types, for example Fig.10 , Fig.11 In the third prompt page shown, the prompt subpage corresponding to the event type "TV drama calendar" includes pattern 1 corresponding to the TV drama calendar, and the prompt subpage corresponding to the event type "important days" includes pattern 2 corresponding to important days.

[0181] Each reminder subpage may include at least one item display area, each item display area displays information of a to-do item, and the specific displayed content may include the item content of the to-do item and the corresponding reminder time, for example Fig.10 , 11 The content of the event is "***Update Reminder" and the corresponding reminder time is "2024.3.9 9:00 am".

[0182] Based on this embodiment, when viewing stored to-do items, the third prompt page for displaying the stored to-do items includes at least one prompt sub-page, and each prompt sub-page corresponds to an item type, so that the stored to-do items can be classified and displayed based on the item type in combination with the prompt sub-page, so that the stored to-do items under each item type can be conveniently and quickly viewed, and the stored to-do items can be viewed in categories.

[0183] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0184] During the process of displaying the first prompt page, a second item viewing instruction including the first item type is received and responded to, and a fourth prompt page is displayed through the display, and the fourth prompt page includes: information on to-do items of the first item type in the to-do items stored in the database of the target application.

[0185] The second item viewing instruction including the first item type is an instruction to instruct to view the to-do items of the first item type. It can be understood that the first item type here is only used to indicate that a specific item type has been determined, and is not used to limit the specific item type. Fig.10 , Fig.11 As shown in the example, the first item type may be “drama calendar”, “important days”, “health” and other item types.

[0186] When displaying the fourth prompt page, the fourth prompt page can be displayed in full screen or in a partial area of ​​the display area of ​​the display. The specific location of the partial area is not limited, such as the upper left, lower left, upper right, lower right or other partial area of ​​the display area of ​​the display.

[0187] Based on this embodiment, when the first prompt page displays information about a new to-do item added based on the current voice command, a second item viewing instruction including the first item type can also be received to view the to-do items stored in the database of the target application, the to-do items of the first item type. Therefore, after adding a new to-do item, the to-do items of the first item type in the already stored to-do items can be further viewed, so that the to-do items of the specified item type can be quickly and easily viewed, which is convenient.

[0188] There is no limitation on the method of receiving the second item viewing instruction, and several of the methods are described below with examples.

[0189] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0190] The first prompt page displayed on the display is controlled to include at least one item display area, and the content displayed in each item display area includes: information of a to-do item contained in the voice command, and an item type control corresponding to the item type of the to-do item;

[0191] In response to the operation instruction of the matter type control, the fourth prompt page is displayed through the display.

[0192] In this embodiment, the item display area is an area for displaying a to-do item. The specific displayed content may include the item content of the to-do item and the corresponding reminder time, and may also include the corresponding item type, wherein the item type may be displayed in the form of text or in the form of a pattern.

[0193] The implementation of the item type control is not limited. In some embodiments, the item type may be displayed in the item display area in the form of a control, such as Figure 6 , Figure 7 In the example shown, "TV series calendar", "holidays and festivals", and "sports schedule" can be displayed as controls.

[0194] Based on this embodiment, when the first prompt page is displayed, each to-do item is displayed in a different item display area, and each item display area includes an item type control corresponding to the item type of the to-do item, so that by directly moving the control focus of the display device to the item type control, a second item viewing instruction containing the item type can be issued by operating the item type control to display the fourth prompt page. The operation is convenient and simple, which helps to improve the efficiency of viewing stored to-do items.

[0195] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0196] During the process of displaying the first prompt page, if a voice instruction containing a second item viewing instruction of the first item type is received, the display is controlled to display the fourth prompt page.

[0197] There is no limitation on the method of obtaining the voice instruction of the second item viewing instruction containing the first item type. For example, after converting the external sound collected by the sound collector into voice text, the voice text is semantically understood to identify the second item viewing instruction containing the first item type.

[0198] Based on this embodiment, when the first prompt page is displayed, the user can issue a voice command containing an instruction to view the second item of the first item type through voice. Therefore, without having to perform relevant operations on the first prompt page, the display can be controlled by voice to search for to-do items of the first item type and display them through the fourth prompt page. The operation is convenient and simple, which helps to further improve the efficiency of viewing stored to-do items.

[0199] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0200] If a voice command for viewing a third item including time information is received, a fifth prompt page is displayed through the display, and the fifth prompt page includes: the reminder time of the to-do items in the to-do items stored in the database of the target application and the information of the to-do items associated with the time information.

[0201] The third item viewing instruction including time information is an instruction for viewing to-do items indicating a certain time range, wherein the time information may be a certain time node or a certain time range.

[0202] When the fifth prompt page is displayed, the fifth prompt page can be displayed in full screen, such as Fig.12 As shown, it can also be displayed in a partial area of ​​the display area of ​​the display, and the specific position of the partial area is not limited, such as the upper left, lower left, upper right, lower right or other partial areas of the display area of ​​the display, Fig.13 The illustration is based on the lower right corner of the display area of ​​the display as an example.

[0203] Based on this embodiment, the to-do items corresponding to the specified time information in the to-do items stored in the database of the target application can also be viewed through voice commands. The to-do items corresponding to the specified time information can be viewed quickly and easily, which is convenient.

[0204] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0205] After receiving the voice command, identifying the scene corresponding to the voice command;

[0206] If the scenario corresponding to the voice instruction is a to-do application scenario, perform item recognition on the voice instruction to obtain an item recognition result, the item recognition result including: an item instruction type, and if the recognized item instruction type includes an item adding instruction, item content of at least one to-do item included in the voice instruction, or item content of at least one to-do item included in the voice instruction and a corresponding item type;

[0207] Determine a reminder time for the to-do item.

[0208] The display device based on this embodiment, after receiving a voice command, first identifies the scenario corresponding to the voice command, and if it is a to-do list application scenario, then performs item recognition on the voice command to identify the item instruction type in the voice command, and when the item instruction type includes an item adding instruction, identifies the item content of at least one to-do item in the voice command, and can also simultaneously identify the item type of the item content and determine the prompt time corresponding to the item content, so that item recognition is performed on the voice command only when the scenario of the voice command is a to-do list application scenario, which can improve the efficiency of voice command recognition.

[0209] The reminder time is the time that triggers the reminder of the stored to-do items after the to-do items are stored. In the case where the voice instruction includes clear time information, such as the voice instruction includes time information of *year*month*day*time, the time information in the voice instruction can be used as the reminder time of the to-do items. In other embodiments, in the case where the voice instruction does not include clear time information, such as the time-related information included in the voice instruction is *days later, *weeks later, *days later, next month, etc., the reminder time corresponding to the item content can be determined based on the time information in the voice instruction and the reference time.

[0210] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0211] If the item recognition result includes an item adding instruction, a reference time is obtained, and a prompt time corresponding to the item content corresponding to the item adding instruction is determined based on the reference time.

[0212] Based on the display device of this embodiment, when the recognized matter recognition result includes an matter adding instruction, the display device determines the prompt time corresponding to the matter content corresponding to the matter adding instruction in combination with the reference time. Therefore, when the voice instruction does not include the accurate prompt time, the accurate prompt time corresponding to the matter content can be determined in combination with the reference time.

[0213] The reference time is a time used to assist in determining the reminder time of the to-do item. If the voice text does not accurately indicate the specific reminder time of the to-do item, the specific reminder time of the to-do item can be determined in combination with the reference time.

[0214] The specific selection method of the reference time is not limited. In the relevant embodiments of the present application, the reference time may be the current time of the display device, which may be the time when the voice command is received, or the storage time when the to-do item information is stored. In the case where the voice command is sent to the server for item recognition, the reference time may also be any one of the time when the voice command is sent to the server and the time when the recognition processing result returned by the server is received, but is not limited thereto.

[0215] There are no restrictions on the method of prompting the time of the to-do items determined based on the reference time. The following only provides examples of several of these methods.

[0216] As described above, if the voice instruction includes specific time content corresponding to the to-do item, such as "*month*day*hour", the time in the time content in the voice instruction can be directly determined as the reminder time of the to-do item.

[0217] In some embodiments, if the voice command only includes the first type of time content, the reminder time of the to-do item can be determined by combining the reference time, the first type of time content, and the preset time point. The first type of time content is used to indicate content related to the length of time, such as tomorrow, the day after tomorrow, next week*, * days later, the *th day of next month, etc.

[0218] The preset time point is a specific time point within the time range of a day, which may include multiple time points, such as 9 o'clock, 12 o'clock, 18 o'clock (or 6 o'clock in the afternoon), 20 o'clock (or 8 o'clock in the afternoon), etc. It can be understood that in other embodiments, the preset time point can also be set otherwise.

[0219] When determining the reminder time of a to-do item by combining the reference time, the first type of time content, and the preset time point, the date in the reminder time can be determined by combining the date in the reference time and the first type of time content, and the time point in the reminder date can be determined by combining the time point in the reference time and the preset time point.

[0220] In some embodiments, the reminder time can be determined based on the first type of time content. If the reminder time is not on the same day as the current time, the time point in the reminder time can be directly confirmed from multiple preset time points. If the reminder time is determined based on the first type of time content and is on the same day as the current time, the time point in the reminder time can be determined based on the time point in the current time and the preset time point.

[0221] Taking "Remind me of XX in the morning" as an example in the voice text, assuming that the current time is 10:10 on March 8, 2024, the reminder date of the corresponding to-do item is March 8, 2024, and because 10 o'clock has exceeded the default time point of 9 o'clock, you can add a preset time length based on 10 o'clock, for example, add 1 hour, that is, 11 o'clock is determined as the time point in the reminder time.

[0222] Taking "Remind me ** the day after tomorrow" in the voice text as an example, assuming that the current time is 10:10 on March 8, 2024, the reminder date of the corresponding to-do item is March 10, 2024, and the first default time point of 9 o'clock is used as the time point in the reminder time of the to-do item, that is, the reminder time of the to-do item is 9 o'clock on March 10, 2024.

[0223] It is understandable that in other embodiments, the reminder time of the to-do item may also be confirmed in other ways.

[0224] The display device can recognize the voice command in various possible ways to obtain the matter recognition result.

[0225] In some of the embodiments, the processor is further configured to execute instructions to cause the display device to:

[0226] If the scenario corresponding to the voice command is a to-do list application scenario, the voice command is subjected to item recognition through an item recognition model to obtain the item recognition result.

[0227] The item recognition model is a model used to recognize the instruction type (such as add, search, delete, etc.) for to-do items in voice instructions. In some examples, the item recognition model can also recognize the item type in the voice instruction. The item recognition model deployed on the display device can be deployed on the display device after being trained on other devices such as a server.

[0228] Based on this embodiment, the display device can be deployed with an item recognition model. The display device can perform item recognition on the voice command through the item recognition model deployed on the display device to obtain the item instruction type in the voice command and the item content and item type corresponding to the item instruction type. The item recognition result can thereby achieve analysis, recognition and processing of voice text in the target application without being connected to the server.

[0229] Among them, the types of matter instructions identified by the matter recognition model may include, in addition to matter adding instructions, matter viewing instructions and / or matter deleting instructions.

[0230] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0231] If the identified task instruction type includes a task viewing instruction, information of at least one to-do task corresponding to the task viewing instruction is obtained and displayed.

[0232] The information of the to-do items may be stored in the display device or in a server communicating with the display device. In the case of storing the information in the display device, the information of the to-do items corresponding to the item viewing instruction may be obtained from the memory of the display device for display. In the case of storing the information in the server, the information of the to-do items corresponding to the item viewing instruction may be obtained from the server for display.

[0233] Based on this embodiment, the item viewing instruction in the voice instruction can be identified, and based on the identified item viewing instruction, the item content corresponding to the item viewing instruction, the item type to which the item content belongs, and the corresponding reminder time are obtained and displayed. Thus, the user can conveniently view the item content and reminder time of the stored to-do items by inputting the voice content, which is convenient and efficient, and can efficiently and conveniently realize the viewing of the item content and corresponding reminder time of multiple stored to-do items, thereby improving the convenience of viewing the stored to-do items through the display device.

[0234] Among them, the information of at least one to-do item corresponding to the item viewing instruction obtained can be displayed in the second prompt page displayed on the display. The third prompt page and the second prompt page are both used to display the queried to-do items, wherein the second prompt page is a page displayed directly after the query is performed based on the received voice instruction, and the third prompt page is displayed after the first prompt page is displayed based on the received first item viewing instruction. Therefore, the second prompt page and the third prompt page can be the same page or different pages, and the embodiment of the present application does not limit this.

[0235] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0236] If the identified item instruction type includes an item deletion instruction, the to-do item corresponding to the item deletion instruction stored in the display device is deleted.

[0237] After deleting the to-do item, the display device may also display a prompt message indicating that the to-do item has been deleted. The specific method of displaying the prompt message is not limited, and it may be a full-screen display, such as Fig.14 As shown, it can also be displayed in a partial display area of ​​the display screen, such as Fig.15 As shown. It is understandable that Fig.15The example shown is displayed in the lower right corner of the display screen. In other embodiments, it can also be displayed in other areas of the display screen.

[0238] Based on this embodiment, the item recognition model deployed by the display device can identify the item deletion instruction in the voice text, and based on the identified item deletion instruction, the stored to-do items can be deleted.

[0239] The model structure of the event recognition model is not limited. In some embodiments, the structure of the event recognition model is as follows: Fig.16 As shown, it includes sentence segmentation model, operation classification sub-model and event type classification sub-model, among which:

[0240] The sentence segmentation model is used to segment the input text to obtain at least one sentence after segmentation;

[0241] The operation classification sub-model is used to identify the operation classification of the sentences obtained after the sentence segmentation model, such as adding a to-do item, querying a to-do item, deleting a to-do item, or other operation categories;

[0242] The task type classification sub-model is used to analyze the task types of the to-do items in the input sentence, such as "drama calendar", "important days", etc.

[0243] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0244] If the scenario corresponding to the voice command is a to-do list application scenario, the voice command is subjected to item recognition through an item recognition model to obtain the item recognition result, wherein:

[0245] Segment the voice instruction using the sentence segmentation model in the event recognition model to obtain at least one sentence;

[0246] The sentences are analyzed and processed respectively by the operation classification sub-model in the matter recognition model to obtain the matter instruction type corresponding to each sentence;

[0247] The matter content of the sentence and the matter type to which the matter content belongs are respectively identified through the matter type classification sub-model in the matter recognition model.

[0248] Based on this embodiment, the display device can be deployed with an item recognition model. The display device can identify the item instruction type in the voice instruction and the item content corresponding to the item instruction type through the item recognition model deployed on the display device. It can also determine the item type to which the item content belongs, so that the item recognition processing of the voice instruction can be realized without being connected to the server. Moreover, the item recognition model can include a sentence segmentation model, an operation classification sub-model and an item type classification sub-model to respectively perform sentence segmentation processing, analysis and processing of the item instruction type of the sentence, and processing of the item type of the sentence. Different processing is performed respectively through different sub-models. In the case where the sentence segmentation model recognizes multiple sentences, it can be processed multiple times through the operation classification sub-model and the item type sub-model, so as to analyze each sentence separately, which helps to further improve the accuracy of the analysis of the voice text. In the case where there are multiple different to-do items or multiple different event instruction types in the voice text, it can also be identified and processed on this basis, which helps to further improve the processing efficiency.

[0249] Among them, when the display device sends the voice command to the server to obtain the recognition processing result returned by the server, the display device can directly send the received voice command to the server to instruct the server to recognize and process the voice command, or it can perform scene recognition on the voice command, and then send the voice command to the server when the scene corresponding to the voice command is a to-do application scene. The recognition processing result returned by the server and received by the display device may include the reminder time of the to-do item, or may not include the reminder time of the to-do item, which is explained below with examples.

[0250] In some embodiments, the display device may send a voice command to a server, which identifies the scene, the type of matter command in the voice command, the matter content and the matter type of the to-do item in the voice command, and the display device determines the corresponding prompt time.

[0251] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0252] Sending the speech text to a server to obtain a recognition processing result returned by the server;

[0253] If the item instruction type in the identification processing result returned by the server includes an item adding instruction,

[0254] A reference time is obtained, and a reminder time of each to-do item in the recognition processing result is determined based on the reference time.

[0255] Based on the display device of this embodiment, after obtaining the voice instruction, it directly sends the voice instruction to the server, so that the server can identify whether the voice instruction is a scenario of a to-do list application. If it is a scenario of a to-do list application, it can identify the item instruction type in the voice instruction including an item adding instruction, and identify the item content in the voice instruction and the item type to which the item content belongs. After obtaining the recognition processing result returned by the server, the display device determines the prompt time corresponding to the item content in combination with the reference time, and stores it. In this way, the powerful processing power of the server can be used to realize the recognition and analysis of the item instruction type, item content and item type in the voice text, which can improve the efficiency of voice text processing, reduce the demand for processing power of the display device, and help reduce the cost of the display device. In addition, when the voice instruction does not contain an accurate indication of the prompt time, the accurate prompt time corresponding to the item content can also be determined in combination with the reference time.

[0256] In some embodiments, the display device may send a voice command to a server, which recognizes the scene, the type of matter command in the voice command, the matter content of the to-do items in the voice command, the matter type, and the reminder time of the matter content.

[0257] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0258] If the item instruction type in the recognition and processing result includes an item adding instruction, obtain the reminder time of at least one to-do item corresponding to the item adding instruction in the recognition and processing result, wherein the to-do item information also includes the reminder time of the to-do item.

[0259] Based on this embodiment, the recognition processing results returned by the server may also include the reminder time of the to-do items. After obtaining the recognition processing results returned by the server, the display device can store them based on the information in the recognition processing results. There is no need to determine the reminder time, which can further reduce the demand for the processing power of the display device, and can also help reduce the cost of the display device and improve the processing efficiency of the display device.

[0260] In some embodiments, the display device may recognize the scene, and the server may recognize the item instruction type of the voice instruction, recognize the item content and item type of the to-do item in the voice text, and may also determine or not determine the reminder time of the item type. The display device may store the information based on the information returned by the server.

[0261] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0262] If the scenario corresponding to the voice instruction is a to-do list application scenario, the voice text is sent to a server to obtain a recognition processing result returned by the server.

[0263] In which, when the identification processing result returned by the server includes the reminder time of the to-do item, or does not include the reminder time of the to-do item, the display device can determine the reminder time in the same way as in the above embodiment.

[0264] The display device based on this embodiment first identifies the scenario corresponding to the voice instruction after receiving the voice instruction. If it is an application scenario with a to-do list, the voice instruction is sent to the server so that the server can identify the item instruction type in the voice instruction and, when the item instruction type is an item adding instruction, identify the item content and item type in the voice text, and at the same time, determine and not determine the prompt time corresponding to the item content. After obtaining the recognition processing result returned by the server, the display device can store it based on the information in the recognition result, thereby making use of the powerful processing power of the server to realize the recognition and analysis of the item instruction type, item content, item type and corresponding prompt time in the voice text, thereby improving the efficiency of voice text processing, reducing the demand for processing power of the display device, and helping to reduce the cost of the display device.

[0265] Among them, the item instruction types contained in the identification processing results returned by the server may include item viewing instructions and / or item deletion instructions in addition to item adding instructions.

[0266] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0267] If the identification processing result returned by the server includes an item viewing instruction, information of at least one to-do item corresponding to the item viewing instruction is obtained and displayed.

[0268] Wherein, when the identification processing result includes an item viewing instruction, in some embodiments, the identification processing result may also include a corresponding to-do item identifier. The to-do item identifier is used to uniquely identify a to-do item. In some examples, the to-do item identifier may be the item content, so that the information of the to-do item can be found based on the item content.

[0269] In some embodiments, the identification processing results may also include the item type of the to-do item, so that based on the item type, the to-do item corresponding to the to-do item identifier can be found from the to-do items corresponding to the item type, which helps to improve the efficiency of finding and obtaining the information of the to-do items.

[0270] In some embodiments, when the recognition processing result includes an item viewing instruction, the recognition processing result may also include the item content, item type, and prompt time of the to-do item, so that the display device can obtain the item content, item type, and prompt time of the to-do item from the recognition processing result. Among them, the display device can store the to-do items of the display device, that is, the to-do items of the display device can be stored in the display device, can also be stored in the server, or can be stored in the display device and the server at the same time. Therefore, after receiving the item viewing instruction, the server can find the information of the to-do item corresponding to the item viewing instruction from the to-do items stored in the server, and return it to the display device through the recognition processing result.

[0271] Based on this embodiment, when the recognition processing results returned by the server include an item viewing instruction, the item content, item type and prompt time corresponding to the item viewing instruction can be obtained and displayed, thereby enabling the stored to-do items to be viewed.

[0272] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0273] If the identification processing result returned by the server includes an item deletion instruction, obtaining a to-do item identifier of at least one to-do item corresponding to the item deletion instruction in the identification processing result;

[0274] The to-do item corresponding to the to-do item identifier and stored in the display device is deleted.

[0275] As described above, the to-do item identifier is used to uniquely identify a to-do item, so the corresponding to-do item can be found and deleted through the to-do item identifier.

[0276] In which, when the server stores to-do items corresponding to the display device, the server can delete the to-do items corresponding to the item deletion instruction stored in the server, and return the identification processing result including the to-do item identifier to the display device at the same time or after deleting the to-do items stored in the server.

[0277] It can be understood that in some embodiments, when the to-do items are stored only on the server, the identification processing result returned by the server to the display device may be a notification message that the to-do item has been deleted, and the display device may display the notification message to prompt the user that the corresponding to-do item has been deleted.

[0278] After deleting the to-do item, the display device may also display a prompt message indicating that the to-do item has been deleted. The specific method of displaying the prompt message is not limited, and it may be a full-screen display, such as Fig.14As shown, it can also be displayed in a partial display area of ​​the display screen, such as Fig.15 As shown. It is understandable that Fig.15 The example shown is displayed in the lower right corner of the display screen. In other embodiments, it can also be displayed in other areas of the display screen.

[0279] Based on this embodiment, when the identification processing result returned by the server includes an item deletion instruction, the to-do item corresponding to the to-do item identifier stored in the display device can be deleted based on the item deletion instruction, thereby deleting the to-do item stored in the display device.

[0280] Among them, in the above embodiments, the server is not limited to the way of recognizing and processing the voice command, for example, it can be processed through an event recognition model.

[0281] Among them, the event recognition model used by the display device or server to process voice commands can be obtained through training. For example, after the event recognition model is trained by a server or other device, the trained event recognition model is deployed to the display device or server to perform event recognition processing.

[0282] There is no limitation on the method of training to obtain the event recognition model. In some embodiments, the method of training to obtain the event recognition model may be as described below.

[0283] First, a sample data set is obtained, where each sample data in the sample data set includes text and a label corresponding to the text, wherein the number of labels may include multiple ones, such as the number of sentences in the text, the type of matter instruction of each sentence, and the type of matter of each sentence.

[0284] After obtaining the sample data set, the sample data set is divided into a training set, a validation set, and a test set. For example, 70% of the sample data constitutes the training set, 15% of the sample data constitutes the validation set, and the remaining 15% of the sample data constitutes the test set to evaluate the generalization ability of the model.

[0285] After the sample data set is divided, the model training process can be carried out. In the model training process, the item recognition model is trained through the training set, and the performance of the item recognition model is evaluated using the validation set during or after the training. The trained item recognition model is finally tested using the test set to ensure the performance of the model in practical applications. It is understandable that in the process of training the item recognition model, the item recognition model performs sentence segmentation processing on the sample data to obtain at least one sentence, and analyzes and processes each sentence to obtain the item instruction type and item type of the sentence, and compares it with the number of sentences annotated with the sample data, the item instruction type and the item type of each sentence to calculate the training loss, and adjusts the item recognition model in combination with the calculated training loss to achieve the update of the item recognition model. After the model training is completed, the item recognition model updated for the last time is used as the item recognition model obtained by training.

[0286] In the above embodiment, the method of identifying the scene corresponding to the voice command is not limited.

[0287] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0288] Determining, by a first scene recognition model, a first classification probability value that the voice command belongs to a preset classification scene of the display device;

[0289] If the largest first classification probability value is greater than a first preset threshold, and the scenario category corresponding to the largest first classification probability value is a to-do list application scenario, it is determined that the scenario corresponding to the voice instruction is a to-do list application scenario.

[0290] Among them, the first scene recognition model is a model that can recognize the to-do list application scenario and other application scenarios of the display device. The preset classification scenarios include the to-do list application scenario and other application scenarios, and the specific categories of other application scenarios are not limited.

[0291] Among them, the first scene recognition model can be obtained through training. After the event recognition model is obtained through training through a server or other device, the trained first scene recognition model is deployed to a display device or server to perform scene recognition.

[0292] The method for training to obtain the first scene recognition model is not limited. In some embodiments, the method for training to obtain the first scene recognition model may be as described below.

[0293] First, a sample data set is obtained. Each sample data in the sample data set includes text and a label corresponding to the text, wherein the label includes the scene category to which the sample data belongs, such as a to-do application scenario (such as a calendar, memo, notes, etc.) or other scene categories (such as movie mode, etc.).

[0294] After obtaining the sample data set, the sample data set is divided into a training set, a validation set, and a test set. For example, 70% of the sample data constitutes the training set, 15% of the sample data constitutes the validation set, and the remaining 15% of the sample data constitutes the test set to evaluate the generalization ability of the model.

[0295] After the sample data set is divided, the model training process can be carried out. During the model training process, the first scene recognition model is trained through the training set, and the performance of the first scene recognition model is evaluated using the validation set during or after the training, and the trained first scene recognition model is finally tested using the test set to ensure the performance of the model in practical applications. It is understandable that in the process of training the first scene recognition model, the first scene recognition model performs scene recognition processing on the sample data to obtain the identified scene category, and compares the scene category with the scene category annotated with the sample data to calculate the training loss, and adjusts the first scene recognition model in combination with the calculated training loss to achieve the update of the first scene recognition model. After the model training is completed, the first scene recognition model updated for the last time is used as the trained first scene recognition model.

[0296] Based on this embodiment, in the process of identifying the scene corresponding to the voice command, the first scene recognition model can be used to identify multiple scenes of the display device including the scene of the target application, so that multiple scenes including the scene of the target application can be identified through a first scene recognition model. The model deployment is convenient, and the simplicity of model deployment and application can also be improved.

[0297] In some embodiments, recognizing that a voice command belongs to a to-do list application scenario and other scenario categories other than the to-do list application scenario may be performed using different models, that is, the structure of the scenario recognition is as follows: Fig.16 As shown, it includes a second scene recognition model and a to-do scene recognition model, wherein:

[0298] A second scene recognition model is used to identify whether the text belongs to a preset classification scene of the display device, which can output the probability that the speech text belongs to the preset classification scene of the display device;

[0299] The to-do scenario recognition model is used to identify whether the text belongs to the classification scenario of the target application. It can output the probability that the speech text belongs to the classification scenario of the target application.

[0300] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0301] Determining, by a second scene recognition model, a second classification probability value of the voice command belonging to a preset classification scene of the display device;

[0302] Determining, by a to-do list scenario recognition model, a third classification probability value that the voice instruction is classified as a category of the to-do list application scenario;

[0303] If the third classification probability value is greater than the second preset threshold, determining that the scene corresponding to the voice instruction is a to-do list application scene;

[0304] If the third classification probability value is less than the second preset threshold, and the largest second classification probability value is greater than the third preset threshold, an action corresponding to the scene category corresponding to the largest second classification probability value is executed.

[0305] The second scene recognition model is a model that can recognize scenes other than the to-do list application scene. The second scene recognition model can be obtained through training, and the specific training method can be the same as the method of training the first scene recognition model.

[0306] The to-do list scene recognition model is a model for identifying whether a voice command is a to-do list application scene. The to-do list scene recognition model can be obtained through training, and the specific training method can be the same as the method of training the first scene recognition model.

[0307] When the third classification probability value of the voice instruction output by the to-do list scenario recognition model being classified as a to-do list application scenario is greater than the second preset threshold, it indicates that the probability that the voice instruction belongs to the to-do list application scenario is relatively high, and thus it can be directly determined that the scene corresponding to the voice instruction is a to-do list application scenario.

[0308] When the third classification probability value output by the to-do list scene recognition model is less than the second preset threshold, it means that the probability that the voice command belongs to the to-do list application scenario is not particularly high, and when the second classification probability value output by the second scene recognition model is greater than the third preset threshold, it means that the probability that the voice command belongs to other applications is relatively high. Therefore, it is possible to directly determine that the scene corresponding to the voice text is the scene category corresponding to the largest second classification probability value of the second scene recognition model, and execute the corresponding processing under this scene category.

[0309] Among them, the second preset threshold is a threshold used to distinguish the probability size of the output results of the second scene recognition model, and the third preset threshold is a threshold used to distinguish the probability size of the output results of the to-do scene recognition model. Therefore, the second preset threshold and the third preset threshold can be set separately, and the two can be set to the same value or to different values.

[0310] The second scene recognition model and the to-do scene recognition model can be obtained by training separately. For example, if the display device has already deployed the second scene recognition model, the to-do scene recognition model can be obtained by retraining and added to the display device, which can improve the compatibility with the model deployed on the display device and improve development efficiency.

[0311] Based on this embodiment, in the process of identifying the scene corresponding to the voice command, the to-do list application scene can be identified through the to-do list scene recognition model, and the multiple scenes of the display device can be identified through the second scene recognition model. In combination with the recognition result of the to-do list scene recognition model and the recognition result of the second scene recognition model, it is comprehensively determined whether the scene is a to-do list application scene or other scene. Therefore, in the case that the display device has been deployed with the second scene recognition model, different scenes can be identified by deploying the to-do list scene recognition model and comprehensively determining the scene category by combining the recognition results of the two models without adjusting the second scene recognition model, so that it can have high compatibility with display devices that have been deployed with scene recognition models.

[0312] In the above embodiments, the to-do application and the target application may be the same application or different applications.

[0313] Accordingly, in some embodiments, the processor is further configured to execute instructions to enable the display device to:

[0314] If the scenario corresponding to the voice command is a to-do application scenario, the voice command is identified by the to-do application to obtain information of the to-do, and the target application includes the to-do application.

[0315] Based on this embodiment, the to-do application is used as the target application. The to-do application can recognize voice instructions to obtain to-do information and store it in the database of the to-do application, so that the to-do item recognition, addition and storage can be realized through the same application.

[0316] In some embodiments, the processor is further configured to execute instructions to cause the display device to:

[0317] If the scenario corresponding to the voice command is a to-do list application scenario, the voice command is identified by the to-do list application to obtain information of the to-do list;

[0318] The information of the to-do item is sent to the target application, so that the information of the to-do item is added to a database of the target application through the target application.

[0319] Based on this embodiment, the to-do application and the target application are different applications, for example, the to-do application is a memo, and the target application is a calendar, but not limited to this, so that the to-do application can recognize the voice instructions to obtain the to-do information and store it in the database of the target application, so that the to-do items can be identified, added and stored separately through different applications.

[0320] Based on the example of the display device described above, in the processing of the display device in some embodiments, the relevant interaction flow chart is as follows: Fig.17 As shown, Fig.17 In the figure, an example is taken in which the display device sends the voice command to the server for processing after obtaining the voice command.

[0321] Combination Fig.17 As shown, after the sound collector of the display device collects the external sound, automatic voice recognition is performed on the external sound to obtain the corresponding voice text, thereby obtaining the voice instruction, and the voice text (voice instruction) is sent to the background server.

[0322] After receiving the voice text, the backend server identifies the scene corresponding to the voice text, such as combining Fig.16 The second scene recognition model and the to-do scene recognition model shown in the figure recognize the scene corresponding to the voice text. If it belongs to the to-do application scene, the voice text is sent to the calendar sub-service, and the calendar sub-service calls the event recognition model to process the voice text, including: segmenting the voice text, analyzing the event instruction type of each sentence and the corresponding event type.

[0323] Among them, in the above-mentioned processing of the voice text, if the corresponding instruction type is not analyzed in a sentence, the item instruction type of the voice text or the item instruction type of other sentences (such as adjacent sentences) can be determined as the item instruction type of the sentence, so that the same item instruction type can be processed for multiple to-do items through the same voice text (i.e. the same voice). Therefore, the same voice input by the user can realize the processing of different item instruction types for different to-do items, and can also realize the processing of the same item instruction type for multiple different to-do items through the same voice.

[0324] The model output results obtained by the calendar subservice calling the event recognition model may include a variety of different situations.

[0325] If the item instruction type in the model output result includes an item adding instruction, the calendar sub-service obtains the item content and the corresponding item type of the to-do item corresponding to the item adding instruction, and adds the corresponding to-do item to the database of the server. The item content of the to-do item can be obtained from the corresponding sentence of the item adding instruction. The reminder time of the to-do item can be determined by the server in combination with the reference time. In some embodiments, if the model output result only includes the item adding instruction, each to-do item in the model output result is an item that needs to be added, and the calendar sub-service adds multiple to-do items to the database of the server respectively.

[0326] After obtaining the item content and corresponding item type of the to-do item corresponding to the item adding instruction, the calendar sub-service also returns the recognition processing result to the display device through the server. The recognition processing result includes the item adding instruction, the item content and corresponding item type of the to-do item.

[0327] After receiving the recognition processing result, the display device adds the to-do items in the recognition processing result to the storage space corresponding to the memo / calendar application of the display device. In some embodiments, the recognition processing result may include the reminder time of each to-do item, and the display device may directly add each to-do item based on the reminder time in the recognition result. In other embodiments, the recognition processing result may only include information related to the indicated time in the voice text or the sentence obtained after the voice text is segmented. The display device determines the reminder time of each to-do item based on the information and the reference time, and then adds each to-do item.

[0328] If the item instruction type in the model output result includes an item deletion instruction, the calendar sub-service obtains the item content of the to-do item corresponding to the item deletion instruction, and may also obtain the corresponding item type at the same time, and deletes the to-do item corresponding to the item deletion instruction in the server database. In some embodiments, if the model output result only includes an item deletion instruction, then all to-do items in the model output result are items that need to be deleted, and the calendar sub-service deletes all corresponding multiple to-do items in the server database.

[0329] After obtaining the item content and the corresponding item type of the to-do item corresponding to the item deletion instruction, the calendar sub-service also returns the identification processing result to the display device through the server, and the identification processing result may include the item deletion instruction and the item content of the to-do item. In some embodiments, the identification processing result may also include the item type of the to-do item.

[0330] After receiving the recognition processing result, the display device deletes the to-do item of the item deletion instruction from the storage space corresponding to the memo / calendar application. Wherein, if the recognition processing result includes the item type of the to-do item, the to-do item can be searched based on the item type and then based on the item content to improve the efficiency of searching and deleting the corresponding to-do item.

[0331] If the event instruction type in the model output result includes an event search instruction, the calendar sub-service obtains the query information contained in the event search instruction, which may be time-related information, keywords related to the event content, or event types, and searches the server database for to-do items corresponding to the query information.

[0332] After obtaining the to-do item corresponding to the item search instruction, the calendar sub-service also returns the recognition processing result to the display device through the server. The recognition processing result may include the item search instruction, the item content of the to-do item, the item content and the reminder time.

[0333] After receiving the recognition processing result, the display device displays the to-do items included in the recognition processing result through the display.

[0334] Among them, in the case of to-do items stored in the display device, when the reminder time of a to-do item is reached, for example, the time of the clock of the display device is the same as the reminder time of the to-do item, the to-do application of the display device (such as a memo or calendar) will remind the to-do item. The prompt method is not limited, for example, it can be one or more of a variety of methods such as voice prompt, pop-up text prompt, prompt through a page in a certain display area of ​​the display device, etc.

[0335] In the case where the server stores to-do items, when the reminder time of a to-do item is reached, for example, the time of the server's clock is the same as the reminder time of the to-do item, the server can push the to-do item to the display device to prompt the to-do item through the display device.

[0336] Among them, when the display device is associated with a corresponding recipient identifier, for example, the server stores the correspondence between the device identifier of the display device and the recipient identifier, the server searches for the corresponding recipient identifier based on the device identifier to which the to-do item belongs, and pushes the to-do item to the terminal corresponding to the corresponding recipient identifier to achieve remote reminder.

[0337] The device identifier is used to uniquely identify a device. The specific identifier type is not limited, such as the physical address of the display device or a unique identifier generated based on the collected display device information. The receiver identifier is used to uniquely identify an identifier that can receive information. The specific receiver identifier type is not limited, such as a mobile terminal number, an instant messaging account, etc., but not limited to this, as long as it can remind the user.

[0338] Based on the display device as described above, some embodiments of the present application also provide a method for controlling the display device.

[0339] refer to Fig.18 As shown, the control method of the display device in some embodiments includes:

[0340] Step S1801: receiving a voice instruction including a representation of an added item;

[0341] Step S1802: Identify at least one to-do item information contained in the voice command, wherein the to-do item information includes the item content and the reminder time of the item content, or the to-do item information includes the item content, the reminder time of the item content and the item type to which the item content belongs.

[0342] Step S1803: adding the information of the to-do items to a database of a target application, where the target application is an application capable of storing information of the to-do items;

[0343] Step S1804: Displaying a first reminder page indicating that a to-do item has been added via the display, wherein the first reminder page includes information of the at least one to-do item.

[0344] refer to Fig.19 As shown, the control method of the display device in some embodiments includes:

[0345] Step S1901: receiving a voice instruction including a representation of an added item, and sending the voice instruction to a cloud server;

[0346] Step S1902: Receive the recognition processing result returned by the cloud server, and the recognition processing result includes: the type of matter instruction contained in the voice instruction.

[0347] Step S1903: if the item instruction type in the recognition processing result includes an item adding instruction, obtaining the item content of at least one to-do item corresponding to the item adding instruction in the recognition processing result, or the item content of at least one to-do item and the item type to which the item content belongs;

[0348] Step S1904: adding the information of the to-do items to the database of the target application, the target application being an application capable of storing the information of the to-do items, the information of the to-do items including: the item content of the to-do items, the item content of the to-do items and the item type to which the item content belongs;

[0349] Step S1905: Displaying a first reminder page indicating that a to-do item has been added via the display, wherein the first reminder page includes information of the at least one to-do item.

[0350] The specific implementation method of the control method of the display device in the embodiment of the present application can be the same as the implementation method of the above-mentioned display device, which will not be repeated here.

[0351] Some embodiments of the present application also provide a voice-text processing method, which can be performed by a display device, a server, or any other related device that can obtain voice-text content and needs to identify, add, delete, and / or search for to-do items from the voice text.

[0352] refer to Fig. 20 As shown, the speech text processing method in some embodiments includes:

[0353] Step S2001: receiving speech text content to be processed.

[0354] The voice text content to be processed is the voice text content that needs to be processed, which may be text content obtained by recognizing acquired voice information, such as the voice text or voice command obtained by the display device in the above-mentioned embodiment processing external sound.

[0355] Step S2002: segment the speech text content to be processed to obtain at least one sentence.

[0356] Step S2003: Perform item recognition on each of the sentences respectively to obtain the item instruction type, item content and corresponding to-do item type of the to-do item corresponding to each of the sentences.

[0357] Based on the speech-text processing method of this embodiment, after obtaining the speech-text content to be processed, it can segment the speech-text content to be processed to obtain at least one sentence, and then perform item recognition on each sentence to obtain the item instruction type, item content and corresponding item type corresponding to each sentence, so that information of multiple different to-do items can be identified through one text content to be processed, which can help improve the efficiency and convenience of information processing, and on this basis help improve the experience of information processing based on the speech-text processing method.

[0358] The specific method of segmenting the text content to be processed to obtain at least one sentence, and the method of identifying the matter for each sentence can be the same as the processing method of the above-mentioned display device, such as segmenting the text content to be processed through an matter recognition model to obtain at least one sentence, and analyzing and processing each of the sentences separately to obtain the matter instruction type, matter content and corresponding to-do matter type of the to-do matter corresponding to each of the sentences, which will not be repeated here.

[0359] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0360] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for controlling a display device in any of the embodiments described above are implemented.

[0361] In some embodiments, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the method for controlling a display device in any of the embodiments described above.

[0362] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0363] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0364] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A display device, characterized in that: The display device comprises: monitor; and at least one processor configured to execute instructions to cause the display device to: After receiving a voice command containing a character indicating adding an item, identifying information of at least one to-do item contained in the voice command, the to-do item information including the item content and a reminder time of the item content, or the to-do item information including the item content, a reminder time of the item content and an item type to which the item content belongs; Adding the information of the to-do items to a database of a target application, where the target application is an application capable of storing information of the to-do items; A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

2. The display device according to claim 1, characterized in that The processor is further configured to execute instructions to cause the display device to: After receiving the voice command, identifying the scene corresponding to the voice command; If the scenario corresponding to the voice instruction is a to-do application scenario, perform item recognition on the voice instruction to obtain an item recognition result, the item recognition result including: an item instruction type, and if the recognized item instruction type includes an item adding instruction, item content of at least one to-do item included in the voice instruction, or item content of at least one to-do item included in the voice instruction and a corresponding item type; Determine a reminder time for the to-do item.

3. The display device according to claim 2, characterized in that The processor is further configured to execute instructions to cause the display device to: If the scenario corresponding to the voice command is a to-do list application scenario, the voice command is subjected to item recognition through an item recognition model to obtain the item recognition result, wherein: Segment the voice instruction using the sentence segmentation model in the event recognition model to obtain at least one sentence; The sentences are analyzed and processed respectively by the operation classification sub-model in the matter recognition model to obtain the matter instruction type corresponding to each sentence; The matter content of the sentence and the matter type to which the matter content belongs are respectively identified through the matter type classification sub-model in the matter recognition model.

4. The display device according to claim 2 or 3, characterized in that: The processor is further configured to execute instructions to cause the display device to: Determining, by a second scene recognition model, a second classification probability value of the voice command belonging to a preset classification scene of the display device; Determining, by a to-do list scenario recognition model, a third classification probability value that the voice instruction is classified as a category of a to-do list application scenario; If the third classification probability value is greater than the second preset threshold, determining that the scene corresponding to the voice instruction is a to-do list application scene; If the third classification probability value is less than the second preset threshold, and the largest second classification probability value is greater than the third preset threshold, an action corresponding to the scene category corresponding to the largest second classification probability value is executed.

5. A display device, characterized in that: The display device comprises: monitor; and at least one processor configured to execute instructions to cause the display device to: After receiving the voice instruction including the representation of the added item, sending the voice instruction to the cloud server; Receiving a recognition processing result returned by the cloud server, the recognition processing result including: a matter instruction type included in the voice instruction; If the item instruction type in the recognition processing result includes an item adding instruction, obtaining the item content of at least one to-do item corresponding to the item adding instruction in the recognition processing result, or the item content of at least one to-do item and the item type to which the item content belongs; Adding the information of the to-do items to a database of a target application, wherein the target application is an application capable of storing the information of the to-do items, wherein the information of the to-do items includes: the item content of the to-do items, the item content of the to-do items, and the item type to which the item content belongs; A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

6. The display device according to any one of claims 1 to 5, characterized in that: The processor is further configured to execute instructions to cause the display device to: If the scenario corresponding to the voice command is a to-do list application scenario, the voice command is identified by the to-do list application to obtain information of the to-do list; The information of the to-do item is sent to the target application, so that the information of the to-do item is added to a database of the target application through the target application.

7. The display device according to any one of claims 1 to 5, characterized in that: The processor is further configured to execute instructions to cause the display device to: The first prompt page displayed on the display is controlled to include at least one item display area, and the content displayed in each item display area includes: information of a to-do item contained in the voice command, and an item type control corresponding to the item type of the to-do item; In response to the operation instruction of the matter type control, the fourth prompt page is displayed through the display.

8. A method for controlling a display device, characterized in that: The method comprises: receiving a voice command including a representation of an added item; Identify at least one to-do item information contained in the voice command, wherein the to-do item information includes item content and a reminder time of the item content, or the to-do item information includes item content, a reminder time of the item content, and an item type to which the item content belongs; Adding the information of the to-do items to a database of a target application, where the target application is an application capable of storing information of the to-do items; A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

9. A method for controlling a display device, characterized in that: The method comprises: Receiving a voice command including a representation of an added item, and sending the voice command to a cloud server; Receiving a recognition processing result returned by the cloud server, the recognition processing result including: a matter instruction type included in the voice instruction; If the item instruction type in the recognition processing result includes an item adding instruction, obtaining the item content of at least one to-do item corresponding to the item adding instruction in the recognition processing result, or the item content of at least one to-do item and the item type to which the item content belongs; Adding the information of the to-do items to a database of a target application, wherein the target application is an application capable of storing the information of the to-do items, wherein the information of the to-do items includes: the item content of the to-do items, the item content of the to-do items, and the item type to which the item content belongs; A first reminder page in which a to-do item has been added is displayed through the display, and the first reminder page includes information of the at least one to-do item.

10. A method for processing speech and text, characterized in that: The method comprises: Receive the voice text content to be processed; Performing sentence segmentation processing on the speech text content to be processed to obtain at least one sentence; Item recognition is performed on each of the sentences respectively to obtain the item instruction type, item content and corresponding item type of the to-do item corresponding to each of the sentences.

Images