Display device and interaction method

Abstract

The invention relates to a display device and an interaction method. The display device comprises: a display configured to display at least one display item, the display item being current display content or at least one historical response result generated based on historical interaction content; the input module is configured to receive first interaction content input by a user, and the first interaction content at least comprises ordinal words or anonyms; and the controller is configured to respond to a control instruction corresponding to the first interaction content, and display first response content matched with an interaction target on the display by taking the display item referred to by the ordinal number word or the referral word as the interaction target. By adopting the display equipment, effective fusion of the content displayed on the display and the first interaction content input by the user can be realized, and an interaction response result with clear reference is obtained, so that the accuracy of the interaction result can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and an interaction method. Background Art

[0002] With the rapid development of artificial intelligence technology, human-computer interaction dialogue systems have been widely used in various terminal products, such as smart TVs, smart home appliances, smart phones, and car voice.

[0003] The human-computer interaction system in traditional technology has its dialogue capabilities limited to dialogue generation services such as chatting and question-and-answering. The human-computer interaction system can generate answers and feedback to users through context understanding.

[0004] However, for scenarios where corresponding content is displayed in real time on a display device, such as movie search results or a player playing a TV series, when a user wants to interact based on these displayed contents, the human-computer interaction process often finds it difficult to obtain accurate interaction results. Summary of the invention

[0005] The present application provides a display device and an interaction method to solve the problem in the prior art that accurate interaction results cannot be obtained during the human-computer interaction process.

[0006] In a first aspect, some embodiments provide a display device, comprising: a display, configured to display at least one display item, the display item being current display content or at least one historical response result generated based on historical interaction content; an input module, configured to receive first interaction content input by a user, the first interaction content comprising at least an ordinal number or a pronoun; a controller, configured to respond to a control instruction corresponding to the first interaction content, take the display item referred to by the ordinal number or the pronoun as an interaction target, and display first response content matching the interaction target on the display.

[0007] Through the above display device, during the interaction process, the display item referred to by the ordinal number or the pronoun in the first interactive content can be clarified directly based on the current display content or at least one historical response result generated based on the historical interactive content. By taking the display item as the interactive target and displaying the first response content matching the interactive target on the display, the content displayed on the display and the first interactive content input by the user can be effectively integrated to obtain an interactive response result with a clear reference, thereby improving the accuracy of the interactive result.

[0008] In a second aspect, some embodiments provide a display device, comprising: a display, configured to display at least one display item, the display item being a current display content or at least one historical response result generated based on a historical first interactive content; an input module, configured to receive a first interactive content input by a user, the first interactive content comprising at least an ordinal numeral or a pronoun, and a keyword belonging to a target category; a controller, configured to respond to a control instruction corresponding to the first interactive content, take the content referred to by the ordinal numeral or the pronoun in the display item as an interactive target, and display a first response content provided by a target application of the target category on the display, the first response content being a response result to the first interactive content and the interactive target.

[0009] Through the above display device, during the interaction process, the display item referred to by the ordinal number or the pronoun in the first interactive content can be clarified directly based on the current display content or at least one historical response result generated based on the historical interactive content. By taking the display item as the interaction target and obtaining and displaying the first response content provided by the target application of the target category when the first interactive content contains keywords belonging to the target category, the content displayed on the display and the interactive content input by the user can be effectively integrated, and an interactive response result with clear reference can be obtained. Moreover, cross-application interaction can be realized, thereby improving the accuracy of the interaction result.

[0010] In a third aspect, some embodiments further provide an interaction method, which is applied to the display device provided in the first aspect, and the method includes:

[0011] Obtain target data reported by the display device when receiving interactive content, the target data including the interactive content and interface display information when the interactive content is received, the interface display information including at least one display item, the display item being the current display content or at least one historical response result generated based on the historical interactive content; when the interactive content includes ordinal numbers or pronouns, rewrite the interactive content based on the display items in the interface display information to obtain rewritten content; generate response content for the rewritten content, and feed the response content back to the display device for display.

[0012] Through the interaction method provided in the above embodiments, during the interaction process, it is possible to directly clarify the display item referred to by the ordinal number or the pronoun in the first interactive content based on the current display content or at least one historical response result generated based on the historical interactive content, and rewrite the interactive content based on the display item in the interface display information to obtain the rewritten content, which can achieve effective fusion of the content displayed on the display and the first interactive content input by the user, and obtain an interactive response result with clear reference, thereby improving the accuracy of the interactive result.

[0013] In a fourth aspect, some embodiments further provide an interaction method, which is applied to the display device provided in the second aspect, and the method includes:

[0014] Obtain target data reported by the display device when receiving interactive content, the target data including the interactive content and interface display information when the interactive content is received, the interface display information including at least one display item, the display item being the current display content or at least one historical response result generated based on historical interactive content; in the case where the interactive content includes ordinal numbers or pronouns, and keywords belonging to a target category, rewrite the interactive content based on the display items in the interface display information to obtain rewritten content; call a target application of the target category, obtain response content for the rewritten content based on the target application, and feed back the response content to the display device for display.

[0015] Through the interaction method provided in the above embodiment, during the interaction process, the display item referred to by the ordinal number or the pronoun in the first interactive content can be clarified directly based on the current display content or at least one historical response result generated based on the historical interactive content. By taking the display item as the interaction target and, when the first interactive content contains keywords belonging to the target category, rewriting the interactive content based on the display item in the interface display information, obtaining the first response content provided by the target application of the target category and displaying it, the content displayed on the display and the interactive content input by the user can be effectively integrated, and an interactive response result with clear reference can be obtained. Moreover, cross-application interaction can be achieved, thereby improving the accuracy of the interactive result. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0018] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0019] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0020] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0021] Figure 5 A schematic diagram of page changes of a display device provided in some embodiments of the present application;

[0022] Figure 6 A flowchart of an interactive method provided in some embodiments of the present application;

[0023] Figure 7 A flowchart of an interactive method provided in some embodiments of the present application;

[0024] Figure 8 A timing diagram of an interactive method provided in some embodiments of the present application;

[0025] Fig. 9 A flowchart of an interactive method provided in some embodiments of the present application;

[0026] Fig.10 A schematic diagram of a flow chart for determining whether to rewrite provided in some embodiments of the present application;

[0027] Fig.11 A schematic diagram of the large model training process provided for some embodiments of the present application. DETAILED DESCRIPTION

[0028] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0029] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0030] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0031] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0032] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0033] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0034] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0035] The mobile terminal 300 can be used as a control device to perform human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device to establish a communication connection with the display device 200 and perform data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0036] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0037] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0038] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0039] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0040] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0041] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0042] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0043] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0044] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0045] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0046] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0047] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0048] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0049] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0050] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0051] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0052] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0053] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0054] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0055] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0056] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0057] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0058] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0059] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0060] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0061] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0062] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0063] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0064] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0065] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0066] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0067] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0068] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0069] With the rapid development of artificial intelligence technology, human-computer interaction dialogue systems have been widely used in various terminal products, such as smart TVs, smart home appliances, smart phones, and car voice. The human-computer interaction system in traditional technology has a dialogue capability limited to chat, question-and-answer and other dialogue generation services. The human-computer interaction system can generate answers and feedback to users through context understanding. However, for scenes with real-time display of corresponding content on the display device, such as movie search results and the player playing a TV series, when users want to interact based on these displayed contents, the human-computer interaction process often finds it difficult to obtain accurate interaction results.

[0070] In some embodiments, a display device is provided, the display device comprising:

[0071] The display is configured to display at least one display item.

[0072] The display item is the current display content or at least one historical response result generated based on the historical interaction content.

[0073] The current display content may specifically be media asset data. Media asset data refers to various forms of content resources in the media industry and various types of data and information related thereto. In terms of content form, media asset data includes different types of materials such as audio, video, pictures, and text. Examples include TV program tapes, radio station audio files, news agency photo libraries, and text content of various publications. Media asset data may specifically be data stored locally on a display device or data received over a network.

[0074] When a user turns on a display device to play media data, the display device can play and display the media data in full screen, such as playing TV series, movies, variety shows, etc. At this time, the number of display items displayed on the display is 1, that is, the current display content such as media data.

[0075] At least one historical response result generated based on historical interaction content is a response result generated based on the interaction content input by the user in the past when the user has triggered human-computer interaction. For example, when a user inputs a search keyword and the search results for the keyword are displayed, if there are multiple search results containing the keyword, there are multiple response results for the interaction result, and each response result is a display item.

[0076] When the user triggers the interactive mode and performs human-computer interaction with the display device, the display device can display the historical response results generated based on the historical interaction content. In some embodiments, the historical response results displayed on the display device can be only the response results obtained from the last interaction, or the response results obtained from the previous K (K is a positive integer) interactions. The number of rounds of historical response results displayed on the display can be set according to the actual scenario, which can be a system setting when the display device leaves the factory, or it can be set by the user through the setting items.

[0077] The input module is configured to receive first interactive content input by a user, where the first interactive content at least includes an ordinal number or a pronoun.

[0078] The input module is an interface that can receive user operations. The specific form of the input module is not limited, as long as it can receive the user's input content. The interactive content received by the input module can be a mode of text, picture, voice, or a combination of multiple modes. The controller can convert the interactive content obtained through the input module into input text for the display device for subsequent processing.

[0079] In some embodiments, the user controls the display device to enter an interactive mode by interacting with the display device, and in the interactive mode, the input module can receive interactive content input by the user. In the interactive mode, the user inputs interactive content by one or more combinations of manual input and voice input.

[0080] Optionally, after the display device is triggered to enter the manual input mode, the display device may call up a keyboard built into the display device, and text or pictures may be selected and sent through keyboard input.

[0081] Optionally, the user can trigger the keyboard through the touch screen or remote control buttons, and the display device can display candidate text or candidate images corresponding to the triggered keyboard combination. The display device can further display the user's selected candidate text or candidate image in the input bar. After the user confirms that the editing is completed and triggers the sending operation, the display device can display the sent text content, or image content, or a combination of text content and image content in the interactive dialog box. Using the above method, the user can choose pictures or texts to interact with the display device, thereby realizing multi-modal interaction methods.

[0082] Optionally, when the display device is triggered to enter the voice input mode, the controller may receive voice content input by the user through the voice input interface.

[0083] Optionally, the user can control the display device to enter the voice input mode by operating a designated button on the remote control. In the actual application process, the correspondence between the voice input mode command and the remote control button is pre-bound. For example, a voice button is set on the remote control. When the user touches the button, the remote control sends the voice input mode command to the controller. At this time, the display device enters the voice input mode and can smoothly receive the voice command input by the user.

[0084] Optionally, the controller may first bind the corresponding relationship between the voice input mode instruction and multiple remote control buttons. When the user touches the multiple buttons bound to the voice input mode instruction, the remote control issues the voice input mode instruction. In a feasible embodiment, the buttons bound to the voice input mode instruction are direction keys (left, down, left, down). Only when the user continuously touches the buttons (left, down, left, down) within a preset time, the remote control sends the voice input mode instruction to the controller. The above binding method can be used to avoid the voice input mode instruction being issued due to user misoperation. The embodiments of the present application only provide several exemplary binding relationships between voice input mode instructions and buttons. In actual application, the binding relationship between the voice input mode instruction and the button can be set according to the user's habits, and no excessive restrictions are made here.

[0085] Optionally, the user can control the display device to enter the voice input mode by voice. For example, the display device can be triggered to enter the voice input mode by speaking the far-field wake-up word. For example, after the user speaks the voice of "let's get together", the display device can enter the voice input mode. When the display device is triggered to enter the voice input mode, the user's voice command can be monitored in real time to enable the user to further speak the voice command. The user can use the microphone to input voice commands to the display device.

[0086] Optionally, when the display device is triggered to enter the voice input mode, the user can also send instructions to the display device in text form through a mobile phone, remote control or other device to prevent the display device from being unable to receive the user's voice commands when there is a problem with the microphone.

[0087] Optionally, when the display device receives a voice command input by the user, the controller can send the received voice data to the voice recognition service, convert it into interactive text, and display the interactive text on the display. The recognition operation of the user's voice command can refer to the relevant technology, and the embodiments of the present application will not be described one by one.

[0088] Among them, ordinal numbers are a type of numerals, mainly used to indicate order and sequence. In Chinese, common ordinal numbers include "first", "second", "third", etc. Ordinal numbers are usually formed by adding specific affixes to cardinal numbers or by specific expressions, such as adding the word "first" and the corresponding number after the cardinal number to indicate a specific order. In English, ordinal numbers are generally derived from cardinal numbers, such as "first", "second", "third", etc.

[0089] Pronouns are a type of word that has an indicative or substitution function. In terms of indicative function, pronouns can be used to point out specific people, things, places, times, etc. For example, "this", "that", "this", "that", "these", "those", "here", "there", etc., "this", "these", "this", "that", "that", "those" indicate specific people or things. "Here" and "there" indicate specific places and locations respectively. In terms of substitution function, pronouns can replace the nouns mentioned above, etc. For example, "he", "she", "it" are used to replace men, women and things respectively, and "we", "you", "they", etc. can replace a corresponding group of people.

[0090] In daily life, the use of ordinal numbers and pronouns can make sentences more concise and avoid repetition in language expression. It also helps to clarify the objects referred to, enhance the coherence and expressiveness of the language, and be more in line with daily expression habits. For the human-computer interaction process, clearly understanding the objects referred to by ordinal numbers and pronouns can effectively improve the accuracy of the interaction and enhance the user experience.

[0091] The controller is configured to, in response to a control instruction corresponding to the first interactive content, use the display item indicated by the ordinal word or the pronoun as the interactive target, and display the first response content matching the interactive target on the display.

[0092] In some embodiments, when the first interactive content input by the user contains an ordinal word, the processor responds to the control instruction corresponding to the first interactive content and first identifies the ordinal word. For example, the user inputs "Who is the director of the third movie?", and the processor of the display device or the processor connected to the display device through the network can identify the ordinal word "third" through natural language processing technology. Then, the processor determines the arrangement order of each display item in the current display interface, and finds the corresponding display item according to the ordinal word. If the current display interface is in the form of a list, the processor will determine the serial number of each display item in a predetermined order from top to bottom or from left to right. In the above embodiment, the processor will find the third display item in the list as the interactive target. Then, the processor generates the first response content according to the interactive target. For example, if the interactive target is the third movie in multiple movie display pages, the processor can display the name, representative works, biography and other detailed information of the director of the movie on the display as the first response content. When there are multiple directors for the third movie, the directors of the third movie can be displayed in turn as display items, and the user can conduct the next round of interaction based on the multiple directors displayed. For example, the user can continue to input the interactive content "What new works has the first director recently produced?", and the processor can further obtain the latest works of the first director in the displayed director list and feedback them to the user.

[0093] If the first interactive content contains a pronoun, the processor will also identify it. For example, if the user inputs "I want to watch this movie", the processor will identify the pronoun "this". Then, the processor determines the display item referred to by the pronoun based on the current interactive context. If the user has just browsed several movies on the movie display page and enters "this movie", the processor can determine the specific display item referred to by the pronoun based on information such as the user's browsing history, interface focus, or mouse position. Once the interactive target is determined, the processor generates the first response content that matches it and displays it on the display. For example, if the interactive target is a specific movie, the first response content may include information such as the detailed introduction of the movie and a playback link.

[0094] Through the above embodiments, when the user inputs the first interaction content containing an ordinal number or a pronoun, the processor can accurately determine the interaction target and display the corresponding first response content, thereby improving the efficiency and accuracy of the interaction.

[0095] In this embodiment, during the interaction process, the display item referred to by the ordinal number or the pronoun in the first interactive content can be clarified directly based on the current display content or at least one historical response result generated based on the historical interactive content. By taking the display item as the interactive target and displaying the first response content matching the interactive target on the display, the content displayed on the display and the first interactive content input by the user can be effectively integrated to obtain an interactive response result with a clear reference, thereby improving the accuracy of the interactive result.

[0096] In some embodiments, the controller is further configured to: when there are multiple display items and the first interactive content includes ordinal numbers, use display items whose display order is the same as the order represented by the ordinal numbers as interaction targets.

[0097] When the user inputs the first interactive content through the input device and the content contains an ordinal number, the processor first starts the natural language processing module to analyze the first interactive content and accurately identify the ordinal number. For example, the user inputs "the fifth TV series looks very attractive", and the controller recognizes the ordinal number "fifth".

[0098] At the same time, the processor determines the display order of the multiple display items displayed on the display page of the display. If the display items are presented in a list, they may be arranged in order from top to bottom; if they are presented in a specific layout in a graphical interface, the display order may be determined according to rules such as from left to right, clockwise or counterclockwise.

[0099] After clarifying the display order of the ordinal numerals and display items, the processor will take the display items whose display order is the same as the order represented by the ordinal numerals as the interaction target. Further, the processor will find the fifth display item in the established display order among the multiple display items and determine it as the interaction target. Assume that the display items are a series of TV series, each TV series has its own cover picture or name. When the user inputs "I want to know more about the fifth one", the processor determines the fifth TV series "TV series A (ID: 002244)" as the interaction target according to the display order.

[0100] After determining the interaction target, the processor can rewrite the first interaction content based on the interaction target. For example, the processor will rewrite "I want to know more about the fifth one" to "I want to know more about TV series A (ID: 002244)", thereby generating corresponding response content for "I want to know more about TV series A (ID: 002244)" and displaying it on the display device.

[0101] In this embodiment, when there are multiple display items and the first interactive content includes ordinal numbers, by taking display items whose display order is the same as the order represented by the ordinal numbers as interactive targets, the interactive target indicated by the user can be accurately located, providing the user with a more accurate and efficient interactive service.

[0102] In some embodiments, the controller is further configured to:

[0103] When there are multiple display items, one of the display items is displayed as the interface focus; when the first interactive content includes an ordinal number, and the order represented by the ordinal number is the same as the display order of the display item where the interface focus is located, the display item where the interface focus is located is used as the interaction target.

[0104] The display is used to display multiple display items in a specific order, and can highlight the display item that is the interface focus. When there are multiple display items, the controller first selects a display item from the multiple display items as the interface focus for display. This interface focus can be distinguished from other display items by specific visual effects (such as highlighting, different color borders, etc.).

[0105] The display item where the interface focus is located will switch as the control signal changes. The control signal can come from the user's input device operation, such as the direction keys of the control device (such as a remote control), the movement and click of the mouse, etc. For example, when the user presses the right direction key of the control device, the interface focus will switch from the current display item to the next display item, and move in the established display order.

[0106] Optionally, when the user inputs the first interactive content through the input device and the content contains an ordinal number, the controller calls the natural language processing module to analyze the first interactive content and accurately identify the ordinal number. If the order represented by the ordinal number is the same as the display order of the display item where the interface focus is located, the controller uses the display item where the interface focus is located as the interactive target. For example, the current interface focus is on the third display item, and the user inputs "Who is the leading actor of the third movie?", the controller will determine the display item where the interface focus is located as the interactive target.

[0107] In this embodiment, when there are multiple display items, the interaction target is determined by switching the interface focus and judging whether the order of the ordinal number and the object referred to by the interface focus is the same, so that the interaction target can be quickly located, thereby achieving a more flexible and efficient interaction experience.

[0108] In some embodiments, the controller is further configured to, when there are multiple display items, display one of the display items as the interface focus; the display item where the interface focus is located switches following changes in the control signal; and when the first interactive content includes an ordinal number, and the order represented by the ordinal number is different from the display order of the display item where the interface focus is located, the display item with the same display order as the order represented by the ordinal number is used as the interaction target.

[0109] When the user inputs the first interactive content through the input device, the controller calls the natural language processing module to analyze the first interactive content to identify whether it contains an ordinal number. If an ordinal number is detected, the controller further determines the order represented by the ordinal number. Then, the controller compares the order represented by the ordinal number with the display order of the display item with the interface focus in the current display page. If the two are different, it means that the object referred to by the ordinal number input by the user is not the display item with the current interface focus.

[0110] When it is determined that the order represented by the ordinal word is different from the display order of the display item where the interface focus is located, the controller will search for the corresponding display item in the multiple display items according to the order represented by the ordinal word, and use it as the interaction target. For example, if the current interface focus is on the fourth display item, and the user inputs "I want to see the third one", the controller will find the third display item in the display order among the multiple display items and determine it as the interaction target.

[0111] In this embodiment, when there are multiple display items, the controller positions the interface focus and determines the interaction target according to the ordinal number. This can reduce the possibility of the interface focus misunderstanding the content referred to in the first interactive content input, improve the accuracy of positioning the interaction target, and provide users with a more flexible and efficient interaction experience to meet the interaction needs in different application scenarios.

[0112] In some embodiments, the controller is further configured to: during multiple rounds of interaction, control the display to display the display items of each round of interaction; when the first interaction content includes an ordinal number, take the interaction round to which the display item where the interface focus is located as the target interaction round; and take the display items in the target interaction round whose display order is the same as the order represented by the ordinal number as the interaction target.

[0113] In multiple rounds of interaction, each round of interaction will present specific display items on the display. These display items can be content in various modes such as text, images, videos, etc. In the process of multiple rounds of interaction, the controller is responsible for controlling the display to display the display items of each round of interaction in sequence on the display page. The display items of each round of interaction can be different according to the user's input, the system's preset rules, or the changes in dynamic data. Figure 5As shown, for example, in the first round of interaction, the display may display a list of a group of film and television works; in the second round of interaction, according to the user's selection, detailed information of a specific film and television work may be displayed.

[0114] When the user inputs the first interactive content during the interaction process and the content contains an ordinal number, the controller first calls the natural language processing module to identify the ordinal number. Then, the interaction round to which the display item where the interface focus is located belongs is taken as the target interaction round. The interface focus can be identified by specific visual effects (such as highlighting, borders of different colors, etc.). If the display item where the current interface focus is located is displayed in the second round of interaction, then the second round of interaction is determined as the target interaction round. After determining the target interaction round, the controller will search for display items in the target interaction round whose display order is the same as the order represented by the ordinal number, and use them as the interaction target. For example, in the target interaction round, the display is displaying five different film and television works, and the user enters "Who is the third starring actor?" The controller will find the third film and television work in the display order in the display items of this round and determine it as the interaction target.

[0115] In this embodiment, during the multi-round interaction process, a more accurate and efficient multi-round interaction experience can be achieved by controlling the display of the interface focus on the display item and determining the interaction target according to the ordinal number, thereby meeting the needs of users in complex interaction scenarios.

[0116] In some embodiments, the controller is further configured to: when the number of display items is 1 and the first interactive content includes a pronoun, use the display item as an interactive target.

[0117] For a display device, the displayed content may be multiple items or one item. For example, when the display device plays media data in full screen, the number of displayed items on the display device is 1, and the display interface of the display device does not display the interface focus.

[0118] After determining that the number of display items is 1 based on the display content of the display page, the controller calls the natural language processing module to analyze the first interactive content input by the user, and uses natural language processing technology to identify whether it contains pronouns. Common pronouns include "this", "that", "it", etc. If a pronoun is detected in the first interactive content, the controller uses the only display item as the interaction target. Since there is only one display item at this time, when the user uses the pronoun to interact, it must point to this only display item.

[0119] For example, movie A is currently playing in full screen on the display, and the user inputs "Who is its director?", the controller will identify this unique movie A as the interaction target, and rewrite the user input "Who is its director?" to "Who is the director of movie A?", and generate a corresponding answer as the response content based on the question-answering model, and display it on the display page.

[0120] In this embodiment, when the number of display items is 1 and the first interactive content includes a pronoun, through the operation process of the above implementation, the controller can accurately determine the only display item as the interactive target, quickly locate the interactive target, and provide the user with a more convenient and efficient interactive experience.

[0121] In some embodiments, the controller is further configured to: when there are multiple display items, display one of the display items as the interface focus; switch the display item where the interface focus is located following the change of the control signal; when the first interactive content includes a pronoun, use the display item where the interface focus is located as the interactive target.

[0122] When there are multiple display items, the controller will randomly or according to a preset rule select one of the display items as the interface focus for display. The interface focus is usually presented with a specific visual effect, such as highlighting, different color borders, etc., so that the user can clearly identify the display item currently in focus.

[0123] The display item where the interface focus is located will switch as the control signal changes. The control signal can be sent by the user through an input device, such as the direction keys of a control device (such as a remote control), the movement and click of a mouse, etc. For example, when the user presses the right direction key of the control device, the interface focus will switch between different display items in the established display order; when the user moves the mouse and clicks, the interface focus will also switch to the display item pointed to by the mouse accordingly.

[0124] When the user inputs the first interactive content through the input device, the controller will call the natural language processing module to analyze the first interactive content input by the user to identify whether it contains pronouns. Such as "this", "that", "it", etc. If a pronoun is detected in the first interactive content, the controller will use the display item where the current interface focus is located as the interactive target. Because in the case of multiple display items, when the user uses pronouns to interact, it is usually for the display item that is currently in focus. For example, the current interface focus is on the third display item, and the user inputs "When will this movie be released?" The processor will determine the movie A displayed by the third display item in focus as the interactive target. And rewrite "When will this movie be released" to "When will movie A be released", and generate the corresponding answer as the response content based on the question-answering model, and display it on the display page.

[0125] In this embodiment, when there are multiple display items, the processor switches the interface focus and determines the interaction target based on the pronoun, so as to quickly locate the interaction target, provide users with a more flexible and efficient interaction experience, and meet the interaction needs in different application scenarios.

[0126] In some embodiments, in the process of displaying at least one historical response result generated based on the historical first interactive content on the display, the controller is further configured to:

[0127] On the display, M response results generated based on the interactive content are arranged and displayed along a first direction, where M≤N, M and N are positive integers, and N is the maximum number of display items that can be accommodated on the display along the first direction.

[0128] The display has a specific display area, and can arrange and display content along a first direction (for example, a horizontal direction or a vertical direction). On the display, M response results generated based on the interactive content are arranged and displayed along the first direction. The controller determines the spatial range currently available for displaying the response results, that is, the maximum number N of display items that can be accommodated along the first direction. If the number M of response results of the interactive content is greater than N, N results are selected from the M response results according to rules such as time order or importance for display. If the number M of response results is less than N, all of them are displayed.

[0129] For example, the display can accommodate at most 6 display items (N=10) in the horizontal direction, and there are currently 8 response results, then the controller will select the first 6 response results and display them in sequence in the horizontal direction on the display.

[0130] Furthermore, in order to improve the user's recognition and interactive experience, each response result can be displayed with a specific visual effect, such as a different font color, background color, or adding an icon, etc. The user can select a historical response result by clicking, touching, or other input methods to perform further operations, such as viewing detailed content, re-executing the corresponding interactive operation, etc.

[0131] In this embodiment, by arranging and displaying M response results generated based on the interactive content along the first direction on the display, a convenient viewing method can be provided for the user, and at the same time, it is convenient to define the arrangement order of multiple display items, facilitate determining the interactive target referred to by the ordinal number, and improve the processing efficiency of the interactive process and the user experience.

[0132] In some embodiments, the currently displayed content includes the system home page or the played media asset data; the controller is further configured to:

[0133] Keep displaying the system main page or keep playing the media data, and display the first response content in the first display area of ​​the display.

[0134] Among them, during the interaction process, the current display content of the display may be the system home page or the media data being played. The system home page is the first interface that the user sees after entering the system, and may include various menu options, icons, notifications and other information. The media data may be multimedia content such as video, audio, and images, such as movies, TV series, music, and photos. The user may select the media data he wants to watch and play the media data on the display by sending a playback control instruction to the display device.

[0135] In some optional embodiments, regardless of whether the interactive mode is triggered, the display device can continue to display the system main page or continue to play the media data. That is to say, during the interactive process, as long as no specific switching operation is performed, the system main page or media data will always remain displayed or played on the display.

[0136] Optionally, when the system home page is displayed on the monitor, the information on the page can be automatically updated during the user-triggered interaction, such as displaying new notifications, new push content, etc., to ensure that the user can always see the latest information, and the interaction process will not affect the content display of the system home page.

[0137] Optionally, in the case of displaying media data on a display, during the process of user triggering interaction, the display can play according to a predetermined play order and time, ensuring that the play of video, audio and other content will not be interrupted due to the interaction process.

[0138] Optionally, the user may also configure the interaction mode on the display device, such as setting it to pause the playback of media data during the interaction. In this case, when the user triggers the interaction, the media data being played will be paused, and will automatically continue to be played after the interaction ends, so as to prevent the user from focusing on the interactive content and response content during the interaction and missing important content in the media data.

[0139] In some embodiments, while the system main page is displayed or the media data is played, the controller controls the first response content to be displayed in the first display area of ​​the display. The first display area can be a specific part of the display, such as the upper right corner or lower left corner of the screen, or an independent area divided by split screen or the like. The display method of the first response content can be adjusted according to the specific situation. For example, the first response content can be displayed in a pop-up window, scrolling subtitles, icon prompts, etc. to ensure that the user can notice and obtain relevant information in time.

[0140] In this embodiment, the user can obtain the first response content without interrupting the display of the system main page or the playback of media data, thereby achieving a smoother and more convenient interactive experience. In addition, the controller can adjust the position, size and display mode of the first display area according to the needs and preferences of the user. For example, the user can customize the position of the first display area to better adapt to his or her usage habits; the controller can also automatically adjust the size and display effect of the first display area according to different devices and resolutions to ensure that the first response content can be displayed normally in various environments.

[0141] In some embodiments, the controller is further configured to:

[0142] Keep displaying the system main page, and display the first response content in a floating layer on the upper layer of the system main page; the first response content includes at least one media data; in response to a playback instruction for one of the media data, cancel the display of the system main page and the first response content, and display the playback page of the media data on the display; during the playback of the media data, the input module is also configured to receive other interactive content input by the user.

[0143] The display device keeps displaying the system home page during operation. The system home page may contain various menu options, icons, notifications and other information, providing users with access to different functional modules. After the user triggers the interaction, the controller will display the first response content in a floating layer on the upper layer of the system home page, realizing the layered display of the first response content and the system home page, so that the display of the system home page and the display of the first response content do not interfere with each other.

[0144] The first response content includes at least one media data. Media data can be information of multimedia content such as video, audio, image, such as video thumbnail, title, introduction, etc. Each media data is displayed in a specific format in the floating layer, which is convenient for users to quickly browse and select. For example, when the user issues a play command for one of the media data in the floating layer, the system responds immediately. The play command can be issued in a variety of ways, such as clicking the play button of the media data, moving the interface focus to the media data through the remote control and issuing a play command, and triggering automatic playback by hovering the mouse.

[0145] When the user selects a media data in the first response content and plays it, the controller will control the display of the playback page of the media data on the monitor to achieve full-screen playback of the media data, and cancel the display of the system main page and the first response content. That is, only the content at this level of the media data playback page is displayed, and the floating layer that blocks the media data playback page is removed, so that the user can understand the effective execution of the playback command.

[0146] Optionally, the playback page of the media asset data can be customized according to the type of the media asset data to provide the best playback experience. For example, for video media asset data, the playback page may include functions such as a video player, playback control buttons, and volume adjustment; for audio media asset data, the playback page of the media asset data may include functions such as an audio player, a playlist, and lyrics display.

[0147] Furthermore, during the playback of the media data, the input module is still activated and is configured to receive other interactive content input by the user. The other interactive content may include interactive content input by the user through voice, and may be other interactive content unrelated to the playback of the media data, such as searching for new media data, sending messages, etc.

[0148] In this embodiment, by keeping the system main page displayed, and displaying the first response content in a floating layer on the upper layer of the system main page, as well as receiving other interactive content input by the user during the media data playback process, independent control of the system display and playback process and the interactive process can be achieved, which can provide users with a more convenient, efficient and personalized interactive experience.

[0149] In some embodiments, the input module is further configured to: receive second interactive content input by the user, where the second interactive content is interactive content input after the display displays the first response content;

[0150] The controller is further configured to: move the first response content along the second direction in the first display area, and display the second response content for the second interactive content at the initial display position of the first response content.

[0151] After receiving the second interactive content, the controller first moves the first response content along the second direction in the first display area. The first display area is a specific area on the display for displaying the first response content and the second response content. The second direction can be a pre-set direction, such as from left to right, from top to bottom, etc.

[0152] Optionally, the method of moving the first response content can be designed according to the specific situation. For example, animation effects such as smooth scrolling and gradual disappearance can be used to make the movement of the first response content more natural and smooth.

[0153] Optionally, the display mode of the second response content may be similar to or different from that of the first response content, depending on the application scenario and design requirements. For example, different fonts, colors, icons, etc. may be used to distinguish the first response content from the second response content.

[0154] In this embodiment, the user can continue to input the second interactive content based on the first response content, and the controller will respond and display the corresponding second response content in a timely manner to achieve continuous interaction. By moving the display of the response results that have been displayed, the user can see the latest response content in the same area, providing the user with a smoother, continuous and flexible interactive experience.

[0155] In some embodiments, at least one of the first interactive content and the response content is voice content; the media data is audio data or audio-video data; the controller is also configured to: during voice input or voice output, reduce the playback volume of the audio data or audio-video data; in response to stopping voice input or stopping voice output, restore the playback volume of the audio data or audio-video data to the default state.

[0156] In a specific application, the user may provide the first interactive content by voice input, and the system may also provide the first response content by voice output. For example, the input module may include a voice input device such as a microphone to receive the user's voice input. The processing module may use voice recognition technology to convert the user's voice input into text content for processing. The output module may include a voice output device such as a speaker to output the system's response content to the user in the form of voice.

[0157] In some embodiments, the controller can monitor the state of voice input or output. When it is detected that the user starts voice input or the system starts voice output, the controller sends an instruction to the audio output management module of the audio data or audio and video data, requesting to reduce the playback volume of the audio data or audio and video data. The degree of volume reduction can be set according to actual conditions. For example, the volume can be reduced to a certain percentage, or the reduction range can be automatically adjusted according to the size of the ambient noise, or the audio of the audio data or audio and video data can be directly set to a silent playback state.

[0158] In some embodiments, the controller can restore the playback volume of the audio data or audio and video data to a default state in response to stopping voice input or stopping voice output. The default state can be a volume level preset by the user, or a volume setting set by the system by default. For example, the controller continuously monitors the state of voice input or output. When it is detected that the user stops voice input or the system stops voice output, the audio output management module of the audio data or audio and video data is requested to restore the playback volume of the audio data or audio and video data to the default state.

[0159] Optionally, the process of restoring the volume may adopt a smooth transition method to avoid discomfort to the user caused by sudden changes in the volume.

[0160] In this embodiment, voice content is supported in the first interactive content and the response content. By adjusting the playback volume of audio data or audio and video data during the voice interaction process, and restoring the playback volume to the default state when the voice input or output stops, it can ensure that the user can clearly hear his or her voice input or the voice output of the response content during the voice input or voice output process, avoid being disturbed by the media data being played, and improve the user experience.

[0161] In some embodiments, a display device is provided, the display device comprising:

[0162] The display is configured to display at least one display item.

[0163] The display item is the current display content or at least one historical response result generated based on the historical first interactive content. The specific display methods of the current display content and the historical response results can be referred to the above embodiments, which will not be described in detail here.

[0164] The input module is configured to receive first interactive content input by a user, where the first interactive content at least includes an ordinal number or a pronoun, and a keyword belonging to a target category.

[0165] The input module is an interface capable of receiving user operations. For the specific presentation form of the input module and the first interactive content, please refer to the above embodiments, which will not be described in detail here.

[0166] The controller is configured to respond to a control instruction corresponding to a first interactive content, take the content referred to by an ordinal word or a pronoun in a display item as an interactive target, and display a first response content provided by a target application of a target category on a display, wherein the first response content is a response result to the first interactive content and the interactive target.

[0167] The target category can be preset according to the specific application scenario. For example, in a display device, the target category can be the category for the application installed in the display device, such as weather, food, shopping, etc. For another example, in a shopping system, the target category can be electronic products, clothing, food, etc.

[0168] For the human-computer interaction system, target applications of various target categories are pre-installed in the storage module, and a correspondence between the application and the target category is established. For example, the weather application corresponds to the "weather" target category, the food application corresponds to the "food" target category, and the film and television application corresponds to the "film and television" target category. At the same time, the startup parameters and interface display parameters of the application are set so that the application interface can be quickly and accurately called up and displayed later.

[0169] For example, when the first interactive content input by the user through the input module contains weather-related keywords, such as "weather", "temperature", "rainfall", "sunny", etc., or the user explicitly mentions that he wants to check weather information, the controller determines that the interactive content belongs to the "weather" target category. If the first interactive content input by the user involves words related to food, restaurant, cooking, food evaluation, etc., such as "braised pork", "Sichuan restaurant", "food recommendation", etc., the controller determines it as the "food" target category. When the first interactive content input by the user contains words related to movies, TV series, variety shows, actors, directors, etc. related to film and television, such as "XXX (movie name)", "XXX (actor name) movie", "popular TV series", etc., the controller determines that the interactive content belongs to the "film and television" target category.

[0170] In some optional embodiments, after the user inputs the first interactive content, the processor first uses natural language processing technology to analyze the first interactive content to identify whether it contains keywords belonging to the target category. If the first interactive content contains keywords of the target category, the controller determines that the interactive content belongs to the corresponding target category. For example, the user inputs "How is the weather in this place?", where "weather" is a keyword of the target category, and the controller determines that the interactive content belongs to the weather category.

[0171] After determining the target category, the processor continues to analyze the first interactive content to identify whether it contains ordinal numbers or pronouns. Ordinal numbers are such as "first", "second", etc., and pronouns are such as "this", "that", etc. If the first interactive content contains ordinal numbers or pronouns, the processor uses the content referred to in the display item as the interactive target. For example, the user inputs "How is the weather in this place?", the controller recognizes the pronoun "this place" and uses the location indicated by the interactive target as the interactive target.

[0172] For example, if the location indicated by the interactive target is Guangzhou, then "how is the weather in this place" is rewritten as "how is the weather in Guangzhou", and detailed information about the weather in Guangzhou is provided through a weather category application. Users can directly view the specific weather information in Guangzhou through the weather application.

[0173] The interactive process provided by this embodiment has high flexibility and adaptability. Different target categories can have different keywords. The processor can accurately identify the target category to which the keyword belongs, and determine the interactive target indicated by the ordinal word or the pronoun, and provide the user with a personalized response result. For example, in the film and television category, the user enters "this movie is very good", the controller recognizes the pronoun "this", and finds the corresponding movie in the film and television work display item as the interactive target, and then displays the link provided by the film and television application as the response content.

[0174] In this embodiment, through the above display device, during the interaction process, the display item referred to by the ordinal number or the pronoun in the first interactive content can be clarified directly based on the current display content or at least one historical response result generated based on the historical interactive content. By taking the display item as the interaction target and, when the first interactive content contains keywords belonging to the target category, obtaining and displaying the first response content provided by the target application of the target category, the content displayed on the display and the interactive content input by the user can be effectively integrated, and an interactive response result with clear reference can be obtained. Moreover, cross-application interaction can be realized, thereby improving the accuracy of the interaction result.

[0175] In some embodiments, during the process of executing the first response content provided by the target application of the target category and displaying it on the display, the controller is also configured to: invoke the target application of the target category and display the application interface of the target application on the display; and display the first response content for the first interactive content and the interactive target in the application interface.

[0176] In the process of displaying the first interactive content, the controller can call up the corresponding application according to the correspondence between the target category and the application. Taking the target category as the weather category as an example, the controller calls up the weather application, and after the weather application is started, the display page of the weather application is displayed on the display, and the weather data of the specified location is obtained through the communication module, and the obtained weather data is displayed on the display page of the weather application.

[0177] For example, if the user's display items included content related to "Beijing" in the previous round of interaction, and the user inputs "How will the rainfall and temperature change here next week?" in the current round of interaction, the controller recognizes that "here" refers to "Beijing" indicated in the previous round of interaction, and directly rewrites "How will the rainfall and temperature change here next week?" to "How will the rainfall and temperature change in Beijing next week?". The controller will call up the weather application and display a weather forecast chart for Beijing next week on the application interface of the weather application, intuitively showing the rainfall and temperature change trends.

[0178] In this embodiment, by displaying the first response content in the target application, it is possible to support responses to further operations performed by the user in the target application. For example, the user can edit, save, share, and perform other operations on the first response content in the application interface, and the target application will process these operations in a timely manner and update the displayed content. This can provide users with a more accurate and efficient interactive experience and meet the needs of users in different application scenarios.

[0179] In some embodiments, in the process of displaying the first response content provided by the target application of the target category on the display, the controller is further configured to:

[0180] On the display, a first response content provided by a target application of a target category is displayed in a floating layer manner.

[0181] Among them, the floating layer is a semi-transparent or opaque display layer covering the display interface. The floating layer usually adopts specific visual effects, such as different colors, transparency, borders, etc., to make it more prominent on the display and attract the user's attention. The floating layer is relatively independent of the underlying interface and will not affect the layout and operation of the underlying interface. Users can view and operate the content on the floating layer without closing the underlying interface. The floating layer is usually displayed temporarily. When the user completes viewing or operating the first response content, the floating layer can be closed and restored to the original interface state. Users can move and adjust the position of the floating layer as needed to better adapt to different display environments and user operating habits.

[0182] Optionally, when it is determined that the first response content needs to be displayed, the controller creates a floating window on the display. The size and shape of the floating window can be dynamically adjusted according to the amount and type of the first response content. For example, if the first response content is a short text message, the floating window can be a smaller rectangle; if it is a complex message containing pictures and text, the floating window can be appropriately enlarged.

[0183] Optionally, the position of the floating window can be set according to the user's operating habits and the layout of the current interface. For example, it can be displayed in the center of the screen by default so that the user can see it quickly; it can also be dynamically adjusted according to the user's mouse pointer position or touch point position to make it closer to the user's operating focus.

[0184] Optionally, the background of the floating window can be semi-transparent to reduce the obstruction of the underlying interface while highlighting the content on the floating window. For example, light gray, light blue, etc. can be used as the background color of the floating window.

[0185] Optionally, the first response content is displayed in a floating window. The typesetting and layout of the content can be optimized according to its type to improve readability and usability. For example, for text information, a suitable font size and color can be used for clear typesetting; for image information, the size and position of the image can be appropriately adjusted so that it can be fully displayed in the floating window.

[0186] Optionally, a close button or operation icon can be set on the pop-up window so that the user can easily close the pop-up window when it is not needed. The close button is usually located in the upper right corner or upper left corner of the pop-up window. The user can close the pop-up window by clicking the button and restore to the original interface state. The operation icon can be set according to the type of the first response content. For example, if the first response content is a link, the operation icon can be an "open link" icon. The user can directly open the page corresponding to the link by clicking the icon.

[0187] In some feasible embodiments, the user can perform various interactive operations on the floating layer, and the controller will respond in a timely manner according to the user's operation. For example, the user can trigger a text link on the floating layer, and the controller will open the corresponding web page; the user can enter information in the input box on the floating layer, and the controller will search or submit data according to the input content.

[0188] In some optional embodiments, the controller can also provide corresponding feedback based on the user's operation. For example, when the user clicks a link, the controller can display a loading progress bar on the floating layer to prompt the user that the link is opening; when the user submits data, the controller can display a confirmation message on the floating layer to inform the user that the data has been successfully submitted.

[0189] In this embodiment, by displaying the first response content provided by the target application of the target category in a floating layer on the display, this implementation can provide users with a more convenient and efficient interactive experience without causing excessive interference to the underlying interface, thereby improving usability and user satisfaction.

[0190] In some embodiments, during the process of displaying the first response content provided by the target application of the target category in a floating layer manner, the controller is further configured to display the first response content provided by the target application of the target category in a card style.

[0191] A card is a carrier for displaying specific content. In some embodiments, the card is rectangular in shape and has a certain border and shadow effect to highlight the display on the display. The border color can be set according to the target category. For example, a weather card can have a light blue border, a food card can have an orange border, and a movie card can have a dark purple border.

[0192] Optionally, the background color of the card can be light-colored to improve readability and visual comfort. At the same time, different background patterns or textures can be set according to different target categories to increase the beauty and recognition of the card. For example, weather cards can use a background pattern of blue sky and white clouds, food cards can use a texture background of food pictures, and film and television cards can use a background pattern of movie posters.

[0193] Optionally, the top of the card usually displays the logo of the target category, such as "weather", "food", "film and television", etc., so that users can quickly identify the content type of the card. The logo can be in a larger font and eye-catching color to improve readability. The middle of the card is the main part of the first response content, and different layout designs are performed according to different target categories. For example, for weather cards, the middle can display the current weather icon (such as sunny, cloudy, rainy, etc.), temperature, humidity, wind direction and other information; for food cards, the middle can display restaurant names, dish pictures, food reviews and other information; for film and television cards, the middle can display movie posters, movie names, starring actors, ratings and other information. Some operation buttons or links can be set at the bottom of the card, such as "view more", "share", "collect", etc., so that users can perform further operations. The operation buttons or links can use smaller fonts and simple icons to avoid affecting the overall aesthetics of the card.

[0194] Optionally, when the user moves the focus of the control interface to a card, the card can display some interactive effects, such as brightening the border color, deepening the shadow, and appearing a floating prompt, to prompt the user that the card can be interactively operated. When the user triggers the card, the controller can respond accordingly based on the user's operation. For example, if the user triggers a weather card, the controller can expand and display more detailed weather forecast information; if the user triggers a food card, the controller can open the detailed introduction page of the restaurant; if the user triggers a movie card, the controller can play a movie trailer, etc.

[0195] Optionally, when the controller obtains the first response content of multiple target categories, multiple cards can be displayed on the display in a card style. The arrangement of the cards can be set according to the needs and habits of the user, for example, horizontal arrangement, vertical arrangement, grid arrangement, etc. can be used.

[0196] In order to facilitate users to manage and view cards, the controller can provide some card management functions. For example, users can adjust the position of cards by dragging them; users can close unnecessary cards by clicking the close button in the upper right corner of the card; users can quickly find cards of a specific target category through the search function, etc.

[0197] In this embodiment, by displaying the first response content provided by the target application of the target category in a card style, this implementation can provide users with a more intuitive, beautiful and convenient interactive experience, while also facilitating users to manage and view information of different target categories.

[0198] In some embodiments, during the process of displaying the first response content provided by the target application of the target category in a floating layer manner, the controller is further configured to display the first response content provided by the target application of the target category in a pop-up window style.

[0199] Among them, a pop-up window refers to a window that pops up automatically during the display page display process. When the controller obtains the first response content, the pop-up window will appear on the display with animation effects. The animation effect can be sliding in from the edge of the screen, fading in and out, zooming, etc., to enhance the user's visual experience.

[0200] The location of the pop-up window can be set according to the user's operating habits and the layout of the display. For example, it can appear in the center of the screen by default so that the user can see it quickly; it can also be dynamically adjusted according to the user's mouse pointer position or the interface focus position to make it closer to the user's operating focus.

[0201] Optionally, when the user moves the interface focus to the pop-up window, the pop-up window can display some interactive effects, such as brightening the border color, deepening the shadow, and appearing a floating prompt, to prompt the user that the pop-up window can be interactively operated. When the user clicks the operation button in the pop-up window, the controller will respond accordingly according to the user's operation. For example, when the user clicks the "Close" button, the pop-up window will disappear with an animation effect; when the user clicks the "View More" button, the controller will open the corresponding page and display more detailed information; when the user clicks the "Share" button, the controller will provide a sharing function, allowing the user to share the first response content to social media or other platforms.

[0202] Optionally, in order to avoid too many pop-up windows affecting the user experience, the controller can manage the pop-up windows. For example, when the first response content of multiple target categories needs to be displayed in the form of pop-up windows at the same time, the controller can display the pop-up windows in sequence according to priority or time order, or merge multiple pop-up windows into a comprehensive pop-up window for display. The controller can optimize the pop-up windows based on user feedback and usage habits. For example, if the user often closes the pop-up windows of a certain target category, the controller can reduce the frequency of such pop-up windows in subsequent interactions; if the user frequently clicks on the operation button of a pop-up window, the controller can optimize the position and function of the button to improve the user's operational convenience.

[0203] In this embodiment, by displaying the first response content provided by the target application of the target category in a pop-up window style, this implementation can provide users with timely and intuitive information feedback, while also facilitating user interactive operations, thereby improving interactivity and user experience.

[0204] In some embodiments, the first response content includes multiple candidate items; the controller is further configured to: display an application interface of the target application in response to a trigger operation on one of the candidate items; and display detailed content of the triggered candidate item in the application interface.

[0205] When the target application generates the first response content according to the first interactive content, the controller analyzes and organizes the response content and divides it into multiple candidate items for display. For example, if the first interactive content is "find nearby restaurants", the target application may return multiple nearby restaurant information as candidate items for display on the display page, each candidate item including restaurant name, address, rating and other information.

[0206] The display method of the candidate items can be designed according to the specific situation. For example, it can be displayed on the display in the form of a list, with each candidate item occupying a line, clearly showing its key information; it can also be displayed in the form of icons plus text, making the candidate items more intuitive and easy to identify.

[0207] The user can trigger the candidate through the input device. The trigger operation can be triggered by the control device (remote control), touching the screen, keyboard input, etc. For example, the user can use the control device to select a candidate, or touch a candidate on the touch screen to trigger the corresponding operation. When the controller detects the user's trigger operation on a candidate, it will immediately respond and perform the corresponding operation. For example, if the user clicks on a restaurant candidate, the controller will call up the application interface of the target application and display the details of the triggered restaurant candidate in the interface.

[0208] When the user triggers a candidate, the controller displays the target application's application interface on the display. The layout and design of the application interface can be customized according to the characteristics of the target application and user needs. For example, for a restaurant search application, the application interface may include elements such as a map display, a restaurant list, and a search box.

[0209] The details of the triggered candidate items are displayed in the application interface. The details may include detailed descriptions, pictures, user reviews, contact information, etc. For example, if a user triggers a restaurant candidate, the application interface will display the restaurant's detailed description, dish pictures, customer reviews, address, phone number, etc.

[0210] To improve the user experience, the application can be adapted to the display device to optimize and layout the details. For example, clear fonts and layouts can be used to display the details to make the information easier to read; functions such as image zooming and sliding can be provided to facilitate users to view images; important information such as ratings and special dishes can be highlighted to attract users' attention.

[0211] In this embodiment, users can select different candidate items to trigger according to their needs and interests, and directly obtain the corresponding detailed content from the application by opening the application. By providing multiple candidate items in the first response content, and displaying the application interface and detailed content of the target application in response to the user's triggering operation on the candidate items, this embodiment can provide users with a more convenient, efficient and personalized interactive experience to meet the needs of users in different scenarios. At the same time, the controller can also continuously optimize the generation and display methods of candidate items according to user feedback and usage habits to improve the efficiency and quality of interaction.

[0212] In some embodiments, the controller is further configured to display a third response result to the third interactive content in response to the third interactive content input by the user when the display displays the application interface of the target application.

[0213] After the controller determines the target category and evokes the corresponding target application based on the first interactive content input by the user, the application interface of the target application will be displayed on the display. When the application interface of the target application is displayed, the user can input the third interactive content through the input device. The controller monitors the user's input operation in real time and identifies and analyzes the third interactive content. The identification process can use natural language processing technology, keyword matching and other methods to determine the meaning and intention of the third interactive content.

[0214] For example, if the user enters "Will it rain tomorrow?" on the interface of the weather application, the controller will recognize that this is a question about the weather. If the user enters "Where is the address of this restaurant?" on the interface of the food application, the controller will recognize that this is a question about the location of the restaurant.

[0215] When the controller recognizes the third interactive content, it will generate a corresponding third response result according to the meaning and intention of the third interactive content. The third response result can be generated by querying a database, calling a related service interface, performing calculations, etc.

[0216] The third response result will be displayed to the user on the display. The display method can be designed according to the type and content of the third response result. For example, if the third response result is text information, it can be directly displayed in a specific area of ​​the application interface; if the third response result is a picture, chart, etc., it can be displayed in the form of a pop-up window, a new page, etc.

[0217] For example, if a user enters "Will it rain tomorrow?" on the interface of a weather application, the controller will query the weather database, generate a third response result on whether it will rain tomorrow, and display "There is a [X]% probability of rain tomorrow" on the application interface. If a user enters "Where is the address of this restaurant?" on the interface of a food application, the controller will query the restaurant database, obtain the address information of the restaurant, and display "Restaurant address: [specific address]" on the application interface.

[0218] In this embodiment, by displaying the application interface of the target application on the display, in response to the third interactive content input by the user, the third response result for the third interactive content is displayed. When the target application is opened, the response to the user input can continue to be achieved to achieve continuous multiple rounds of interaction, thus avoiding the interruption of the interaction process caused by opening the application, and providing users with a more convenient, efficient and personalized interactive experience.

[0219] In some embodiments, in the process of executing the third interactive content in response to the user input and displaying the third response result to the third interactive content, the controller is further configured to:

[0220] In response to the third interactive content input by the user, the target application is exited, and a third response result for the third interactive content is displayed.

[0221] The third interactive content is the input content received after completing the interactive response to the first interactive content. The user can input the third interactive content through the input device during the use of the target application. The input device can be a control device (such as a remote control), a keyboard, a mouse, a touch screen, a voice input device, etc. The controller monitors the user's input operation in real time, and identifies and analyzes the received third interactive content. The recognition process can use natural language processing technology, keyword matching and other methods to determine the meaning and intention of the third interactive content. For example, if the user enters "Exit Weather Query" while using the weather application, the system will recognize that this is an instruction to exit the weather application; if the user enters "Query Movie Information", the system will recognize that this is an instruction to exit the weather application to display the movie information query results or an instruction to switch to the film and television target application.

[0222] When the controller recognizes that the third interactive content is an instruction to exit the target application, it will immediately respond and execute the exit operation. The exit operation includes closing the application interface of the target application, releasing related resources, stopping processes related to the target application, etc. Optionally, before exiting the target application, the controller can perform some cleanup and save operations as needed. For example, if the user has unsaved operations or data in the target application, the controller can prompt the user to save; if the target application has ongoing tasks in the background, the controller can pause or cancel these tasks. Optionally, the method of exiting the target application can be selected according to the user's habits and the design of the controller. For example, you can directly close the application interface, or you can use a gradual exit method, first returning to the previous menu, and then exiting the application.

[0223] If the third interactive content is not an instruction to exit the target application, but other types of instructions, the controller will generate a corresponding third response result based on the third interactive content and display it on the display. The way the third response result is generated depends on the specific meaning of the third interactive content and the function of the controller. For example, if the third interactive content is to query movie information, the controller will call up the film and television target application and display the third response result of movie information for the instruction on the display; if the third interactive content is to ask a question, the controller will generate a third response result by searching the knowledge base or calling related services, and display it on the display.

[0224] Optionally, the display method of the third response result can be designed according to the type of content and user needs. For example, the third response result can be displayed in the form of text, image, chart, etc.; it can be displayed in a separate window or in a specific area of ​​the current interface.

[0225] In this embodiment, by exiting the target application in response to the third interactive content input by the user and displaying the third response result for the third interactive content, the target application can be opened and exited quickly and conveniently without interrupting the interaction process, thereby providing users with a more convenient, efficient and flexible interaction experience.

[0226] In some embodiments, the controller is further configured to: start the interaction mode when receiving interactive content input by the user; maintain the interaction mode in an on state during multiple rounds of interaction; and exit the interaction mode when it is detected that the interaction end condition is met.

[0227] When the controller receives interactive content input by the user, the interactive mode is immediately turned on. The interactive content can be in various forms such as text input through the keyboard, voice input, touch screen input, etc. After turning on the interactive mode, the controller performs a series of initialization operations, including but not limited to: allocating resources for interactive processing, setting interactive status flags, starting related interactive processes, etc. For example, when the user enters a question or instruction in the input box, the controller detects the input content and determines it as a trigger condition for turning on the interactive mode. It then starts the interactive mode and displays interactive prompt information on the display, such as "Interaction is in progress, please continue to enter."

[0228] During multiple rounds of interaction, the controller always keeps the interaction mode on. Multiple rounds of interaction refer to multiple consecutive interactions between the user and the controller to gradually clarify needs or obtain more detailed information. The controller continuously monitors the user's input and processes and responds accordingly based on the input content. After each round of interaction, the controller waits for the user's next input without turning off the interaction mode. For example, the user first enters "Find nearby restaurants", and the controller returns a list of some restaurants. The user then enters "Is there a Chinese restaurant?", and the controller again filters out Chinese restaurants and displays them. During this process, the interaction mode remains on until the user completes the required interaction operations.

[0229] The controller continuously detects whether the interaction end condition is met. The interaction end condition can be set according to the specific application scenario and needs, for example: the user explicitly inputs an instruction to end the interaction (such as "end interaction", "exit", etc.); no user input is detected for a period of time; a specific interaction task is completed, etc. When it is detected that the interaction end condition is met, the controller exits the interaction mode. Exiting the interaction mode includes operations such as releasing resources occupied during the interaction process, clearing the interaction status flag, and stopping related interaction processes. For example, if the user enters "end interaction", the controller recognizes that this is an instruction to end the interaction, and immediately exits the interaction mode, displaying a prompt message "Interaction has ended" on the display. Alternatively, if the controller does not detect the user's input within a certain period of time, it will automatically exit the interaction mode to save controller resources.

[0230] In this embodiment, by turning on the interaction mode when receiving interactive content input by the user, maintaining the interaction mode in an turned-on state during multiple rounds of interaction, and exiting the interaction mode when it is detected that the interaction end condition is met, it can ensure that the interaction process continues, and the interaction will not be interrupted due to other interference factors, thereby improving the interaction experience.

[0231] In some embodiments, during the process of exiting the interactive mode when it is detected that an interaction termination condition is satisfied, the controller is further configured to: when the interactive mode is in an on state, exit the interactive mode in response to a received exit control signal.

[0232] When the controller receives specific interactive content input by the user (such as instructions to start interaction, specific questions, etc.), the interactive mode is turned on. After the interactive mode is turned on, the controller sets a flag or state variable to indicate that the interactive mode is on. The controller continuously monitors the state of the interactive mode in order to respond to various operations in a timely manner. For example, while the interactive mode is on, the controller can display specific interactive interface elements on the display, such as prompt information, interactive progress bar, etc., to indicate to the user that the interactive mode is currently in progress.

[0233] Users can input various signals through input devices to control the interaction process. When the interactive mode is on, the controller continuously monitors the input device to detect whether an exit control signal is received. The exit control signal can be in various forms, such as a specific physical button on the display device, a special button on the control device, a specific virtual button on the display interface, etc. The controller analyzes and identifies the input signal to determine whether it is an exit control signal.

[0234] When the controller recognizes the exit control signal, it immediately executes the operation of exiting the interactive mode. These operations include but are not limited to: clearing the data and status related to the interaction, closing the interactive interface, releasing the controller resources occupied during the interaction, etc. For example, if the interactive mode involves communication with an external server, the controller can disconnect from the server when exiting the interactive mode; if specific interactive interface elements are displayed on the display during the interaction, the controller can hide or close these elements; if temporary files or data structures are created during the interaction, the controller can delete these temporary resources when exiting the interactive mode.

[0235] After exiting the interactive mode, the controller can control the display page to return to the initial state or display other interfaces in non-interactive mode, for example, return to the main menu interface, display the default information page, etc.

[0236] In this embodiment, when the interactive mode is in the on state, the interactive mode can be directly exited in response to a received exit control signal, thereby providing the user with a flexible interactive control method.

[0237] In some embodiments, when executing the process of exiting the interaction mode when it is detected that the interaction end condition is met, the controller is also configured to: exit the interaction mode when the interaction mode is in the on state and the duration of not receiving the interaction content reaches a preset time.

[0238] When the interactive mode is on, the controller starts a timer to monitor the duration of time when no interactive content is received. Whenever the controller receives new interactive content, the timer is reset to zero and restarts. If no new interactive content is received within a certain period of time, the timer continues to accumulate time. The controller constantly compares the accumulated time of the timer with the preset time. The preset time can be set according to actual application requirements, for example, it can be set to several minutes, tens of seconds, etc.

[0239] If the duration of not receiving interactive content reaches a preset duration, the controller determines that the conditions for exiting the interactive mode are met and starts to execute the operation of exiting the interactive mode. If the controller receives new interactive content before the preset duration is reached, the timer is reset to zero again and continues to monitor the duration of not receiving interactive content. If the user triggers the start condition of the interaction again after exiting the interactive mode, the controller can reopen the interactive mode and perform the corresponding initialization operation.

[0240] In this embodiment, when the interactive mode is on and the duration of not receiving interactive content reaches a preset duration, by exiting the interactive mode, interactive resources can be reasonably utilized, the efficiency and stability of the system can be improved, and a flexible interactive experience can be provided to users.

[0241] In some embodiments, when executing the process of exiting the interaction mode when it is detected that the interaction end condition is met, the controller is also configured to: exit the interaction mode when the interaction mode is in the on state and the keywords contained in the received interaction content hit the exit instruction.

[0242] For voice commands, the controller can use voice recognition technology to convert the user's voice input into text, and then perform keyword matching to determine whether it is an exit control signal.

[0243] Optionally, the controller pre-sets a series of keywords related to exiting the interactive mode as signs of the exit command. These keywords can be set according to actual application requirements, such as "exit", "end", "stop", etc. When the controller recognizes that the interactive content contains preset exit command keywords, it is determined that the exit command is hit. For example, if the user enters "I want to exit the interaction", the controller recognizes the keyword "exit", and thus determines that the exit command is hit.

[0244] The controller determines that the interactive content hits the exit command, and the controller starts to execute the operation of exiting the interactive mode. In order to ensure the stability and fault tolerance of the interactive mode, the controller can optimize the recognition of keywords. For example, for keywords that may be ambiguous, they can be analyzed in combination with the context to determine whether they really represent an exit command. The controller can also set a certain confirmation mechanism. When a possible exit command keyword is identified, the controller can display a confirmation message on the display, asking the user whether they are sure to exit the interactive mode to avoid misoperation.

[0245] In this embodiment, when the interactive mode is on and the keywords contained in the received interactive content hit the exit command, exiting the interactive mode can provide the user with a simple and flexible interactive control method while ensuring the stability and reliability of the interactive system.

[0246] In some embodiments, during the process of exiting the interaction mode when it is detected that an interaction termination condition is met, the controller is further configured to exit the interaction mode when the interaction mode is in an on state and the number of interaction rounds reaches a preset threshold number.

[0247] When the interactive mode is turned on, the controller starts an interactive round counter. Every time the controller receives an interactive content from the user and responds accordingly, it is considered to have completed a round of interaction, and the value of the counter increases by one. The controller continuously monitors the value of the interactive round counter and compares it with the preset number threshold. The preset number threshold can be set according to actual application requirements, for example, it can be set to 5 times, 10 times, etc. If the interactive round reaches the preset number threshold, the controller determines that the conditions for exiting the interactive mode are met and starts to execute the operation of exiting the interactive mode.

[0248] In order to improve the flexibility and adjustability of the interaction mode, the controller can allow users or administrators to adjust the preset times threshold according to actual conditions. For example, in some complex interaction scenarios, the preset times threshold can be increased to allow more interaction rounds; in simple interaction scenarios, the preset times threshold can be lowered to exit the interaction mode faster. The controller can also provide a visual display of the interaction rounds, such as displaying the current number of interaction rounds and the preset times threshold on the interaction interface, so that the user can understand the progress and status of the interaction.

[0249] In this embodiment, when the interaction mode is turned on and the number of interaction rounds reaches a preset threshold, by exiting the interaction mode, the interaction process and resource usage can be reasonably controlled, while providing the user with an adjustable interaction experience.

[0250] In some embodiments, an interactive system is provided, the interactive system comprising a display device and a server. The display device is used to display content and receive interactive content from the user, and report target data to the server when receiving the interactive content. The server is responsible for processing the target data, generating response content and feeding it back to the display device for display.

[0251] In some embodiments, an interaction method is provided, such as Figure 6 As shown, the interactive method can be implemented by a server, and the interactive method includes the following steps:

[0252] Step 602: Acquire target data reported by the display device when receiving interactive content.

[0253] The target data includes interactive content and interface display information when the interactive content is received. The interface display information includes at least one display item, which is the current display content or at least one historical response result generated based on the historical interactive content.

[0254] Step 604: When the interactive content includes an ordinal number or a pronoun, the interactive content is rewritten based on the display item in the interface display information to obtain the rewritten content.

[0255] Step 606: Generate response content for the rewritten content, and feed the response content back to the display device for display.

[0256] When the display device receives the interactive content input by the user, the target data is immediately reported to the server. The target data includes the interactive content itself and the interface display information when the interactive content is received. The interface display information includes at least one display item. The display item can be the content currently being displayed, such as text, image, video, etc.; it can also be at least one historical response result generated based on the historical interactive content, such as previous query results, recommended content, etc.

[0257] After receiving the target data, the server analyzes whether the interactive content contains ordinal numbers or pronouns. Ordinal numbers include "first", "second", etc., and pronouns include "this", "that", etc. If the interactive content contains ordinal numbers or pronouns, the server rewrites the interactive content based on the display items in the interface display information. The specific rewriting method is as follows:

[0258] For ordinal numbers, the corresponding display items in the display interface are determined. For example, if the interactive content is "I want to watch the second program", the server determines that the specific content corresponding to the "second result" is the variety show "AAA (variety show name)" according to the order of the display items in the display interface, and rewrites the interactive content into a description of the specific content "I want to watch the variety show "AAA (variety show name)"".

[0259] When the server detects that the interactive content contains ordinal numbers, it first determines the number and order of display items in the display interface. If the display item is the current display content, the server will analyze the layout and structure of the current page to determine the order of each display item. If the display item is a historical response result generated based on historical interactive content, the server will determine the order of the display items based on the time sequence of generation or specific tags.

[0260] The server determines the corresponding specific display item according to the position and meaning of the ordinal word in the interactive content. For example, if the interactive content is "I want to watch the second show", assuming that the current interface displays a set of search results, namely "AAA (variety show name)" "BBB (variety show name)" "CCC (variety show name)". The server will find the second display item "BBB (variety show name)" according to the determined display item sequence, and rewrite the interactive content into "I want to watch the variety show "BBB (variety show name)"".

[0261] In some complex interfaces, there may be multiple levels of display items. For example, there are multiple categories on a page, and each category has multiple specific items. Suppose the current interface has three categories, namely "electronic products", "household items", and "clothing accessories", and each category has multiple specific product displays. If the interactive content is "view the details of the first item in the third category", the server will first find the third category "clothing accessories", and then find the first item under this category, assuming it is "fashion T-shirts", and rewrite the interactive content to "view the details of fashion T-shirts in clothing accessories".

[0262] For the pronoun, the server can determine the specific display item it refers to based on the context and the display interface. For example, if the interactive content is "this is interesting", the server determines that the specific display item referred to by "this" is game A based on the content displayed on the current interface and the context of the interaction, and rewrites the interactive content into the description of the display item "game A is interesting".

[0263] For pronouns, the server will combine the context and interface display information to determine the specific display item it refers to. First, the server will analyze the context of the interactive content to understand the user's intention and focus in the current interaction. For example, if a user enters "this looks good" after browsing a list of products, the server will determine that "this" may refer to a product that the user has just browsed based on the user's previous browsing history and the display items on the current interface, and then rewrite it based on the product.

[0264] The server will analyze the interface display information, including the currently displayed content and historical response results. If there are multiple display items on the current page, the server will determine the display item that the pronoun is most likely to refer to based on factors such as the location of the pronoun, the layout of the interface, and the user's operating habits. For example, the current interface displays three mobile phones, namely "Brand A Mobile Phone", "Brand B Mobile Phone", and "Brand C Mobile Phone". The user has just swiped and browsed on these three mobile phones, and then entered "How is the performance of this one?" The server may think that "this" refers to the mobile phone within the user's current field of vision, assuming it is "Brand B Mobile Phone", and rewrite the interactive content to "How is the performance of Brand B Mobile Phone?". The server analyzes the rewritten content and generates response content for the rewritten content. The generation of response content can adopt different methods and algorithms according to specific application scenarios and needs.

[0265] For example, if the rewritten content is a question, the server can obtain the answer by querying the database, calling external services, etc., and use the answer as the response content. If the rewritten content is an instruction, the server can perform the corresponding operation and use the operation result as the response content. The server feeds the generated response content back to the display device for display. After receiving the response content, the display device displays it on the interface for the user to view.

[0266] Optionally, in order to improve the performance and accuracy of the interaction process, the judgment and rewriting process of the interaction content can be optimized. For example, more advanced natural language processing technology can be used to improve the recognition accuracy of ordinal words and pronouns; the referent of a pronoun can be determined more accurately based on the user's historical behavior and preferences.

[0267] In this embodiment, during the interaction process, the display item referred to by the ordinal number or the pronoun in the first interactive content can be clarified directly based on the current display content or based on at least one historical response result generated based on the historical interactive content, and the interactive content is rewritten based on the display item in the interface display information to obtain the rewritten content. This can achieve effective fusion of the content displayed on the display and the first interactive content input by the user, and obtain an interactive response result with clear reference, thereby improving the accuracy of the interactive result.

[0268] In some embodiments, when the interactive content includes an ordinal number or a pronoun, the interactive content is rewritten based on the display item in the interface display information to obtain the rewritten content, including:

[0269] When the interactive content includes an ordinal number or a pronoun and there is an interface focus in the interface display information, a target display item referred to by the ordinal number or the pronoun is determined based on the display items and the interface focus in the interface display information; and the interactive content is rewritten according to the display content of the target display item to obtain the rewritten content.

[0270] The interface display information includes at least one display item, which can be the text, image, video and other content currently being displayed, or the historical response results generated based on the historical interactive content. For example, in a shopping application, the display items can be product images, product descriptions, user reviews, etc.; in a news application, the display items can be news titles, news summaries, news pictures, etc. When the display device is running, if there are multiple options in the interface of the display device, an interface focus will be displayed on the interface. The interface focus can be controlled and moved in a variety of ways, such as through user operations (such as remote control, touch screen, etc.). The interface focus is usually displayed with specific visual effects, such as highlighting, bold borders, etc., so that users can clearly identify it. For example, in a web browser, when a user selects a link, the area where the link is located may become the interface focus; when entering an image browsing application, the first image displayed is the interface focus.

[0271] If the interactive content contains an ordinal number or a pronoun, and there is an interface focus in the interface display information, the server will determine the target display item referred to by the ordinal number or the pronoun based on the display items and the interface focus in the interface display information.

[0272] For ordinal numbers: the server first determines the number and order of display items in the interface display information. If the display items are arranged according to specific rules, such as time order, alphabetical order, etc., the server can determine the corresponding specific display item based on the position and meaning of the ordinal number in the interactive content. For example, if the interactive content is "view the details of the second product", the server will find the second product as the target display item based on the order of product display on the interface.

[0273] For pronouns: the server will determine the target display item that the pronoun refers to based on the interface focus and context. If the pronoun appears near the interface focus or in interactive content related to the interface focus, the server can assume that the pronoun refers to the display item where the interface focus is. For example, if a user is browsing a list of products and the current interface focus is on a certain product, and the user enters "how much is this product", the server can determine that "this" refers to the product where the interface focus is, that is, the target display item.

[0274] After determining the target display item, the server will rewrite the interactive content according to the display content of the target display item to obtain the rewritten content. The purpose of rewriting is to make the interactive content clearer and more specific so that the server can better understand the user's needs and generate an accurate response.

[0275] For example, if the target display item is a product and the interactive content is "How is this product?", the server can rewrite the interactive content into "How is [product name]?", where "[product name]" is the specific product name of the target display item. If the target display item is a news article and the interactive content is "What is the source of this news?", the server can rewrite the interactive content into "What is the source of [news title]?", where "[news title]" is the news title of the target display item.

[0276] In this embodiment, when the interactive content includes an ordinal number or a pronoun and there is an interface focus in the interface display information, the target display item referred to by the ordinal number or the pronoun is determined based on the display item in the interface display information and the interface focus, so that the target display item can be quickly located, and the interactive content can be rewritten according to the display content of the target display item, which can improve the description accuracy of the rewritten interactive content, thereby obtaining an accurate interactive response result, and improving the intelligence of the interactive system and user experience.

[0277] In some embodiments, the method further includes: performing semantic detection on the rewritten content to obtain a semantic detection result; generating response content for the rewritten content, including: generating response content based on the rewritten content when the semantic detection result is a passed detection.

[0278] The purpose of semantic detection of rewritten content is to determine whether the rewritten content has clear semantics and whether it can be accurately understood and processed by the system. Semantic detection can be carried out in a variety of ways, such as semantic analysis algorithms in natural language processing technology, machine learning models, etc. If the semantics of the rewritten content are clear and unambiguous and can be correctly parsed and processed by the system, the semantic detection result is passed; if the rewritten content has ambiguous semantics, is incomplete, or cannot be understood by the system, the semantic detection result is failed.

[0279] For example, for the rewritten content "Who plays the emperor in [film and television work]", the server can perform semantic detection by analyzing whether [film and television work] has been released and whether the grammatical structure is correct. If the character "emperor" exists in [film and television work] and the grammatical structure is correct, the semantic detection result is passed; if the character "emperor" does not exist in [film and television work] or the grammatical structure is wrong, the semantic detection result is failed.

[0280] If the semantic detection result is passed, the server generates response content based on the rewritten content. For example, if the rewritten content is "Who plays the emperor in [film and television work]", the system will query the role and actor information of the film and television work, and generate the name of the actor of the role of "Emperor" as the response content. The response content can be in text form, such as "The emperor in [film and television work] is played by [Zhang San]", or in other forms, such as video, image, etc., depending on the application scenario and user needs.

[0281] If the semantic detection result is failed, the response content is generated based on the interaction content before rewriting. The system will try to generate the response content based on the original interaction content using different processing methods.

[0282] For example, if the original interactive content is "Who plays the emperor in [film or TV work]", and the semantic detection fails, the system can ask the user a question, such as "Please clarify which film or TV work you are referring to", to obtain more information to generate accurate response content.

[0283] In this embodiment, by performing semantic detection on the rewritten content and generating response content based on the rewritten content if the semantic detection passes, the semantic detection result can be used as a basis for judging whether to use the rewritten content for interaction, in case the rewritten content cannot be accurately understood and processed by the system, thereby improving the accuracy of the response results in the interaction process based on the semantic detection rule.

[0284] In some embodiments, an interaction method is provided, such as Figure 7 The interactive method may be implemented by a server, and the interactive method includes the following steps:

[0285] Step S702: Acquire target data reported by the display device when receiving interactive content.

[0286] The target data includes interactive content and interface display information when the interactive content is received, the interface display information includes at least one display item, and the display item is the current display content or at least one historical response result generated based on the historical interactive content;

[0287] Step S704: When the interactive content includes ordinal numbers or pronouns, and keywords belonging to the target category, the interactive content is rewritten based on the display items in the interface display information to obtain rewritten content.

[0288] Step S706 , calling the target application of the target category, obtaining response content for the rewritten content based on the application, and feeding back the response content to the display device for display.

[0289] When the display device receives the interactive content input by the user, the target data is immediately reported to the server. The target data includes the interactive content itself and the interface display information when the interactive content is received. The interface display information includes at least one display item. The display item can be the content currently being displayed, such as text, image, video, etc.; it can also be at least one historical response result generated based on the historical interactive content, such as previous query results, recommended content, etc.

[0290] After receiving the target data, the server analyzes whether the interactive content contains ordinal numbers or pronouns. Ordinal numbers are such as "first", "second", etc., and pronouns are such as "this", "that", etc. If the interactive content contains ordinal numbers or pronouns, the server rewrites the interactive content based on the display items in the interface display information.

[0291] After receiving the target data, the server first analyzes whether the interactive content contains ordinal numbers or pronouns and keywords belonging to the target category. The target category can be pre-set according to the specific application scenario. For example, in a shopping system, the target category can be electronic products, clothing, food, etc.; in a news information system, the target category can be politics, economy, entertainment, etc.

[0292] If the interactive content contains ordinal numbers or pronouns and keywords of the target category, the server rewrites the interactive content based on the display items in the interface display information. The specific rewriting method is as follows:

[0293] For ordinal numbers, the server determines the corresponding display item in the display interface. For pronouns, the server determines the specific display item it refers to based on the context and the display interface.

[0294] After determining the target category to which the interactive content belongs, the server calls the application of the corresponding category. For example, if the interactive content involves the electronic product category, the server calls the electronic product-related application. After the application is called, the server passes the rewritten interactive content to the application so that the application can process and query according to the specific interactive content.

[0295] The target application of the target category processes the rewritten interactive content and obtains the response content for the rewritten content by querying the database, calling external services, etc. The response content can be in various forms, such as a detailed introduction of a product, the content of a news article, the answer to a question, etc. The server feeds the obtained response content back to the display device for display. After receiving the response content, the display device displays it on the interface for the user to view.

[0296] In this embodiment, during the interaction process, the display item referred to by the ordinal number or the pronoun in the first interactive content can be clarified directly based on the current display content or at least one historical response result generated based on the historical interactive content. By taking the display item as the interactive target and, when the first interactive content contains keywords belonging to the target category, the interactive content is rewritten based on the display item in the interface display information, and the first response content provided by the target application of the target category is obtained and displayed. This can achieve effective integration of the content displayed on the display and the interactive content input by the user, obtain an interactive response result with a clear reference, and achieve cross-application interaction, thereby improving the accuracy of the interactive result.

[0297] In one of the optional embodiments, an interaction method is provided, which is applied to a server that is communicatively connected to a display device.

[0298] like Figure 8 As shown, taking the display device as a television as an example, the server includes a multi-round rewriting module, a natural language understanding module, a dialogue management module and a natural language generation module.

[0299] During the interaction process, the user inputs the interactive content through language input or remote control operation. After the TV receives the interactive content input by the user, it reports the real-time scene information representing the real-time display screen of the TV and the input interactive content (i.e., dialogue information) to the server. The multi-round rewriting module in the server will rewrite the interactive content and output the rewriting result to the natural language understanding module. After the natural language understanding module performs NLU analysis on the rewriting result, the parsing result is sent to the dialogue management module. The dialogue management module makes dialogue decisions based on the parsing result, generates business results, sends them to the natural language generation module to obtain response results, and feeds them back to the TV for display to the user, thereby realizing an accurate and reliable human-computer interaction process.

[0300] Among them, Fig. 9 As shown, the multi-round rewriting module specifically includes a dialogue preprocessing module, an admission judgment module, an instruction construction module, a model rewriting module and an exit judgment module.

[0301] The input of the dialogue preprocessing module is the current round request of the user interacting with the voice assistant, the current terminal scene information, the previous n rounds of dialogue history and the terminal scene information. Taking the request in the example as an example, when the user request 2 enters the module, the current round request refers to "I want to watch the movie she and Lei (actor name) have acted in", and the dialogue history refers to the user requests, system responses and scene information of each round in the previous two rounds.

[0302] The main function of the dialogue preprocessing module is to integrate various information in the dialogue history and terminal scene information. For example, the historical scene information also contains the focus information that has been reported. In actual applications, if new scene information is reported, it means that the focus on the terminal scene has been updated. Through preprocessing, only the focus information closest to the current round is retained, and other focuses are invalidated. The advantage of this is that it can reduce the ambiguity of downstream understanding.

[0303] The admission judgment module is used to make admission judgments for multiple rounds of rewriting of the user input in this round in combination with the above information. Admission judgment can be implemented through rule methods and classification models. Among them, the rule method is mainly used to filter the conversation history by judging whether there is a matching intention pair between the historical input of a certain round and the historical input of the next round. If there is a matching intention pair, the historical information of this round is retained, otherwise the historical information of this round and its previous round of conversation history will not participate in the subsequent multiple rounds of rewriting. The classification model uses a deep learning model to implement the screening of user requests, such as a text classification model based on Bert or LSTM network. In actual applications, we use the bert+linear model structure to determine whether multiple rounds of rewriting are required through the binary classification of the user's current request.

[0304] Input: input = {[CLS]w1 w2…w3[SEP]}, [CLS] and [SEP] are special tokens in the vocabulary, the output vector corresponding to [CLS] is used as the semantic representation of the entire context and is input into the linear layer to train the classification task, w1 w2…wn represents the current sentence, and [SEP] is the separator between two sentences. If only two rounds of completion are performed, the two sentences of the previous request and the current request can be concatenated and encoded to obtain more semantic information. However, considering that in more rounds of dialogue, if all the previous sentences are concatenated, the model reasoning time will be longer, and if each round is concatenated and called in parallel with the current round, it will involve the priority issues of the nearest and cross-rounds, which will make the decision more complicated. Considering the role of this model in the link, it is mainly to filter out as many semantically complete or meaningless statements as possible, reduce the large model call, the error rate of multi-round rewriting, save resource overhead and reduce the link time. Therefore, only the user request of the current round is used to determine whether to allow admission, and the accuracy rate of negative samples and the recall rate of positive samples are guaranteed to be high during training. For example, the interactive content is "Who acted in this movie?", and the input is "[cls] Who acted in this movie [SEP]". The specific processing process is as follows: Fig.10 As shown, the final output is: whether to enter multiple rounds of rewriting (0 or 1).

[0305] Through access judgment, the conversation history information can be significantly reduced, the pressure on downstream large model services can be reduced, and endless rounds of rewriting can be avoided. It can also flexibly control whether to perform multiple rounds of understanding under different intentions.

[0306] The multi-round rewriting module is implemented based on a large model. The large model has mastered a large amount of general language knowledge in the pre-training stage and has strong semantic understanding capabilities. However, when facing certain specific tasks, there may be problems such as data bias and task complexity, which makes the model unable to adapt well to the needs of specific tasks. The multi-round rewriting module mainly fine-tunes the large model, using more domain knowledge and more specific data to better adapt to the needs of multi-round rewriting in TV scenarios, further improve model performance, make the model more stable and reliable, and improve the accuracy and generalization ability of multi-round rewriting tasks. In addition, compared to training a new model from scratch, fine-tuning a large model can save a lot of time and computing resources. Since the pre-trained model has learned a general semantic representation, the fine-tuning process usually only needs to be performed on a relatively small data set compared to traditional models, which can achieve model migration and customization faster.

[0307] The input of the multi-round rewriting module is the conversation history information after pre-processing and admission judgment, as well as the current round of user input and scene information. The specific model training process is as follows: Fig.11 shown.

[0308] During the training of the large model, since the multi-round interaction logic based on scene information is very complex and there is no online log data for reference, this application designs a step-by-step data construction method. By constructing appropriate prompts, the most advanced large model GPT4 is used to generate and annotate multi-round dialogue data. In order to make it more in line with the conversation habits of real users in TV scenes, a small amount of high-quality user dialogue data in this scene can also be used as seed data for data expansion.

[0309] Specifically, first, we designed the dialogue scenario seed data based on the existing TV interaction scenarios, and generated more dialogue scenarios and dialogue data through GPT4; then, we simulated the complete link of the interaction process under different services, including user query requests, terminal information reporting, dialogue system understanding, whole machine command execution, dialogue system feedback, terminal information display, user manual operation and other interaction logic, and manually annotated the sample data; finally, we used the big model to annotate the remaining dialogue data in a few-shot manner. Finally, we organized it into a command training set for fine-tuning the big model.

[0310] The instruction construction module adds contextual conversation content and terminal scenario information during instruction construction, and the prompt word format adopts the more readable markdown format, so that the model can understand the user's current context more comprehensively and improve the accuracy of understanding the user's query.

[0311] The training data of the large model used by the model rewriting module includes training samples that do not need to be rewritten, multi-round rewritten training samples based on interface focus, and multi-round rewritten training samples based on interface information understanding. Among them,

[0312] For example, taking the training samples that do not need to be rewritten as an example, specifically including:

[0313] Multiple rounds of session history between the user and the TV assistant: such as I want to watch a movie (session_id).

[0314] If the user's last-round expression is incomplete, complete the user's last-round expression in combination with the session history to make it complete, and try to complete it using the given information.

[0315] Rules to be followed for the task: Give priority to the information in the round with a larger session_id. If the specific information of the person or media asset to be referred to cannot be determined, no reference will be made, and it is necessary to ensure that no ambiguous reference is made.

[0316] The information in the user or the system in the same session_id has the highest priority. If there are pronouns, only consider the focus information, the information of the media asset being broadcast, the information in the user or the system. If there are ordinal numbers, such as \″the. th\″, the ordinal number needs to be replaced with the media asset or character information corresponding to the serial number according to the semantics. If the ordinal number exceeds the maximum value of the given serial number, no reference will be made. If there is no relevant information in the session history that can be completed, no completion and reference will be made.

[0317] Output format: If the complete expression of the user's last round is the same as the original expression, output 0; if the complete expression of the user's last round is different from the original expression, according to whether it contains the media asset and character information shown on the TV interface, the output format is as follows: the complete expression of the user's last round#media asset id#character id. Note that if the media asset id or character id is not included, set the corresponding position to null, and at most two nulls can be included in the output content.

[0318] Final output: 0, indicating that the original interaction content is not rewritten. Using 0 to represent the result of not rewriting can optimize the time consumption and improve the processing efficiency in the subsequent processing process.

[0319] Another example, taking the multi-round rewritten training samples based on interface focus as an example, specifically including:

[0320] The session history between the user and the TV assistant is as follows: I want to watch a movie (session_id). The display interface shows: \"System\": \"Found movies such as "Movie Name X"\"}, {\"Isession_id\": \"2\", \"User\": \"Who is the director of this movie\", \"Media Resource Result\":

[0321] [{\″serial number\″: 1, \″name\″: \″movie name X\″, \″id\″: \″11065437141\″},

[0322] {\″serial number\″: 2, \″name\″: \″movie name A\″, \″id\″: \″11064262260\″},

[0323] {\″serial number\″: 3, \″name\″: \″movie name B\″, \″id\″: \″1054387\″},

[0324] {\″serial number\″: 4, \″name\″: \″movie name C\″, \″id\″: \″11063427704\″},

[0325] {\″serial number\″: 5, \″name\″: \″movie name D\″, \″Iid\″: \″11011923151\″},

[0326] {\″serial number\″: 6, \″name\″: \″movie name E\″, \″id\″: \″11011923177\″},

[0327] {\″serial number\″: 7, \″name\″: \″movie name F\″, \″id\″: \″11011923338\″},

[0328] {\″serial number\″: 8, \″name\″: \movie name G\″, \″id\″: \″11011923567\″}],

[0329] \"Interface focus\":{\"mediaTitle\":\"Movie name X\",\"mediaID\":\"11065437141\"}}]

[0330] Task content: If the user's last expression is incomplete, complete the user's last expression based on the session history to make it complete, and try to use the given information to complete it. :

[0331] Rules to follow for the task: Give priority to information in rounds with a larger session_id. If the specific information about the person or media asset being referred to cannot be determined, do not make a reference. Make sure not to make ambiguous references. In the same session_id, information in the user or system has the highest priority.

[0332] If there are pronouns, only consider focal information, information about the media asset being played, information in the user or system. If there are ordinal numbers, such as "the. th", replace the ordinal number with the media asset or person information corresponding to the serial number according to the semantics. If the ordinal number exceeds the maximum value of the given serial numbers, do not make a reference.

[0333] If there is no relevant information in the session history that can be filled in, do not fill in and make a reference.

[0334] Output format: If the complete expression of the user's last round is the same as the original expression, output: 0; if the complete expression of the user's last round is different from the original expression, according to whether it contains the media asset and person information shown on the TV interface, the output format is as follows: The complete expression of the user's last round#media asset id#person id.

[0335] Finally, output the rewritten content: Who is the director of movie X#11065437141#null

[0336] Again, taking the training samples of multi-round rewriting based on the interface focus as an example, specifically including:

[0337] Multiple rounds of session history between the user and the TV assistant

[0338] "User": I want to watch a movie (session_id: 1).

[0339] "System": Found movies such as "Movie Name X"

[0340] "User": Who is the director of this movie (session_id: 2)

[0341] "System": "The director of Movie Name X is Peng Mouquan", "Media Asset Result":

[0342] [{"serial number": 1, "name": "Movie Name X", "id": "11065437141"},

[0343] {"serial number": 2, "name": "Movie Name A", "id": "11064262260"},

[0344] {\″serial number\″: 3, \″name\″: \″movie name B\″, \″id\″: \″1054387\″},

[0345] {\″serial number\″: 4, \″name\″: \″movie name C\″, \″id\″: \″11063427704\″},

[0346] {\″serial number\″: 5, \″name\″: \″movie name D\″, \″id\″: \″11011923151\″},

[0347] {\″serial number\″: 6, \″name\″: \″movie name E\″, \″id\″: \″11011923177\″},

[0348] {\″serial number\″: 7, \″name\″: \″movie name F\″, \″id\″: \″11011923338\″},

[0349] {\″serial number\″: 8, \″name\″: \″movie name G\″, \″id\″: \″11011923567\″}], \″interface focus\″: {\″mediaTitle\″: \″movie name X\″, \″mediaID\″: \″11065437141\″}}, \″user\″: \″Who is the fourth male lead (session_id: 3)\″}

[0350] Task content: If the user's last expression is incomplete, complete the user's last expression based on the session history to make it complete, and try to use the given information to complete it.

[0351] Rules to be followed in the task: information in rounds with larger session_id is given priority. If the specific information of the person or media asset being referred to cannot be determined, no reference is made. Make sure that no ambiguous reference is made. Information in the user or system with the same session_id has the highest priority. If there is a pronoun, only the focus information, the media asset information being broadcast, and the user or system information are considered. If there is an ordinal number, such as \"the.th\", the ordinal number must be replaced with the media asset or person information corresponding to the ordinal number based on semantics. If the ordinal number exceeds the maximum value of the given ordinal number, no reference is made.

[0352] If there is no relevant information to be completed in the session history, no completion or reference will be performed.

[0353] Output format: If the user's last complete expression is consistent with his original expression, output: 0; if the user's last complete expression is inconsistent with his original expression, the output format is as follows, depending on whether it contains the media and character information displayed on the TV interface: user's last complete expression#media id#character id

[0354] The final output rewrite content: "Who is the male lead of the movie name C#11063427704#null"

[0355] The pass judgment module is used to perform semantic detection on the rewritten content, and its purpose is to determine whether the rewritten content has clear semantics and whether it can be accurately understood and processed by the system. Semantic detection can adopt a variety of methods, such as semantic analysis algorithms and machine learning models in natural language processing technology. If the semantics of the rewritten content are clear and unambiguous, and can be correctly parsed and processed by the system, the semantic detection result is passed; if the rewritten content is semantically ambiguous, incomplete, or cannot be understood by the system, the semantic detection result is failed.

[0356] In one of the specific examples, an example of a conversation session is given:

[0357]

[0358] In this embodiment, multi-round conversation history information and real-time scene information on the device terminal (such as movie search results, TV series being played by the player, etc.) are innovatively combined to more comprehensively understand the user context and intention, and realize cross-business multi-round understanding based on scene information. Through fine-tuning, the sentence rewriting ability of the open source large model is fully stimulated, aiming to improve the multi-round semantic understanding ability of the conversation interaction system in voice interaction scenarios, and provide users with a more natural and smooth interaction experience.

[0359] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method described in the above embodiment is implemented.

[0360] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the above embodiment is implemented.

[0361] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method described in the above embodiment is implemented.

[0362] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0363] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0364] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0365] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A display device, characterized in that: The display device comprises: A display, configured to display at least one display item, wherein the display item is current display content or at least one historical response result generated based on historical interaction content; An input module, configured to receive first interactive content input by a user; the first interactive content at least includes an ordinal number or a pronoun; The controller is configured to respond to a control instruction corresponding to the first interactive content, take the display item referred to by the ordinal word or the pronoun as an interactive target, and display the first response content matching the interactive target on the display.

2. The display device according to claim 1, characterized in that The controller is also configured to: In the case that there are multiple display items and the first interactive content includes ordinal numbers, display items whose display order is the same as the order represented by the ordinal numbers are taken as interactive targets.

3. The display device according to claim 2, characterized in that The controller is also configured to: In the case where there are multiple display items, one of the display items is displayed as the interface focus; the display item where the interface focus is located is switched following the change of the control signal; In the case where the first interactive content includes ordinal numbers, and the order represented by the ordinal numbers is the same as the display order of the display item where the interface focus is located, the display item where the interface focus is located is used as the interactive target.

4. The display device according to claim 2, characterized in that The controller is also configured to: In the case where there are multiple display items, one of the display items is displayed as the interface focus; the display item where the interface focus is located is switched following the change of the control signal; When the first interactive content includes an ordinal number, and the order represented by the ordinal number is different from the display order of the display item where the interface focus is located, the display item with the same display order as the order represented by the ordinal number is used as the interaction target.

5. The display device according to claim 2, characterized in that The controller is also configured to: During multiple rounds of interaction, controlling the display to display display items for each round of interaction; In the case where the first interaction content includes an ordinal number, the interaction round to which the display item where the interface focus is located belongs is taken as the target interaction round; The display items in the target interaction round whose display order is the same as the order represented by the ordinal number are taken as interaction targets.

6. The display device according to claim 1, characterized in that The controller is also configured to: When the number of the display items is one and the first interactive content includes a pronoun, the display item is used as an interactive target.

7. The display device according to claim 1, characterized in that The controller is also configured to: In the case where there are multiple display items, one of the display items is displayed as the interface focus; the display item where the interface focus is located is switched following the change of the control signal; In the case where the first interactive content includes a pronoun, the display item where the interface focus is located is used as an interactive target.

8. The display device according to claim 1, characterized in that In the process of displaying at least one historical response result generated based on the historical interaction content on the display, the controller is further configured to: On the display, M historical response results generated based on the historical interaction content are arranged and displayed along a first direction, wherein M≤N, M and N are positive integers, and N is the maximum number of display items that can be accommodated on the display along the first direction.

9. The display device according to claim 1, characterized in that: The currently displayed content includes the system home page or the played media data; the controller is further configured to: Keep displaying the system main page or keep playing the media asset data, and display the first response content in the first display area of ​​the display.

10. The display device according to claim 9, characterized in that The controller is also configured to: Keep displaying the system main page, and display the first response content on the upper layer of the system main page in a floating layer manner; the first response content includes at least one media asset data; In response to a play instruction for one of the media asset data, canceling display of the system main page and the first response content, and displaying a play page of the media asset data on the display; During the playback of the media asset data, the input module is further configured to receive other interactive content input by the user.

11. The display device according to claim 9, characterized in that The input module is further configured as: receiving second interactive content input by a user, where the second interactive content is interactive content input after the display displays the first response content; The controller is also configured to: The first response content is moved along a second direction in the first display area, and second response content for the second interactive content is displayed at an initial display position of the first response content.

12. The display device according to claim 9, characterized in that At least one of the first interactive content and the response content is voice content; the media resource data is audio data or audio and video data; and the controller is further configured to: During voice input or voice output, reducing the playback volume of the audio data or the audio and video data; In response to stopping voice input or stopping voice output, the playback volume of the audio data or the audio and video data is restored to a default state.

13. A display device, characterized in that: The display device comprises: A display, configured to display at least one display item, wherein the display item is current display content or at least one historical response result generated based on historical first interaction content; An input module, configured to receive first interactive content input by a user; the first interactive content at least includes an ordinal number or a pronoun, and a keyword belonging to a target category; The controller is configured as: In response to a control instruction corresponding to the first interactive content, the content referred to by the ordinal word or the pronoun in the display item is taken as an interactive target, and a first response content provided by a target application of the target category is displayed on the display, wherein the first response content is a response result to the first interactive content and the interactive target.

14. The display device according to claim 13, characterized in that In the process of displaying the first response content provided by the target application of the target category on the display, the controller is further configured to: Invoking a target application of the target category, and displaying an application interface of the target application on the display; In the application interface, first response content for the first interactive content and the interactive target is displayed.

15. The display device according to claim 13, characterized in that In the process of displaying the first response content provided by the target application of the target category on the display, the controller is further configured to: On the display, a first response content provided by the target application of the target category is displayed in a floating layer manner.

16. The display device according to claim 15, characterized in that In the process of displaying the first response content provided by the target application of the target category in a floating layer manner, the controller is further configured to: Displaying the first response content provided by the target application of the target category in a card style; Alternatively, the first response content provided by the target application of the target category is displayed in a pop-up window style.

17. The display device according to claim 15, characterized in that The first response content includes a plurality of candidate items; and the controller is further configured to: In response to a trigger operation on one of the candidate items, displaying an application interface of the target application; In the application interface, detailed content of the triggered candidate items is displayed.

18. The display device according to claim 14 or 17, characterized in that: The controller is also configured to: When the display displays the application interface of the target application, in response to the third interactive content input by the user, a third response result for the third interactive content is displayed.

19. The display device according to claim 18, characterized in that In the process of executing the third interactive content in response to the user input and displaying the third response result for the third interactive content, the controller is further configured to: In response to the third interactive content input by the user, the target application is exited, and a third response result for the third interactive content is displayed.

20. The display device according to claim 13, characterized in that The controller is also configured to: When receiving interactive content input by the user, start the interactive mode; During multiple rounds of interaction, maintaining the interaction mode in an on state; When it is detected that the interaction end condition is met, the interaction mode is exited.

21. The display device according to claim 20, characterized in that In the process of exiting the interactive mode when it is detected that the interaction end condition is met, the controller is further configured to: When the interactive mode is in the on state, the interactive mode is exited in response to the received exit control signal.

22. The display device according to claim 20, characterized in that In the process of exiting the interactive mode when it is detected that the interaction end condition is met, the controller is further configured to: When the interactive mode is in the on state and the duration of not receiving interactive content reaches a preset duration, the interactive mode is exited.

23. The display device according to claim 20, characterized in that In the process of exiting the interactive mode when it is detected that the interaction end condition is met, the controller is further configured to: When the interactive mode is in the on state and the keywords contained in the received interactive content hit the exit instruction, the interactive mode is exited.

24. The display device according to claim 20, characterized in that In the process of exiting the interactive mode when it is detected that the interaction end condition is met, the controller is further configured to: When the interaction mode is in the on state and the number of interaction rounds reaches a preset threshold, the interaction mode is exited.

25. An interactive method, characterized in that: The method comprises: Acquire target data reported by the display device when receiving interactive content, the target data including the interactive content and interface display information when the interactive content is received, the interface display information including at least one display item, the display item being current display content or at least one historical response result generated based on historical interactive content; In a case where the interactive content includes an ordinal number or a pronoun, rewriting the interactive content based on a display item in the interface display information to obtain a rewritten content; Generate response content for the rewritten content, and feed the response content back to the display device for display.

26. The method according to claim 25, characterized in that In the case where the interactive content includes an ordinal number or a pronoun, rewriting the interactive content based on the display item in the interface display information to obtain the rewritten content includes: In a case where the interactive content includes an ordinal word or a pronoun and there is an interface focus in the interface display information, determining a target display item referred to by the ordinal word or the pronoun based on the display items in the interface display information and the interface focus; The interactive content is rewritten according to the display content of the target display item to obtain rewritten content.

27. The method according to claim 26, characterized in that The method further comprises: Performing semantic detection on the rewritten content to obtain a semantic detection result; The generating of response content for the rewritten content includes: When the semantic detection result is detection passed, response content is generated based on the rewritten content.

28. An interactive method, characterized in that: The method comprises: Acquire target data reported by the display device when receiving interactive content, the target data including the interactive content and interface display information when the interactive content is received, the interface display information including at least one display item, the display item being current display content or at least one historical response result generated based on historical interactive content; In a case where the interactive content includes an ordinal number or a pronoun, and a keyword belonging to a target category, rewriting the interactive content based on a display item in the interface display information to obtain a rewritten content; A target application of the target category is called, response content for the rewritten content is obtained based on the target application, and the response content is fed back to the display device for display.

Images