Display device and answer information display method

Abstract

Some embodiments of the invention provide a display device and an answer information display method, first answer information is retrieved through a target application name and question key information, and second answer information is retrieved according to a vectorized target text vector, so that summarization is executed according to the first answer information and the second answer information, and a target answer text is output. And displaying on a corresponding application interface. According to the answer information display method, the accuracy of the answer information is improved through multiple answer information query modes, the target answer text is displayed on the application interface, so that the application interface and the target answer information are displayed on the same interface at the same time, a user does not need to repeatedly switch the interface to view the answer information, and the display process of the answer information is simplified.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a method for displaying answer information. Background Art

[0002] The display device can run a variety of applications and display the application interface corresponding to the application. When the user needs to query based on the elements in the application interface, he needs to temporarily exit the application interface and switch to the search interface to query the information. Taking the game program as an example, the user can use the display device as the game running carrier and display the game screen through the large screen of the display device. During the game, if the user cannot pass the level or complete the task in the game, he needs to switch the game interface to the search interface to query the relevant game strategy, so as to pass the level in the game or complete the task in the game.

[0003] However, when using the same display device to query game strategies, users need to check the queried information against the elements in the application interface, which causes users to switch interfaces repeatedly. For example, when a user queries a game strategy, they need to temporarily exit the game interface, and after consulting the game strategy, return to the game interface and operate the game according to the game strategy. If the game strategy contains too much content and the display device cannot display the application interface and the game strategy on the same screen, it is necessary to repeatedly switch interfaces to consult the game strategy, which makes it cumbersome to view the answer information. Summary of the invention

[0004] The present application provides a display device and an answer information display method to solve the problem that users need to repeatedly switch interfaces when searching for information.

[0005] In a first aspect, some embodiments of the present application provide a display device, including a display, a memory, a sound collector, and a controller, wherein the display is configured to display an application interface, the memory is configured to store an information query model, and the information query model includes an information retrieval module and an answer generation module; the sound collector is configured to collect audio data; and the controller is configured to:

[0006] In response to the voice query instruction, obtaining a query question voice, and performing voice recognition on the query question voice to obtain a text to be queried;

[0007] Inputting the text to be queried into the information retrieval module, parsing the text to be queried by the information retrieval module to obtain the target application name and key information of the problem, and performing vectorization on the text to be queried by the information retrieval module to obtain a target text vector;

[0008] According to the target application name and the key information of the question, first answer information is retrieved from a first information base, and second answer information is retrieved from a second information base according to the target text vector, wherein the first information base is an information base containing application information, and the second information base is an information base generated according to a slice vector, and the slice vector is obtained by performing vectorization based on the slice text of the application information;

[0009] Inputting the first answer information and the second answer information into the answer generation module to generate a target answer text;

[0010] Generate text prompt information according to the target answer text, and control the display to display the text prompt information on the application interface.

[0011] In some embodiments, the controller performs the step of retrieving the first answer information from the first information repository, and is configured to:

[0012] Retrieving an answer information set in the first information database according to the target application name, the answer information set including application information of the target application;

[0013] Calculating a matching value between the application information in the answer information set and the key information of the question;

[0014] The first answer information is determined according to the matching value, where the first answer information is the application information with the highest matching value in the answer information set.

[0015] In some embodiments, the controller performs calculation of a matching value between the application information in the answer information set and the key information of the question, and is configured to:

[0016] Analyze the keywords in the key information of the problem;

[0017] Obtaining the word frequency of the keyword in the application information;

[0018] The matching value is generated according to the term frequency.

[0019] In some embodiments, the controller performs the step of retrieving second answer information from a second information repository according to the target text vector, and is configured to:

[0020] Calculating the vector similarity between the target text vector and the slice vector;

[0021] Marking the slice vectors whose vector similarity is greater than or equal to the similarity threshold as preliminary slice vectors;

[0022] Calculating answer scores for the prepared slice vector and the target text vector;

[0023] The second answer information is determined according to the answer score, where the second answer information is application information corresponding to the prepared slice vector having the highest answer score among the prepared slice vectors.

[0024] In some embodiments, the controller determines the second answer information according to the answer score, and is configured to:

[0025] Obtaining a metadata index of the prepared slice vector;

[0026] Acquire an answer information segment according to the metadata index, wherein the answer information segment and the slice information corresponding to the prepared slice vector are derived from the same application information;

[0027] The second answer information is generated according to the answer information segment and the slice information corresponding to the prepared slice vector.

[0028] In some embodiments, before the controller executes the step of retrieving the second answer information from the second information repository according to the target text vector, the controller is further configured to:

[0029] Acquire application information in the first information database;

[0030] Slicing the application information to obtain slice information, and vectorizing the slice information to obtain a slice vector;

[0031] The second information base is constructed according to the slice vector.

[0032] In some embodiments, the controller executes inputting the first answer information and the second answer information into the answer generation module, and is configured to:

[0033] Extracting a prompt text segment according to the first answer information and the second answer information;

[0034] The first answer information and the second answer information are combined to generate a target answer text according to the prompt text segment.

[0035] In some embodiments, an audio output device is further included, and the controller generates text prompt information according to the target answer text, and is configured to:

[0036] Converting the target answer text into a voice format to obtain a target answer audio;

[0037] In the process of controlling the display to display the text prompt information, the audio output device is controlled to play the target answer audio.

[0038] In some embodiments, the controller performs the step of inputting the text to be queried into the information retrieval module, and is further configured to:

[0039] If the target application name and key problem information are not obtained through parsing, the currently displayed application interface image is obtained;

[0040] Performing feature detection on the application interface image to obtain feature element information;

[0041] The target application name and key information of the problem are identified according to the characteristic element information.

[0042] In a second aspect, some embodiments of the present application provide a method for displaying answer information, which is applied to the display device described in the first aspect, and the method includes:

[0043] In response to the voice query instruction, obtaining a query question voice, and performing voice recognition on the query question voice to obtain a text to be queried;

[0044] Inputting the text to be queried into the information retrieval module, parsing the text to be queried by the information retrieval module to obtain the target application name and key information of the problem, and performing vectorization on the text to be queried by the information retrieval module to obtain a target text vector;

[0045] According to the target application name and the key information of the question, first answer information is retrieved from a first information base, and second answer information is retrieved from a second information base according to the target text vector, wherein the first information base is an information base containing application information, and the second information base is an information base generated according to a slice vector, and the slice vector is obtained by performing vectorization based on the slice text of the application information;

[0046] Inputting the first answer information and the second answer information into the answer generation module to generate a target answer text;

[0047] Generate text prompt information according to the target answer text, and control the display to display the text prompt information on the application interface.

[0048] As can be seen from the above technical solutions, some embodiments of the present application provide a display device and an answer information display method, wherein the method obtains a query question voice by responding to a voice query instruction input by a user, and performs voice recognition on the query question voice to obtain a text to be queried. The text to be queried is input into an information retrieval module, so that the text to be queried is parsed by the information retrieval module to obtain a target application name and key information of the question, and vectorization is performed on the query text to obtain a target text vector. The first answer information is retrieved in a first information library according to the target application name and key information of the question, and the second answer information is retrieved in a second information library according to the target text vector. The first answer information and the second answer information are input into an answer generation module to generate a target answer text and display it. The answer information display method performs voice recognition on the query question voice, queries the corresponding answer information according to the target application name and key information of the question obtained by recognition, and performs a slice text search according to the vectorized target text vector to obtain another answer information, thereby summarizing the two types of answer information obtained to output the final answer text and display it, so that the display device can display the application interface and the target answer information on the same interface at the same time, and the user does not have to repeatedly switch different interfaces to view the answer information, simplifying the display process of the answer information. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0051] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0052] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0053] Figure 4 This is a schematic diagram of switching interfaces when a user plays a game in some embodiments of the present application;

[0054] Figure 5 A flowchart of a display device performing answer information display according to some embodiments of the present application;

[0055] Figure 6A timing diagram of a display device performing answer information display according to some embodiments of the present application;

[0056] Figure 7 A flowchart of a display device retrieving first answer information provided by some embodiments of the present application;

[0057] Figure 8 A flow chart of calculating a matching value for a display device provided in some embodiments of the present application;

[0058] Fig. 9 A flowchart of a display device retrieving second answer information provided by some embodiments of the present application;

[0059] Fig.10 A flowchart of a display device obtaining second answer information according to a metadata index provided in some embodiments of the present application;

[0060] Fig.11 A flow chart of a display device obtaining information to be retrieved according to an application interface image provided in some embodiments of the present application;

[0061] Fig.12 A flowchart of generating target answer audio for a display device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0062] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0063] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0064] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0065] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0066] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0067] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0068] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0069] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0070] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0071] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0072] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0073] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0074] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0075] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0076] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0077] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0078] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0079] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0080] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0081] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0082] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0083] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0084] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0085] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0086] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0087] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0088] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0089] like Figure 3 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0090] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0091] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0092] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0093] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0094] In some embodiments, the display device 200 can respond to the power-on instruction, run the power-on program, and control the display 260 to display a user interface, and the user interface may include multiple application controls to display the application programs that the current display device 200 can run. For example, the user interface may include video playback controls, game controls, browser controls, camera controls, or search engine controls. The user can input a startup instruction based on the application controls in the user interface to control the display device 200 to run the corresponding application controls. And display the application interface of the application control specified by the startup instruction. For example, when the user inputs a startup instruction based on the video playback control, the display device 200 can respond to the startup instruction to display the video playback interface. For another example, when the user inputs a startup instruction based on the game control, the display device 200 can respond to the startup instruction to display the game interface.

[0095] When a user browses or watches an application interface displayed by the display device 200, the user may need to query information based on the content displayed in the application interface. Taking a game program as an example, a user can play the game through the display device 200. Figure 4 A diagram showing how a user switches to a different interface to view a game guide when playing a game on a display device. Figure 4, at this time, the display 260 displays the game interface of "Hot-Blooded Basketball". During the game, if the user cannot pass the level or complete the task in the game, the game progress will be affected. At this time, the user needs to query the relevant game strategy based on the game content displayed by the display device 200 in order to pass the level in the game or complete the task in the game.

[0096] However, when using the same display device 200 to search for game strategies, the user needs to check the information obtained by the search by comparing the elements in the application interface, which causes the user to switch interfaces repeatedly. For example, the user can search for game strategies while running the game program through the display device 200. This causes the user to temporarily exit the game interface and create a search program in the user interface. The search program can generate the following Figure 4 , thereby searching for game strategies. After consulting the game strategies, the user can control the display device 200 to return to the game interface and operate the game according to the game strategies. However, if the game strategies contain a lot of content, the user cannot remember all the game strategies at once, and needs to repeatedly switch between the game interface and the search interface, which affects the flow of the game process.

[0097] Some embodiments of the present application provide a display device 200 to solve the problem that a user needs to repeatedly switch interfaces when performing information query based on the display content of an application interface through the display device 200. The display device 200 includes a display 260, a memory, a sound collector and a controller 260. Among them, the display 260 is configured to display the application interface. The memory is configured to store an information query model. The information query model is used to query target answer information based on the text to be queried. The information query model includes an information retrieval module and an answer generation module. The sound collector is configured to collect audio data, and the controller 250 is configured to execute a method for displaying answer information. Figure 5 A flowchart of a display device performing question and answer information display provided in some embodiments of the present application. Figure 6 A timing flow chart of the display device according to the embodiment of the present application performing question and answer information display. Figure 6 The timing relationship shown is shown in the Q&A information display. Figure 5 and Figure 6 , the method comprises the following contents:

[0098] S100: In response to a voice query instruction, a query question voice is acquired, and voice recognition is performed on the query question voice to obtain a text to be queried.

[0099] The user can input a startup instruction to the display device 200 based on the user interface, and the display device 200 can respond to the startup instruction, run the application specified by the startup instruction, and control the display 260 to display the application interface. Taking a game program as an example, the user can input a startup instruction of the game program to control the display device 200 to start the game program, and at this time, the display 260 displays the game interface.

[0100] When the user is playing the game, when the user needs to perform information query based on the game interface, the user can input a voice query command to the display device 200. At this time, the display device 200 can respond to the voice query command and obtain the query question voice. When the display device 200 is in the screen-off standby state, the user can speak the query question voice after inputting the voice query command, so as to wake up the display device 200 through the voice query command, and then obtain the query question voice. For example, the user can say: "Hello TV, please check the clearance method of the third level of "Hot Blood Basketball"". Among them, "Hello TV" is a voice query command, "Please check the clearance method of the third level of "Hot Blood Basketball"" is the query question voice, and "Hot Blood Basketball" is the name of a game application.

[0101] In some embodiments, if the display device 200 is in working state, the display device 200 can directly use the voice content corresponding to the voice query instruction as the query question voice. For example, when the display device 200 is in working state, the user can say "Hello TV, please check the clearance method of the third level of 'Hot Blood Basketball'" as the voice query instruction, and the display device 200 can respond to the voice query instruction and use "Hello TV, please check the clearance method of the third level of 'Hot Blood Basketball'" as the query question voice.

[0102] After the display device 200 obtains the query voice, in order to facilitate the query of the corresponding answer information, the controller 250 needs to perform voice recognition on the query voice, thereby converting the query voice into a text format to obtain the text to be queried. During the voice recognition process, the controller 250 can perform noise reduction processing on the query voice to reduce noise interference in the query voice and improve the accuracy of voice recognition.

[0103] After obtaining the query voice after noise reduction, the controller 250 can input the query voice after noise reduction into the speech recognition model to obtain the text to be queried. The speech recognition model can be pre-stored in the memory, and when performing speech recognition on the query voice, the controller 250 can call the speech recognition model from the memory to obtain the text to be queried.

[0104] S200: Inputting the text to be queried into the information retrieval module, parsing the text to be queried through the information retrieval module to obtain the target application name and key information of the problem, and performing vectorization on the text to be queried through the information retrieval module to obtain a target text vector.

[0105] After obtaining the text to be queried, the controller 250 can input the text to be queried into an information query model. The information query model can be a large language model (LLMs). The large language model can understand and retrieve natural language text by learning a large amount of text data, thereby improving the efficiency of performing retrieval on the query text.

[0106] The information query model includes an information retrieval module and an answer generation module. When the text to be queried is input into the information query model, the information query model can input the text to be queried into the information retrieval module, so that the text to be queried can be parsed by the information retrieval module to obtain the target application name and key information of the question. Taking the text to be queried as "Please query the method to pass the third level of 'Hot-Blooded Basketball'" as an example, the information retrieval module can first parse the application name to parse out that the target application name in the text to be queried is "Hot-Blooded Basketball". Based on this, the information retrieval module then performs key information parsing on other text contents of the text to be queried, that is, the key information of the question is "the method to pass the third level", and performs answer information retrieval according to the target application name and the key information of the question.

[0107] However, the target application name and the key information of the question are respectively information fragments extracted from the text to be queried. When performing answer information retrieval, the answer information may be inaccurate due to information incoherence. In order to alleviate the above problem, the information retrieval module can perform vector retrieval synchronously while performing answer information retrieval through the target application name and the key information of the question, so as to perform information retrieval through the text to be queried. To this end, the information retrieval module can perform a backup of the text to be queried, and perform vectorization on the backed-up text to be queried to obtain the target text vector, so as to perform the subsequent vector retrieval process.

[0108] S300: Retrieving first answer information from a first information base according to the target application name and the key information of the question, and retrieving second answer information from a second information base according to the target text vector.

[0109] After obtaining the target application name and the key information of the question, the controller 250 can retrieve the first answer information in the first information library according to the target application name and the key information of the question, wherein the first information library is an information library containing application information, and the controller 250 can determine the first information library according to the application category of the target application name. For example, if the target application name is a game application name, the first information library can be a game information library, and the game information library can contain game information, that is, game-related application information, such as game strategies, game introduction information, or game operation information. If the target application name is a video playback application name, the first information library can include media asset information, that is, media asset introduction information, cast and crew information, or related media asset information related to the media asset data.

[0110] like Figure 7 As shown, in the process of retrieving the first answer information in the first information library, the controller 250 can perform a preliminary search in the first information library according to the target application name to obtain a set of answer information related to the target application name, wherein the answer information set can include application information of the target application, which is used to filter out application information of other applications, narrow the search scope, and improve the retrieval efficiency and accuracy of the answer information. After obtaining the answer information set, the controller 250 can perform another search in the application information in the answer information set according to the key information of the question. In order to improve the accuracy of the retrieval, the controller 250 can calculate the matching value between the application information in the answer information set and the key information of the question, so as to sort the application information in the answer information set according to the matching value, thereby determining the first answer information according to the matching value, and the first answer information is the application information with the highest matching value in the answer information set.

[0111] Exemplarily, following the embodiment in step S200, when the target application name is the game application name "Hot Blood Basketball", the first information library is the game information library, and the controller 250 can perform a preliminary search based on "Hot Blood Basketball" through the information retrieval module to obtain an answer information set. At this time, the answer information set includes game information related to "Hot Blood Basketball", such as the game strategy, game introduction information or game operation information of "Hot Blood Basketball".

[0112] After obtaining the answer information set, the information retrieval module can calculate the matching value of the game information in the answer information set according to the key information of the question "how to pass the third level", thereby determining that the game information with the highest matching value with the "how to pass the third level" is the first answer information.

[0113] In some embodiments, Figure 8As shown, the controller 250 can parse the keywords in the key information of the question, for example, "third level" and "passing method" can be used as keywords in "the third level passing method". And according to the keywords, the word frequency of the keywords in the game information is obtained in the answer information set. The game information can be a piece of text information, and the keywords in the text information can appear multiple times. The controller 250 can generate a matching value according to the word frequency of the keywords.

[0114] For example, when the answer information set includes three game information, namely the first game information "how to pass the first level", the second game information "how to pass the second level" and the third game information "how to pass the third level", wherein the first game information, the second game information and the third game information all contain the keyword "how to pass the level", at this time, the game information can be judged based on another keyword. Since the third game information includes the keyword "third level", the word frequency of the keyword in the third game information is greater than that of the first game information and the second game information. Therefore, the matching value of the third game information is greater than the matching value of the first game information and the matching value of the second game information, and the controller 250 can determine the third game information as the first answer information.

[0115] While retrieving the first answer information, the controller 250 may retrieve the second answer information from the second information base according to the target text vector, the second information base being an information base generated according to the slice vector, and the slice vector being vectorized according to the slice text of the application information. Before retrieving the second answer information, the controller 250 may slice the application information to divide the application information into a plurality of slice texts.

[0116] In some embodiments, Fig. 9 As shown, the controller 250 calculates the vector similarity between the target text vector and the slice vector through the information retrieval module, and the target text vector is a vector representation of the complete text to be queried. The information retrieval module can retrieve multiple slice vectors as preliminary slice vectors from the second information library based on the vector similarity. In order to accurately screen out the preliminary slice vectors, the controller 250 can set a similarity threshold, and the information retrieval module can mark the slice vectors whose vector similarity is greater than or equal to the similarity threshold as preliminary slice vectors.

[0117] In some embodiments, the information retrieval module may further include a semantic understanding layer for obtaining semantic information of the preliminary slice vector. After obtaining the preliminary slice vector, the controller 250 may calculate the answer score of the preliminary slice vector and the target text vector. To this end, the controller 250 may input the preliminary slice vector and the target text vector into the semantic understanding layer to perform semantic understanding on the preliminary slice vector, output the semantic understanding result of the preliminary slice vector, and calculate the question-answer matching degree between the semantic understanding result and the target text vector to output the answer score. The controller 250 may determine the second answer information based on the answer score, and the second answer information is the application information corresponding to the preliminary slice vector with the highest answer score in the preliminary slice vector.

[0118] S400: Inputting the first answer information and the second answer information into the answer generation module to generate a target answer text.

[0119] After the information retrieval module outputs the first answer information and the second answer information, the information query model can input the first answer information and the second answer information into the answer generation module. The information query model can summarize the first answer information and the second answer information through the text generation capability of the large model, and generate a coherent target answer text, thereby combining the fragment retrieval form of the target application name and key information of the question with the complete retrieval form of the target text vector to improve the accuracy of the target answer text.

[0120] In some embodiments, the controller 250 can extract a prompt text segment based on the first answer information and the second answer information, which can also be called a prompt text. Taking "How to defeat the tree guard in game A" as the query text, the first answer information can be "Use the fire element skill of the magic wizard to defeat the tree guard." The second answer information is "Use the fire element skill, pay attention to avoid the ultimate skill of the tree guard." The answer generation module can extract prompt words based on the first answer information and the second answer information, such as "magic wizard", "fire element skill" and "avoid ultimate skill", and generate a prompt text segment based on the prompt word. The prompt text segment is "Based on the prompt word, generate a game strategy to defeat the tree guard in game A, prompt words: 'magic wizard', 'fire element skill' and 'avoid ultimate skill'."

[0121] The answer generation module can summarize the first answer information and the second answer information according to the prompt text fragment, and generate a semantically coherent target answer text based on the prompt text fragment. For example, the target answer text can be: "In game A, you can defeat the tree guard through the fire element skill of the magical little wizard, and at the same time, you must pay attention to avoid the ultimate skill issued by the tree guard." By summarizing the first answer information and the second answer information according to the prompt text fragment, the problem of incoherence of the target answer information can be alleviated.

[0122] S500: Generate text prompt information according to the target answer text, and control the display to display the text prompt information on the application interface.

[0123] After the information query model outputs the target answer information, the controller 250 can generate text prompt information according to the target answer text. The display 260 is controlled to display the text prompt information on the application interface, so that the display device 200 can simultaneously display the application interface and the text prompt information in the same interface, alleviating the problem that the user needs to control the display device 200 to repeatedly switch interfaces to view the answer information, and improving the convenience of the display device 200 in displaying the answer information.

[0124] In some embodiments, the controller 250 can also plan the display area of ​​the text prompt information in the application interface. In order to reduce the obstruction of the application interface, the display area can be located at the edge of the application interface, such as the upper right corner, lower left corner, etc. of the application interface. The controller 250 can also set the transparency of the display area. When the display area is displayed without transparency, the display area obscures part of the application interface, causing the user to be unable to view the complete application interface. For this reason, the controller 250 can respond to the transparency adjustment instruction input by the user to adjust the transparency of the display area, so that the display device 200 can normally display the application interface originally obscured by the display area in the transparent display area, so that the user can view the target answer information while viewing the complete application interface.

[0125] In some embodiments, since the preliminary slice vector is a vector generated by the slice document of the application information and does not represent the complete application information, in the process of the information retrieval module retrieving the second answer information, the metadata index of the preliminary slice vector can be obtained, and the answer information fragment can be obtained through the metadata index, wherein the answer information fragment is the slice information corresponding to the preliminary slice vector and is derived from the same application information. For example, when constructing the second information library, the application information can be sliced ​​into the first slice information, the second slice information and the third slice information, and vectorization is performed on the first slice information, the second slice information and the third slice information respectively to obtain the first slice vector, the second slice vector and the third slice vector, and metadata indexes are created for the first slice vector, the second slice vector and the third slice vector respectively, so as to obtain the complete second answer information in the process of performing vector matching. Among them, the second slice vector and the third slice vector can be used as metadata of the first slice vector, and the first slice vector and the second slice vector can be used as metadata of the first slice vector.

[0126] Based on the above scenario, Fig.10 As shown, when the prepared slice vector is the second slice vector, in order to obtain complete second answer information, the first slice information and the third slice information, that is, the answer information fragment, can be obtained through the metadata index corresponding to the first slice vector and the third slice vector, and the second answer information is generated according to the answer information fragment and the slice information corresponding to the prepared slice vector, that is, the second answer information is generated through the first slice information, the second slice information and the third slice information.

[0127] In some embodiments, before retrieving the second answer information, the controller 250 may also construct a second information library. To this end, the controller 250 may obtain the application information in the first information library and slice the application information. During the slicing process, the controller 250 may slice the application information according to a preset number of words. For example, the application information of 1,000 words may be sliced ​​into paragraphs of 100 words to obtain 10 sliced ​​texts of 100 words each. The controller 250 may vectorize the sliced ​​texts of all application information and construct a second information library based on the obtained slice vectors.

[0128] In some embodiments, when the query question voice input by the user does not have a specific reference, the information retrieval module cannot parse the target application name and the key information of the question in the query text corresponding to the query question voice. For example, the query text is "this level is cleared in this way". At this time, the query text does not contain any application name and the corresponding key information, and the information retrieval module cannot parse the target application name and the key information of the question based on "this level is cleared in this way".

[0129] Based on this, Fig.11As shown, when the target application name and key information of the question are obtained by parsing the unparsed text to be queried, the controller 250 obtains the application interface image currently displayed by the display device 200, performs feature detection on the application interface image, obtains feature element information, and identifies the target application name and key information of the question according to the feature element information. For example, when the application interface image is a game interface image, the controller 250 can determine the game type according to the game elements in the game interface image, such as chess and card games, shooting games, sports games or role-playing games, and then perform feature detection on the game elements according to the game type. For example, when the game interface image contains sports game elements, it can be determined that the current running game type is a sports game. Then, according to feature element information such as "player", "court", "basketball" and "third level", the target application name is identified as "hot-blooded basketball" and the key information of the question is "third level". Therefore, when the user inputs a relatively vague query question voice, the target application name and key information of the question are determined according to the application interface image, and the answer information is retrieved according to the target application name and key information of the question.

[0130] In some embodiments, Fig.12 As shown, the display device 200 also includes an audio output device, which is used to play audio data. After the answer generation module generates the target answer text, the controller 250 can generate text prompt information based on the target answer text. At the same time, the controller 250 can also input the target answer text into the speech generation model to convert the target answer text into a speech format through the speech generation model to obtain the target answer audio. After generating the target answer audio, the controller 250 can control the audio output device to play the target answer audio in the process of controlling the display 260 to display the text prompt information.

[0131] The user can also set the playback information of the target answer text and the target answer audio. Before displaying the text prompt information, the controller 250 can display the playback configuration interface. The playback configuration interface can include the text prompt information and the configuration options of the target answer audio, such as language options, speaking speed options, loop display / playback, etc., so that the user can select the appropriate configuration options and enable the display device 200 to display the text prompt information according to the set configuration options, or play the target answer audio.

[0132] Some embodiments of the present application also provide an answer information display method, which is applied to a display device 200, wherein the display device 200 at least includes a display 260, a memory, a sound collector, and a controller 250, wherein the display 260 is configured to display an application interface, the memory is configured to store an information query model, and the information query model includes an information retrieval module and an answer generation module; the sound collector is configured to collect audio data, and the method includes:

[0133] S100: In response to a voice query instruction, obtaining a query question voice, and performing voice recognition on the query question voice to obtain a text to be queried;

[0134] S200: Inputting the text to be queried into the information retrieval module, parsing the text to be queried through the information retrieval module to obtain the target application name and key information of the problem, and performing vectorization on the text to be queried through the information retrieval module to obtain a target text vector.

[0135] S300: Retrieving first answer information from a first information base according to the target application name and the key information of the question, and retrieving second answer information from a second information base according to the target text vector.

[0136] The first information base is an information base containing application information, the second information base is an information base generated according to the slice vector, and the slice vector is obtained by performing vectorization based on the slice text of the application information.

[0137] S400: Inputting the first answer information and the second answer information into the answer generation module to generate a target answer text.

[0138] S500: Generate text prompt information according to the target answer text, and control the display to display the text prompt information on the application interface.

[0139] As can be seen from the above technical solutions, some embodiments of the present application provide a display device and an answer information display method, wherein the method obtains a query question voice by responding to a voice query instruction input by a user, and performs voice recognition on the query question voice to obtain a text to be queried. The text to be queried is input into an information retrieval module, so that the text to be queried is parsed by the information retrieval module to obtain a target application name and key information of the question, and vectorization is performed on the query text to obtain a target text vector. The first answer information is retrieved in a first information library according to the target application name and key information of the question, and the second answer information is retrieved in a second information library according to the target text vector. The first answer information and the second answer information are input into an answer generation module to generate a target answer text and display it. The answer information display method performs voice recognition on the query question voice, queries the corresponding answer information according to the target application name and key information of the question obtained by recognition, and performs a slice text search according to the vectorized target text vector to obtain another answer information, thereby summarizing the two types of answer information obtained to output the final answer text and display it, so that the display device can display the application interface and the target answer information on the same interface at the same time, and the user does not have to repeatedly switch different interfaces to view the answer information, simplifying the display process of the answer information.

[0140] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0141] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0143] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A display configured to display an application interface; A memory configured to store an information query model, wherein the information query model includes an information retrieval module and an answer generation module; A sound collector is configured to collect audio data; The controller is configured as: In response to the voice query instruction, obtaining a query question voice, and performing voice recognition on the query question voice to obtain a text to be queried; Inputting the text to be queried into the information retrieval module, parsing the text to be queried by the information retrieval module to obtain the target application name and key information of the problem, and performing vectorization on the text to be queried by the information retrieval module to obtain a target text vector; According to the target application name and the key information of the question, first answer information is retrieved from a first information base, and second answer information is retrieved from a second information base according to the target text vector, wherein the first information base is an information base containing application information, and the second information base is an information base generated according to a slice vector, and the slice vector is obtained by performing vectorization based on the slice text of the application information; Inputting the first answer information and the second answer information into the answer generation module to generate a target answer text; Generate text prompt information according to the target answer text, and control the display to display the text prompt information on the application interface.

2. The display device according to claim 1, characterized in that The controller executes the retrieval of first answer information from the first information repository and is configured to: Retrieving an answer information set in the first information database according to the target application name, the answer information set including application information of the target application; Calculating a matching value between the application information in the answer information set and the key information of the question; The first answer information is determined according to the matching value, where the first answer information is the application information with the highest matching value in the answer information set.

3. The display device according to claim 2, characterized in that The controller calculates the matching value between the application information in the answer information set and the key information of the question, and is configured to: Analyze the keywords in the key information of the problem; Obtaining the word frequency of the keyword in the application information; The matching value is generated according to the term frequency.

4. The display device according to claim 1, characterized in that The controller performs the retrieval of second answer information in the second information repository according to the target text vector, and is configured to: Calculating the vector similarity between the target text vector and the slice vector; Marking the slice vectors whose vector similarity is greater than or equal to the similarity threshold as preliminary slice vectors; Calculating answer scores for the prepared slice vector and the target text vector; The second answer information is determined according to the answer score, where the second answer information is application information corresponding to the prepared slice vector having the highest answer score among the prepared slice vectors.

5. The display device according to claim 4, characterized in that The controller determines the second answer information according to the answer score, and is configured to: Obtaining a metadata index of the prepared slice vector; Acquire an answer information segment according to the metadata index, wherein the answer information segment and the slice information corresponding to the prepared slice vector are derived from the same application information; The second answer information is generated according to the answer information segment and the slice information corresponding to the prepared slice vector.

6. The display device according to claim 1, characterized in that Before the controller executes the step of retrieving the second answer information from the second information base according to the target text vector, the controller is further configured to: Acquire application information in the first information database; Slicing the application information to obtain slice information, and vectorizing the slice information to obtain a slice vector; The second information base is constructed according to the slice vector.

7. The display device according to claim 1, characterized in that The controller executes inputting the first answer information and the second answer information into the answer generation module, and is configured to: Extracting a prompt text segment according to the first answer information and the second answer information; The first answer information and the second answer information are combined to generate a target answer text according to the prompt text segment.

8. The display device according to claim 1, characterized in that The controller generates text prompt information according to the target answer text and is configured to: Converting the target answer text into a voice format to obtain a target answer audio; In the process of controlling the display to display the text prompt information, the audio output device is controlled to play the target answer audio.

9. The display device according to claim 1, characterized in that: The controller performs the step of inputting the text to be queried into the information retrieval module, and is further configured to: If the target application name and key problem information are not obtained through parsing, the currently displayed application interface image is obtained; Performing feature detection on the application interface image to obtain feature element information; The target application name and key information of the problem are identified according to the characteristic element information.

10. A method for displaying answer information, characterized in that: The display device applied to any one of claims 1 to 9, the display device comprising a display, a memory, a sound collector and a controller, the display being configured to display an application interface, the memory being configured to store an information query model, the information query model comprising an information retrieval module and an answer generation module; The sound collector is configured to collect audio data, and the method comprises: In response to the voice query instruction, obtaining a query question voice, and performing voice recognition on the query question voice to obtain a text to be queried; Inputting the text to be queried into the information retrieval module, parsing the text to be queried by the information retrieval module to obtain the target application name and key information of the problem, and performing vectorization on the text to be queried by the information retrieval module to obtain a target text vector; According to the target application name and the key information of the question, first answer information is retrieved from a first information base, and second answer information is retrieved from a second information base according to the target text vector, wherein the first information base is an information base containing application information, and the second information base is an information base generated according to a slice vector, and the slice vector is obtained by performing vectorization based on the slice text of the application information; Inputting the first answer information and the second answer information into the answer generation module to generate a target answer text; Generate text prompt information according to the target answer text, and control the display to display the text prompt information on the application interface.

Images