Display device and voice retrieval method

By combining the speech recognition model and semantic analysis of multiple rounds of historical conversation data in the display device, the multi-round dialogue and recall capabilities of voice interaction are improved, and the problems of weak multi-round dialogue and poor recall capabilities in traditional voice interaction are solved. Achieve more accurate and relevant information matching.

CN118915986AInactive Publication Date: 2024-11-08HISENSE ELECTRONIC TECH (WUHAN) CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410948952.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional voice interaction is weak in multiple rounds of dialogue and recall capabilities, and cannot accurately match user needs, resulting in inaccurate information provided or deviation from user needs.

Method used

By introducing a speech recognition model into the display device, combining multiple rounds of historical session data for semantic analysis and mixed search, a complete query statement is generated, and a preset recognition model is used to output target recognition results based on the comprehensive score and target sorting results, improving multiple rounds of conversations Ability and recall capability.

Benefits of technology

It improves the multi-round dialogue and recall capabilities during voice interaction, ensures the accuracy and relevance of information, and meets user needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915986A_ABST
    Figure CN118915986A_ABST
Patent Text Reader

Abstract

The invention provides a display device and a voice retrieval method, and the method comprises the steps: carrying out the voice recognition of a voice query request inputted by a user, and requesting a server for multiple rounds of historical session data, the multiple rounds of historical session data being session data in a preset time period or session data of a preset number of conversations; performing semantic analysis on the voice recognition result and the multiple rounds of historical session data, and fusing semantic analysis results to generate a complete query statement; and executing mixed retrieval on the complete query statement, inputting the comprehensive score and the target sorting result into a preset recognition model, and enabling the preset recognition model to output a target recognition result of the voice query request. According to the method, the voice query request input by the user can be retrieved through the preset recognition model obtained through training of the cue words which refer to the context dialogue content and are adjusted and optimized, the multi-round dialogue ability in the voice interaction process can be improved, the recall ability can be improved, and the problems that the multi-round dialogue ability is weak and the recall ability is poor are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a voice retrieval method. Background Art

[0002] Display devices can not only output various media such as movies, TV, and pictures, but also have the function of speech recognition. Speech recognition can also be called automatic speech recognition (ASR), and its goal is to convert the content contained in the speech signal into computer-readable input, such as text sequences, to achieve voice interaction between users and display devices.

[0003] Taking television as an example, voice interaction in television can be achieved through the cooperation of built-in software and hardware, allowing users to operate the television through voice commands or voice requests, such as changing channels, adjusting volume, searching for programs, etc. For example, a user can issue a voice query request, and the user speaks the content he wants to query to the voice receiver of the TV, such as "play the latest science fiction movie" or "what's the weather like today", etc. After the built-in voice recognition system of the TV receives the user's voice input, it can convert it into text. The TV can perform natural language processing on the converted text to understand the user's intentions and needs. According to the user's query, the TV can search for relevant information in the built-in database or through the Internet, and feedback the searched information to the user in voice or visual form to complete the voice interaction between the user and the TV.

[0004] However, traditional voice interaction is often based on simple command and response modes, lacking the ability to understand context, and therefore has weak multi-round dialogue capabilities. Especially when recalling information, it may not accurately match user needs, resulting in inaccurate information provided or deviation from user needs, and therefore, its recall ability is poor. Summary of the invention

[0005] Some embodiments of the present application provide a display device and a voice retrieval method to solve the problems of weak multi-round dialogue capability and poor recall capability during voice interaction.

[0006] In a first aspect, some embodiments of the present application provide a display device, including:

[0007] a display configured to display a user interface;

[0008] A communication device configured to establish a communication connection with a server;

[0009] The controller is configured as:

[0010] Performing speech recognition on the voice query request input by the user to obtain a speech recognition result;

[0011] Requesting a server for multiple rounds of historical conversation data, where the multiple rounds of historical conversation data are conversation data within a preset time period, or the multiple rounds of historical conversation data are conversation data for a preset number of conversations;

[0012] Performing semantic analysis on the speech recognition result and the multiple rounds of historical conversation data to obtain a semantic analysis result; and fusing the semantic analysis result to generate a complete query statement;

[0013] Performing a hybrid search on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement;

[0014] The comprehensive score and the target ranking result are input into a preset recognition model so that the preset recognition model outputs a target recognition result for the voice query request according to the comprehensive score and the target ranking result; the preset model is a neural network model trained using prompt words, and the prompt words are keywords determined by referring to the context dialogue content.

[0015] In a second aspect, some embodiments of the present application provide a voice retrieval method, which can be applied to the display device of the first aspect, the display device comprising a display, a communication device and a controller, the method comprising:

[0016] Performing speech recognition on the voice query request input by the user to obtain a speech recognition result;

[0017] Requesting a server for multiple rounds of historical conversation data, where the multiple rounds of historical conversation data are conversation data within a preset time period, or the multiple rounds of historical conversation data are conversation data for a preset number of conversations;

[0018] Performing semantic analysis on the speech recognition result and the multiple rounds of historical conversation data to obtain a semantic analysis result; and fusing the semantic analysis result to generate a complete query statement;

[0019] Performing a hybrid search on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement;

[0020] The comprehensive score and the target ranking result are input into a preset recognition model so that the preset recognition model outputs a target recognition result for the voice query request according to the comprehensive score and the target ranking result; the preset model is a neural network model trained using prompt words, and the prompt words are keywords determined by referring to the context dialogue content.

[0021] It can be seen from the above technical scheme that some embodiments of the present application provide a display device and a voice retrieval method, the method comprising: performing voice recognition on a voice query request input by a user to obtain a voice recognition result; requesting multiple rounds of historical conversation data from a server, the multiple rounds of historical conversation data are conversation data within a preset time period, or the multiple rounds of historical conversation data are conversation data for a preset number of conversations; performing semantic analysis on the voice recognition result and the multiple rounds of historical conversation data to obtain a semantic analysis result; and, fusing the semantic analysis results to generate a complete query statement; performing hybrid retrieval on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement; inputting the comprehensive score and the target ranking result into a preset recognition model, so that the preset recognition model outputs a target recognition result for the voice query request according to the comprehensive score and the target ranking result; the preset model is a neural network model trained using prompt words, and the prompt words are keywords determined by referring to the context conversation content. The method can retrieve the voice query request input by the user through a preset recognition model obtained by training the prompt words after reference to the context dialogue content and optimization, which can improve the multi-round dialogue ability and recall ability in the voice interaction process, and solve the problems of weak multi-round dialogue ability and poor recall ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate some embodiments of the present application or technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0024] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0025] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0026] Figure 4 A schematic diagram of a flow chart of a display device executing a voice retrieval method provided in some embodiments of the present application;

[0027] Figure 5 A schematic diagram of a flow chart of a display device performing multiple rounds of query fusion according to some embodiments of the present application;

[0028] Figure 6A flow chart of a display device provided in some embodiments of the present application, in which a preset recognition model is trained by prompt words so that the preset recognition model outputs a complete query sentence according to speech recognition results and multiple rounds of historical conversation data;

[0029] Figure 7 A schematic diagram of scene prompts for prompt words provided in some embodiments of the present application;

[0030] Figure 8 A schematic diagram of a flow chart of a display device performing hybrid retrieval on multiple rounds of fusion query results provided in some embodiments of the present application;

[0031] Fig. 9 A schematic diagram of the timing of a display device performing part-of-speech recognition on multi-round query fusion results provided in some embodiments of the present application;

[0032] Fig.10 A schematic diagram of a flow chart of word segmentation performed by a display device provided in some embodiments of the present application;

[0033] Fig.11 A schematic diagram of a flow chart of a display device providing some embodiments of the present application generating a comprehensive score of a complete query statement and a target ranking result corresponding to the complete query statement. DETAILED DESCRIPTION

[0034] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0035] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0036] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0037] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0038] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0039] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0040] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0041] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0042] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0043] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0044] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0045] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0046] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0047] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0048] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0049] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0050] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0051] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0052] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0053] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0054] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0055] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0056] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0057] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0058] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0059] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0060] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0061] like Figure 3 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0062] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0063] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0064] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0065] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0066] The above embodiments illustrate the hardware / software architecture and functional implementation of a display device 200. Based on the above display device 200, not only can various media such as movies, television, and pictures be output, but also a voice recognition function can be provided. Voice recognition can also be referred to as automatic speech recognition (Automatic Speech Recognition, ASR), and its goal is to convert the content contained in the voice signal into a computer-readable input, such as a text sequence, etc., to achieve voice interaction between the user and the display device 200.

[0067] The voice interaction in the display device 200, such as a TV, can be realized through the cooperation of built-in software and hardware, allowing the user to operate the TV through voice commands or voice requests, such as changing channels, adjusting volume, searching for programs, etc. For example, a user can issue a voice query request, and the user speaks the content he wants to query to the voice receiver of the TV, such as "play the latest science fiction movie" or "how is the weather today", etc. After the built-in voice recognition system of the TV receives the user's voice input, it can convert it into text. The TV can perform natural language processing on the converted text to understand the user's intentions and needs. According to the user's query, the TV can search for relevant information in the built-in database or through the Internet, and feedback the searched information to the user in voice or visual form to complete the voice interaction between the user and the TV.

[0068] However, traditional voice interaction is often based on simple command and response modes, lacking the ability to understand context, and therefore has weak multi-round dialogue capabilities. Especially when recalling information, it may not accurately match user needs, resulting in inaccurate information provided or deviation from user needs, and therefore, its recall ability is poor.

[0069] In order to solve the problem of weak multi-round dialogue ability and poor recall ability in the voice interaction process, some embodiments of the present application provide a display device 200, which includes a display 260, a communication device 220 and a controller 250. Among them, the display 260 is configured to display a user interface, the communication device 220 is configured to establish a communication connection with the server 400, and the controller 250 enables the display device 200 to execute a voice retrieval method by running an application. The display device 200 can retrieve the voice query request input by the user through a preset recognition model, and the preset recognition model is obtained after training with the optimized prompt words. The prompt words refer to the context conversation content, which can improve the multi-round dialogue ability and recall ability in the voice interaction process, thereby solving the problem of weak multi-round dialogue ability and poor recall ability.

[0070] In order to facilitate the understanding of the technical solutions in some embodiments of the present application, each step is described in detail below in conjunction with some specific embodiments and drawings. Figure 4 A flow chart of a display device performing a voice retrieval method according to some embodiments of the present application is shown in FIG. Figure 4 As shown, in some embodiments, when the display device 200 performs the voice retrieval method, the following steps S1-S5 may be included, and the specific contents are as follows:

[0071] Step S1: the display device 200 performs voice recognition on the voice query request input by the user to obtain a voice recognition result.

[0072] In some embodiments, the user can input a voice query request through a remote control device or other means. Taking the remote control device as an example, after receiving the voice query request, the remote control device can start the corresponding voice recognition engine to receive and process the user's voice query request. For example, the voice recognition engine can analyze the characteristics of the voice signal in the voice query request, such as pitch, length, intensity, etc., and can convert the voice signal into text data to generate a voice recognition result. In this way, the display device 200 can convert the user's voice query request into processable text form data to provide a data basis for subsequent semantic analysis. After step S1 is executed, the following step S2 can be included.

[0073] Step S2: the display device 200 requests multiple rounds of historical conversation data from the server.

[0074] In order to better understand the user's query intention and context and enhance the accuracy of semantic analysis, in some embodiments, the display device 200, such as a television, can request multiple rounds of historical conversation data from the server 400, where the multiple rounds of historical conversation data are conversation data within a preset time period, or the multiple rounds of historical conversation data are conversation data for a preset number of conversations. For example, a request can be initiated to the server 400 according to the time period and number set by the user or preset by the television system. The server 400 retrieves and returns multiple rounds of historical conversation data related to the current query request and returns it to the display device 200. The display device 200 receives and parses this data for subsequent semantic analysis. The multiple rounds of historical conversation data may include context information requested by the television user within a certain time period, or context information of the user in recent times, and may include text conversation content, timestamps, user behavior and other data.

[0075] In this way, by using multiple rounds of historical conversation data, the display device 200 can better understand the user's query intention and context, enhance the accuracy of semantic analysis, optimize query results according to the user's historical behavior, improve query efficiency, and reduce misunderstandings or repeated queries caused by missing context. After step S2 is executed, the following step S3 may be included.

[0076] Step S3: the display device 200 performs semantic analysis on the speech recognition result and multiple rounds of historical conversation data to obtain a semantic analysis result; and integrates the semantic analysis result to generate a complete query statement.

[0077] In order to more accurately understand the user's voice query request, in some embodiments, the display device 200 can perform semantic analysis on the voice recognition result and multiple rounds of historical conversation data to obtain a semantic analysis result; and fuse the semantic analysis result to generate a complete query statement, that is, perform multiple rounds of query fusion. Generally speaking, the display device 200 performs semantic analysis on the voice recognition result and multiple rounds of historical conversation data, and fuses them to generate a complete query statement.

[0078] Figure 5 A schematic diagram of a flow chart of a display device performing multiple rounds of query fusion according to some embodiments of the present application, such as Figure 5As shown, in some embodiments, the display device 200 can perform rule template recall based on the speech recognition result and multiple rounds of historical conversation data, and then perform regular matching on the template recall result, and determine whether the regular matching is successful. According to the result of the regular matching, it can be divided into two cases. One is that the regular matching is successful, which means that the voice query request does not need to rely on context information, that is, the current speech recognition result can match the preset rule. The preset rule can be a database, and the database can contain corresponding paradigms, rules, etc. In the case of matching, it means that the voice query request does not rely on context information, and the speech recognition result is directly used as the result of multiple rounds of query fusion. The second case is dependent on context information, that is, the regular matching is unsuccessful, then the display device 200 can perform splicing of the voice query result and multiple rounds of historical conversations, and filter the spliced ​​results as the final multi-round fusion query result. The following will be described in detail in combination with the above two cases.

[0079] In some embodiments, the display device 200 can perform document matching recall based on word frequency and inverse document frequency on the speech recognition results to obtain recalled text information, and then use the recalled text information as a regular expression list to search for database text information that is identical to the recalled text information in a preset database to perform regular matching on the speech recognition results; when database text information that is identical to the recalled text information is queried in the preset database, the speech recognition result is output as a complete query statement; when database text information that is identical to the recalled text information is not queried in the preset database, a preset recognition model is trained through prompt words so that the preset recognition model outputs a complete query statement based on the speech recognition results and multiple rounds of historical conversation data.

[0080] Exemplarily, when the display device 200 performs rule template recall, in order to determine whether the voice query request is context-dependent, a document matching recall based on (Term Frequency-Inverse Document Frequency, TF-IDF) can be performed according to the voice recognition result. TF-IDF can calculate the importance of a word to a document by measuring the frequency of a word appearing in a document (TF) and the rarity of the word in the entire document collection (IDF). Among them, term frequency (TF) refers to the frequency of a word appearing in a document, which can be calculated by dividing the number of times the word appears in a document by the total number of words in the document. For example, the more frequently a word appears in a document, the higher its TF value. Inverse Document Frequency (IDF) is an indicator of the general importance of a word, which is inversely proportional to the frequency of the word appearing in a document collection. If a word appears in many documents, its IDF value is low; conversely, if a word appears in only a few documents, its IDF value is high. It should be noted that the documents in this application can be understood as text information, for example, can be paradigmatic questions and answers, such as "help me search for Zhou's songs."

[0081] Based on the above document recall results, the recalled text information can be obtained. The display device 200 can use the tf-idf recalled document content as a regular expression list to perform regular matching on the speech recognition results. If the regular match is successful, that is, when the database text information that is the same as the recalled text information is found in the preset database, the speech recognition result can be directly output as a complete query statement, that is, the speech recognition result is returned as a multi-round fusion query result. If the regular match fails, that is, when the database text information that is the same as the recalled text information is not found in the preset database, the preset recognition model can be trained through the prompt words so that the preset recognition model outputs a complete query statement based on the speech recognition results and multiple rounds of historical conversation data.

[0082] Figure 6 A flow chart of a display device provided in some embodiments of the present application using a prompt word to train a preset recognition model so that the preset recognition model outputs a complete query statement according to a speech recognition result and multiple rounds of historical conversation data, such as Figure 6As shown, in some embodiments, when the regular match fails, the display device 200 can first obtain a prompt word, and tune the prompt word according to multiple rounds of historical conversation data, a preset recognition model and preset rules; the preset rules at least include the paradigm format of the prompt word. Afterwards, the tuned prompt word, the speech recognition result and the multiple rounds of historical conversation data can be spliced ​​to obtain a first splicing result. The preset recognition model is trained based on the tuned prompt word, and the first splicing result is input into the trained preset recognition model, and then the first splicing result is filtered by the preset recognition model to delete invalid words in the first splicing result to obtain a complete query statement; invalid words at least include stop words.

[0083] Exemplarily, when the regular match fails, the following operations may be performed. The display device 200 may assemble the prompt word prompt, that is, splice and assemble according to the optimized prompt speech recognition result and the historical conversation data. Then, the preset recognition model may be called based on the optimized prompt word, and then the results returned by the model may be filtered to filter out some invalid words, such as some stop words or non-formatted content, and then the filtered results may be returned as the final multi-round fusion query results.

[0084] In order to make the voice retrieval results more accurate, in the embodiment of the present application, on the one hand, the prompt word prompt can be tuned, and the optimization of the prompt word can improve the accuracy of the preset recognition model prediction. On the other hand, the preset recognition model can be trained by the tuned prompt. In this way, the prompt word and the preset recognition model are both optimized, which can improve the accuracy of the voice retrieval results.

[0085] In order to optimize the prompt words, in some embodiments, the display device 200 can tune the prompt words according to multiple rounds of historical conversation data, preset recognition models and preset rules. Specifically, the display device 200 can first parse multiple rounds of historical conversation data to obtain the conversation characteristic data in the multiple rounds of historical conversation data, and then obtain the model performance data of the preset recognition model. Then extract the paradigm format in the preset rules, and finally adjust the prompt words based on the conversation characteristic data, model performance data and paradigm format. In this way, the display device 200 adjusts the prompt words by analyzing multiple rounds of historical conversation data, preset recognition models and preset rules, and trains the preset recognition model through the prompt words. The conversation characteristic data, model performance data and paradigm format can be combined to finely adjust the prompt words to make them more accurate and fit the user's intentions, improve the multi-round dialogue ability in the voice interaction process, and improve the recall ability, thereby solving the problem of weak multi-round dialogue ability and poor recall ability in the voice interaction process.

[0086] Exemplarily, when analyzing historical interaction data, the display device 200 may include user query statements, system responses, and user feedback. By statistically analyzing these data, the display device 200 can identify common query patterns, user-preferred prompt words, and high-frequency question types. After that, the display device 200 can analyze the preset recognition model used to process the session data. For example, by analyzing the input-output relationship, accuracy, recall rate and other performance indicators of the model, the display device 200 can obtain model performance data, which can reflect the performance of the model under different scenarios and conditions. In addition, the display device 200 can also use a series of preset rules, which can define the standard format, grammatical structure, keyword combination, etc. of the prompt word. The extraction of the paradigm format can ensure that the prompt word not only conforms to the language habits, but also can guide the user to express the needs more accurately. Based on the above data, the display device 200 can perform actual tuning work. For example, based on the session characteristic data, the display device 200 can understand the user's common expression habits, thereby adjusting the prompt word to better match the user's language style. Based on the model performance data, the display device 200 can identify the model's weaknesses in understanding certain types of queries, and optimize the prompt words accordingly to enhance the model's predictive ability. According to the paradigm format, the display device 200 can ensure that the prompt words have a clear structure and clear semantics to avoid confusion or misunderstanding of users.

[0087] Figure 7 A schematic diagram of scene prompts for prompt words provided in some embodiments of the present application, such as Figure 7 As shown, in some embodiments, the prompt words can be iteratively optimized based on user feedback, and the display device 200 can collect key performance indicators such as the click rate of the prompt words and the query conversion rate of the users. Through continuous iterative learning, the optimization strategy of the prompt words is further refined to make the prompt words more personalized and accurate. After step S3 is executed, the following step S4 can be executed.

[0088] Step S4: the display device 200 performs a hybrid search on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement.

[0089] In order to enable voice retrieval to more comprehensively cover information that users may be interested in and improve the breadth and depth of retrieval, in some embodiments, after obtaining multiple rounds of fusion query results, i.e., complete query statements, the display device 200 may perform hybrid retrieval on the complete query statements to generate a comprehensive score for the complete query statement and a target ranking result corresponding to the complete query statement. In this way, the comprehensive score and ranking results may provide users with an intuitive and orderly query result display, thereby improving user experience.

[0090] Figure 8A schematic diagram of a flow chart of a display device performing hybrid retrieval on multiple rounds of fusion query results provided in some embodiments of the present application, such as Figure 8 As shown, in some embodiments, the display device 200 can perform related content retrieval on the multi-round fusion query results, for example, semantic retrieval and keyword matching, which are processed in parallel. The processing process of the multi-round fusion query results is described in detail below.

[0091] See also Figure 8 The left branch, for multiple rounds of query vectorization and vectorized recall processes, in some embodiments, the display device 200 can first perform vectorization processing on the complete query statement to obtain a vector processing result, and then use an approximate semantic recall method based on (Hierarchical Navigable Small World, HNSW) to perform vectorized recall on the vector processing result to obtain a vector recall result and a first score corresponding to the vector recall result.

[0092] Exemplarily, vectorization is the process of converting text data into a numerical vector, and the text can be represented as a fixed-length vector in which each element represents a specific feature or attribute. When vectorizing a complete query statement, the BCEmbedding model can be used. For example, the BCEmbedding model can be used to encode the query statement to obtain its vector representation. The complete query statement can be input into the BCEmbedding model, and then converted into a semantic vector through the encoding function of the model to obtain a vector processing result. Afterwards, the approximate semantic recall method based on HNSW can be used to perform vectorized recall on the vector processing result. Among them, the approximate semantic recall method based on HNSW can accelerate the search process by constructing a hierarchical graph structure, making it faster to find the most similar vector in a large-scale data set. In this way, through vectorization, text data can be converted into numerical vectors, and the approximate semantic recall method based on HNSW can improve retrieval efficiency. Therefore, the processing and retrieval efficiency of text data can be improved through multiple rounds of query vectorization and vectorized recall.

[0093] See also Figure 8 The right branch can also process the following processes in parallel for the results of multiple rounds of query fusion. The first is keyword extraction. The keyword extraction process can be based on preset rules and preset recognition models. Before performing keyword extraction, part-of-speech recognition can be performed on the complete query statement. Part-of-speech recognition can better understand the structure and semantics of the sentence by analyzing the part of speech of each word in the text.

[0094] Fig. 9 A schematic diagram of a timing sequence of a display device performing part-of-speech recognition on multi-round query fusion results provided in some embodiments of the present application, such as Fig. 9 As shown, in some embodiments, the display device 200 can perform part-of-speech recognition on the results of multiple rounds of query fusion. If all the parts of speech are recognized, keyword extraction can be performed, and the extracted results are returned as recognition results. If the parts of speech are not fully recognized, the prompt words can be spliced ​​based on the complete query statement, and then the preset recognition model is called and content filtering is performed, and the filtered results are used as recognition results, and the final recognition results are returned. The following will be introduced in detail for different situations.

[0095] In some embodiments, when the display device 200 performs part-of-speech recognition, word segmentation can be performed based on various domain knowledge and a customized vocabulary. Fig.10 A schematic diagram of a flow chart of a display device performing word segmentation according to some embodiments of the present application, such as Fig.10 As shown, the display device 200 can first obtain the target database corresponding to the complete query statement, and then perform word segmentation on the complete query statement based on the target database to obtain a word segmentation result, and the word segmentation result is used to identify the part of speech of the complete query statement; the recognition result of the part of speech includes fully recognized and not fully recognized. If the recognition result is fully recognized, extract the keywords in the word segmentation result; if the recognition result is not fully recognized, obtain a second splicing result based on the splicing prompt word of the complete query statement, and input the second splicing result into the trained preset recognition model, so as to filter the second splicing result through the preset recognition model to delete invalid words in the second splicing result, and extract keywords in the filtered second splicing result.

[0096] Exemplarily, when performing word segmentation based on knowledge in various fields and customized vocabulary, the relevant parts of speech can be expressed according to the word segmentation results. In some embodiments, a third-party tool jieba can be used to perform word segmentation. Suppose the target database contains a film and television entity vocabulary videoname, a stop word library stopword, and a playback vocabulary player. Among them, the film and television entity vocabulary videoname is used to store film and television name entity words; the stop word library stopword is used to store invalid words, such as invalid words that may include help me, search, and other invalid words; the playback vocabulary player includes play, play, play, etc. If the complete query sentence entered is "Help me search for the name of movie A", the result after word segmentation is help me (stopword) + search (stopword) + movie A (videoname). According to the results of word segmentation, compared with the target database, it can be determined whether the parts of speech are fully recognized.

[0097] For the keyword extraction stage, when all the parts of speech of the complete query statement are recognized, the keywords in the word segmentation results can be extracted according to the defined keyword part of speech range. For example, if the keyword parts of speech are defined to include: videoname, then enter "help me search for movie A" and the extracted keywords are: movie A. When the parts of speech of the complete query statement are not fully recognized, the second splicing result can be obtained based on the splicing prompt words of the complete query statement, that is, prompt assembly is performed, and prompt assembly is performed according to the optimized prompt and the corresponding multi-round query fusion results. The second splicing result can then be input into the trained preset recognition model, and the results returned by the preset recognition model need to be filtered for redundant information to filter out redundant invalid information. After the filtering is completed, the keywords in the filtered second splicing result can be extracted.

[0098] After the keyword extraction is completed, the display device 200 can perform keyword matching. In some embodiments, the display device 200 can query the target database based on the keyword to obtain a keyword matching result, and then use a deviation normalization method to perform a linear transformation on the keyword matching result to normalize the keyword matching result to obtain a normalized result and a second score corresponding to the normalized result.

[0099] In some embodiments, keyword matching can be performed using Elasticsearch (ES), which is a distributed, highly scalable, and highly real-time search and data analysis engine that can easily enable large amounts of data to be searched, analyzed, and explored. It includes inverted index technology and semantic retrieval technology. In actual use scenarios, you can choose according to your needs, and this application does not make specific restrictions on this.

[0100] For example, Figure 8 The first score result of the left branch after vectorized recall is in the vector form of [0,1], while the score of the right branch after keyword matching is not in vector form. In order to facilitate subsequent re-ranking, the score of keyword matching needs to be converted into vector form, that is, score normalization is performed. For example, the deviation normalization method can be used to perform linear transformation on the keyword matching results so that the results are mapped between [0,1].

[0101] In some embodiments, the conversion may be performed by the following conversion function:

[0102]

[0103] Where score can be the score vector of the keyword matching result, min(score) is the minimum value, and max(score) is the maximum value. *is the normalized score. In this way, after the multiple rounds of query fusion results are processed in parallel by two branches, the scores are expressed in the same form, which can be re-ranked based on them.

[0104] Fig.11 A flow chart of a display device generating a comprehensive score of a complete query statement and a target ranking result corresponding to the complete query statement provided in some embodiments of the present application, such as Fig.11 As shown, in some embodiments, when the display device 200 performs a hybrid retrieval on a complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement, it can first obtain a first score corresponding to the vector recall result and a second score corresponding to the normalized result, and re-rank the vector recall result and the normalized result based on the first score and the second score, and then obtain the target ranking result corresponding to the re-ranked complete query statement, and calculate the comprehensive score of the complete query statement based on the target ranking result.

[0105] In some embodiments, a text matching method can be used when performing vector recall. Text matching refers to calculating the similarity between two texts. Text similarity can include semantic similarity and literal similarity. The similarity scores of two texts are in [0,1]. The higher the score, the higher the similarity between the two texts, and vice versa. The similarity is lower. For example, the present application can consider semantic similarity and obtain the vector recall result through vector cosine similarity. In actual usage scenarios, you can choose according to actual needs, and the present application does not make specific limitations on this.

[0106] When calculating the comprehensive score of the complete query statement according to the target ranking result, the document comprehensive information in the target ranking result can be first obtained, wherein the document comprehensive information includes the vector recall result and the normalization result, and then the first weight value of the vector recall result is set, and the second weight value of the normalization result is set; the sum of the first weight value and the second weight value is equal to the first preset value. When the first score and the second score are both greater than the second preset value, the comprehensive score of the document comprehensive information corresponding to the complete query statement is calculated according to the first score, the second score, the first weight value and the second weight value; when the first score is less than or equal to the second preset value and the second score is greater than the second preset value, the third weight value of the normalization result is set, and the comprehensive score of the document comprehensive information corresponding to the complete query statement is calculated according to the third weight value and the second score; when the first score is greater than the second preset value and the second score is less than or equal to the second preset value, the fourth weight value of the normalization result is set, and the comprehensive score of the document comprehensive information corresponding to the complete query statement is calculated according to the fourth weight value and the second score.

[0107] Exemplarily, after re-ranking the vector recall results and the normalized results based on the first score and the second score, the comprehensive score of the complete query statement can be calculated in the following manner. Taking the first score as the vector recall score vscore and the second score as the keyword recall score kscore as an example, the first weight is weight1, the second weight is weight2, the third weight is weight3, and the fourth weight is weight4, the following conclusions can be obtained:

[0108] When the keyword recall score (kscore)>0 and the vector recall score (vscore)>0, the comprehensive score of the comprehensive information of the document corresponding to the complete query statement is:

[0109] fscore=kscore*weight1+vscore*weight2, where weight1+weight2=1

[0110] When kscore>0 and vscore<=0, the comprehensive score is calculated as follows:

[0111] fscore = kscore * weight3

[0112] When kscore<=0 and vscore>0, the comprehensive score is calculated as follows:

[0113] fscore = kscore * weight4

[0114] By the above method, the comprehensive score of the complete query statement and the target ranking result corresponding to the complete query statement can be generated. After step S4 is executed, the following step S5 can be included.

[0115] Step S5: the display device 200 inputs the comprehensive score and the target ranking result into a preset recognition model, so that the preset recognition model outputs a target recognition result for the voice query request according to the comprehensive score and the target ranking result.

[0116] In order to make the target recognition result output by the preset recognition model closer to the actual needs and expectations of the user, in some embodiments, the display device 200 can input the comprehensive score and the target ranking result into the preset recognition model, so that the preset recognition model outputs the target recognition result for the voice query request according to the comprehensive score and the target ranking result. The preset recognition model is obtained by training the optimized prompt word, and the prompt word can refer to the context conversation content.

[0117] Exemplarily, the preset recognition model is a model based on the tuned prompt words and the contextual conversation content. The display device 200 can pass the comprehensive score and the target ranking result as input to the preset recognition model. The preset recognition model can perform in-depth analysis and learning based on the input score and ranking result, and output the target recognition result for the voice query request.

[0118] In an embodiment of the present application, by referring to the prompt words of the context and training the preset recognition model with the optimized prompt words, the preset recognition model can be made to have the multi-round query fusion capability of specific prompt words. By extracting keywords based on preset rules and preset recognition models, the word segmentation method that relies on a large number of real-time word libraries can be abandoned, and the keywords in the sentence can be extracted with the preset recognition model as the center, and the content recall based on TF-IDF can be performed based on this. Afterwards, the result scores of semantic recall and keyword matching recall are comprehensively scored, and this is used as the final basis for sorting and document truncation. Multi-round query fusion can improve the coherence and consistency of dialogue by integrating multi-round dialogue information, and solve the problem of weak multi-round dialogue ability. Extracting keywords for content recall can improve the accuracy and comprehensiveness of recall, and the re-ranking of retrieved documents can optimize the document sorting by comprehensively scoring different recall methods, so that the most relevant content can be displayed first, which can improve the user search experience. Therefore, the present application can improve the multi-round dialogue ability and recall ability in the voice interaction process, and improve the user voice retrieval experience.

[0119] It can be seen from the above technical solution that the above embodiment provides a display device 200, which performs voice recognition on the voice query request input by the user to obtain a voice recognition result; requests multiple rounds of historical conversation data from the server, and the multiple rounds of historical conversation data are conversation data within a preset time period, or the multiple rounds of historical conversation data are conversation data for a preset number of conversations; performs semantic analysis on the voice recognition result and the multiple rounds of historical conversation data to obtain a semantic analysis result; and, integrates the semantic analysis results to generate a complete query statement; performs hybrid retrieval on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement; inputs the comprehensive score and the target ranking result into a preset recognition model, so that the preset recognition model outputs a target recognition result for the voice query request according to the comprehensive score and the target ranking result; the preset model is a neural network model trained using prompt words, and the prompt words are keywords determined with reference to the context conversation content. The display device 200 can retrieve the voice query request input by the user through a preset recognition model. The preset recognition model is obtained through training with optimized prompt words. The prompt words refer to the context of the conversation content, which can improve the multi-round conversation ability and recall ability during the voice interaction process, thereby solving the problems of weak multi-round conversation ability and poor recall ability.

[0120] Based on the above display device 200, some embodiments of the present application further provide a voice retrieval method, which can be applied to the display device 200 in the above embodiment. In some embodiments, the method may include the following content:

[0121] Performing speech recognition on the voice query request input by the user to obtain a speech recognition result;

[0122] Requesting a server for multiple rounds of historical conversation data, where the multiple rounds of historical conversation data are conversation data within a preset time period, or the multiple rounds of historical conversation data are conversation data for a preset number of conversations;

[0123] Performing semantic analysis on the speech recognition result and the multiple rounds of historical conversation data to obtain a semantic analysis result; and fusing the semantic analysis result to generate a complete query statement;

[0124] Performing a hybrid search on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement;

[0125] The comprehensive score and the target ranking result are input into a preset recognition model so that the preset recognition model outputs a target recognition result for the voice query request according to the comprehensive score and the target ranking result; the preset model is a neural network model trained using prompt words, and the prompt words are keywords determined by referring to the context dialogue content.

[0126] It can be seen from the above technical solution that the above embodiment provides a voice retrieval method, which can retrieve the voice query request input by the user through a preset recognition model obtained by training the prompt words with reference to the contextual conversation content and optimization, thereby improving the multi-round conversation ability and recall ability during the voice interaction process, and solving the problems of weak multi-round conversation ability and poor recall ability.

[0127] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0128] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0130] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; A communication device configured to establish a communication connection with a server; The controller is configured as: Performing speech recognition on the voice query request input by the user to obtain a speech recognition result; Requesting a server for multiple rounds of historical conversation data, where the multiple rounds of historical conversation data are conversation data within a preset time period, or the multiple rounds of historical conversation data are conversation data for a preset number of conversations; Performing semantic analysis on the speech recognition result and the multiple rounds of historical conversation data to obtain a semantic analysis result; and fusing the semantic analysis result to generate a complete query statement; Performing a hybrid search on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement; Inputting the comprehensive score and the target ranking result into a preset recognition model, so that the preset recognition model outputs a target recognition result for the voice query request according to the comprehensive score and the target ranking result; The preset model is a neural network model trained using prompt words, and the prompt words are keywords determined with reference to the context of the conversation.

2. The display device according to claim 1, characterized in that The controller is further configured to: Performing document matching recall based on word frequency and inverse document frequency on the speech recognition result to obtain recalled text information; Using the recalled text information as a regular expression list, searching a preset database for database text information identical to the recalled text information to perform regular expression matching on the speech recognition result; When database text information identical to the recalled text information is found in the preset database, outputting the speech recognition result as the complete query statement; When the database text information identical to the recalled text information is not found in the preset database, the preset recognition model is trained by prompt words so that the preset recognition model outputs a complete query statement according to the speech recognition result and the multiple rounds of historical conversation data.

3. The display device according to claim 2, characterized in that The controller performs training of the preset recognition model through prompt words so that the preset recognition model outputs a complete query statement according to the speech recognition result and the multiple rounds of historical conversation data, and is further configured as follows: Acquire the prompt word, and tune the prompt word according to the multiple rounds of historical conversation data, the preset recognition model and preset rules; the preset rules at least include the paradigm format of the prompt word; Performing splicing on the optimized prompt word, the speech recognition result and the multiple rounds of historical conversation data to obtain a first splicing result; Training the preset recognition model based on the optimized prompt word, and inputting the first splicing result into the trained preset recognition model; The first concatenation result is filtered through the preset recognition model to delete invalid words in the first concatenation result to obtain a complete query statement; the invalid words at least include stop words.

4. The display device according to claim 3, characterized in that The controller performs tuning of the prompt word according to the multiple rounds of historical conversation data, the preset recognition model and the preset rules, and is further configured to: Parsing the multiple rounds of historical conversation data to obtain conversation characteristic data in the multiple rounds of historical conversation data; Acquiring model performance data of the preset recognition model; Extracting the paradigm format in the preset rule; The prompt words are adjusted based on the session characteristic data, the model performance data, and the paradigm format.

5. The display device according to claim 1, characterized in that The controller is further configured to: Performing vectorization processing on the complete query statement to obtain a vector processing result; A vectorized recall is performed on the vector processing result using an approximate semantic recall method based on HNSW to obtain a vector recall result and a first score corresponding to the vector recall result.

6. The display device according to claim 5, characterized in that The controller is further configured to: Obtaining a target database corresponding to the complete query statement; Performing word segmentation on the complete query statement based on the target database to obtain a word segmentation result, wherein the word segmentation result is used to identify the part of speech of the complete query statement; The recognition results of the parts of speech include fully recognized and not fully recognized; If the recognition result is that all the words have been recognized, extract the keywords in the word segmentation result; If the recognition result is incomplete recognition, the prompt word is spliced ​​based on the complete query sentence to obtain a second splicing result, and the second splicing result is input into the trained preset recognition model to filter the second splicing result through the preset recognition model to delete invalid words in the second splicing result, and extract keywords in the filtered second splicing result.

7. The display device according to claim 6, characterized in that The controller is further configured to: Querying the target database based on the keyword to obtain a keyword matching result; A deviation normalization method is used to perform linear transformation on the keyword matching result to normalize the keyword matching result, thereby obtaining a normalized result and a second score corresponding to the normalized result.

8. The display device according to claim 7, characterized in that The controller performs a hybrid search on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement, and is further configured to: Obtaining the first score corresponding to the vector recall result and the second score corresponding to the normalized result; reordering the vector recall result and the normalized result based on the first score and the second score; A target ranking result corresponding to the re-ranked complete query statement is obtained, and a comprehensive score of the complete query statement is calculated according to the target ranking result.

9. The display device according to claim 8, characterized in that The controller calculates the comprehensive score of the complete query statement according to the target ranking result, and is further configured to: Obtaining comprehensive document information in the target ranking result; the comprehensive document information includes the vector recall result and the normalization result; Setting a first weight value of the vector recall result, and setting a second weight value of the normalization result; The sum of the first weight value and the second weight value is equal to a first preset value; When both the first score and the second score are greater than a second preset value, calculating a comprehensive score of the comprehensive information of the document corresponding to the complete query statement according to the first score, the second score, the first weight value, and the second weight value; When the first score is less than or equal to the second preset value and the second score is greater than the second preset value, setting a third weight value of the normalized result, and calculating a comprehensive score of the comprehensive information of the document corresponding to the complete query statement according to the third weight value and the second score; When the first score is greater than the second preset value and the second score is less than or equal to the second preset value, a fourth weight value of the normalized result is set, and a comprehensive score of the comprehensive information of the document corresponding to the complete query statement is calculated based on the fourth weight value and the second score.

10. A speech retrieval method, applied to the display device according to any one of claims 1 to 9, wherein the display device comprises a display, a communication device and a controller, characterized in that: The method comprises: Performing speech recognition on the voice query request input by the user to obtain a speech recognition result; Requesting a server for multiple rounds of historical conversation data, where the multiple rounds of historical conversation data are conversation data within a preset time period, or the multiple rounds of historical conversation data are conversation data for a preset number of conversations; Performing semantic analysis on the speech recognition result and the multiple rounds of historical conversation data to obtain a semantic analysis result; and fusing the semantic analysis result to generate a complete query statement; Performing a hybrid search on the complete query statement to generate a comprehensive score of the complete query statement and a target ranking result corresponding to the complete query statement; The comprehensive score and the target ranking result are input into a preset recognition model so that the preset recognition model outputs a target recognition result for the voice query request according to the comprehensive score and the target ranking result; the preset model is a neural network model trained using prompt words, and the prompt words are keywords determined by referring to the context dialogue content.

Citation Information

Cited By

  • Large model task routing method, device and medium

    CN120429307A

  • Large model task routing method, device and medium

    CN120429307B