Display device and hot word recognition method

Abstract

This application provides a display device and a hot word recognition method. The display device includes: a display; and a controller, communicatively connected to the display, configured to: receive a voice command input by a user; obtain a candidate sequence corresponding to the voice command, the candidate sequence containing multiple single characters and the probability of the single characters; calculate the probability of the candidate sequence according to a speech recognition path graph with custom weights; determine the semantic text corresponding to the voice command based on the probability of the candidate sequence and the probability of the single characters; generate a response result based on the semantic text; and respond based on the response result. This application improves the hot word recognition effect in speech recognition.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a display device and a hot word recognition method. Background Technology

[0002] Speech recognition is the primary entry point for human-computer interaction, undertaking the crucial task of converting user-inputted voice commands into text. It forms the foundation for functions such as semantic understanding, multi-turn dialogue, and information recommendation. The accuracy of speech recognition is a significant factor affecting the human-computer interaction experience. During human-computer interaction, some user-inputted voice commands contain hot words such as movie / TV show titles and actor names. Accurately identifying these hot words can improve the accuracy of voice command recognition. In related technologies, a pre-built hot word list can be used to identify hot words contained in the voice command. However, in this approach, hot words need to be manually added to the hot word list, resulting in poor real-time performance and impacting speech recognition effectiveness. Summary of the Invention

[0003] To address the aforementioned technical problems, this application provides a display device and a hot word recognition method.

[0004] In a first aspect, this application provides a display device, the display device comprising:

[0005] monitor;

[0006] The controller, which is communicatively connected to the display, is configured to:

[0007] Receive voice commands input by the user;

[0008] Obtain the candidate sequence corresponding to the voice command, wherein the candidate sequence contains multiple single characters and the probability of the single character;

[0009] The probability of the candidate sequence is calculated based on the speech recognition path graph with custom weights.

[0010] The semantic text corresponding to the voice command is determined based on the probability of the candidate sequence and the probability of the single character.

[0011] Generate a response result based on the semantic text;

[0012] Respond according to the stated response result.

[0013] In some embodiments, calculating the probability of the candidate sequence based on a speech recognition path graph with custom weights includes:

[0014] Query the path graph corresponding to the hot word table to obtain the weight sum of each candidate sequence;

[0015] The weights are then normalized to obtain the probability of the candidate sequence.

[0016] In some embodiments, determining the semantic text corresponding to the voice instruction based on the probability of the candidate sequence and the probability of the single character includes:

[0017] Based on the probability of the candidate sequence and the probability of the single character, an attention-based rescoring is performed to obtain the predicted probability of the candidate sequence.

[0018] The semantic text corresponding to the voice instruction is generated based on the candidate sequence with the highest predicted probability.

[0019] In some embodiments, the controller is further configured to:

[0020] Obtain the source data used to generate the test text;

[0021] Construct a question template based on the source data;

[0022] Obtain the question templates for matching hot words in the hot word list, and generate test text based on the hot words and question templates to test the recognition success rate of the hot words.

[0023] In some embodiments, the controller is further configured to:

[0024] Calculate the text inclusion accuracy of each hot word in the hot word list;

[0025] Calculate the text equality accuracy of the hot words;

[0026] The recognition success rate of the hot words is calculated based on the text inclusion accuracy and text equality accuracy.

[0027] Remove hot words whose recognition success rate is lower than the success rate threshold from the hot word list.

[0028] In some embodiments, the controller is further configured to:

[0029] Based on the accuracy of the text inclusion of hot words, the hot words in the hot word list are assigned either a first category label or a second category label;

[0030] Increase the weight of the first type of hot words by a first ratio, and decrease the weight of the second type of hot words by a second ratio;

[0031] Recalculate the text inclusion accuracy of the hot words and update the category identifier of the hot words based on the calculation results;

[0032] The weights of the hot words will be readjusted, with the adjustment ratio being reduced.

[0033] The weight correction ends when the accuracy of the text inclusion of hot words reaches the accuracy threshold according to the preset ratio.

[0034] In some embodiments, the weighting dimensions of the hot words include: hot word tag category, hot word length, hot word popularity, hot word timeliness, recognition success rate, hot word triggering degree, hot word perplexity degree, hot word TF-IDF value, and number of hot word segments.

[0035] In some embodiments, the controller is further configured to:

[0036] Obtain a hot word list, wherein the hot words in the hot word list are assigned weights;

[0037] The weight of each character in the hot words is calculated based on the weight of the hot words;

[0038] A speech recognition path graph is constructed based on the weights of the hot words and individual characters.

[0039] Secondly, this application provides a hot word recognition method, which includes:

[0040] Obtain a hot word list, wherein the hot words in the hot word list are assigned weights;

[0041] The weight of each character in the hot words is calculated based on the weight of the hot words;

[0042] A speech recognition path graph is constructed based on the weights of the hot words and individual characters;

[0043] The probability of a candidate sequence for the input audio is calculated based on the path graph, wherein the candidate sequence includes multiple single words;

[0044] Based on the probability of a single character in the candidate sequence and the probability of the candidate sequence itself, one of the candidate sequences is determined as the text corresponding to the input audio.

[0045] In some embodiments, calculating the weight of each character in the hot words based on the weight of the hot words includes:

[0046] For each character in the hot word list, obtain the weight of the hot words containing that character;

[0047] Calculate the average weight of the hot words containing the given character to obtain the weight of the given character.

[0048] The beneficial effects of the display device and hot word recognition method provided in this application embodiment include:

[0049] This application embodiment enhances hot words by customizing the weights in the hot word list and performing hot word enhancement based on the custom weights. This effectively solves the problem that a unified fixed weight cannot flexibly handle the recognition of some hot words, improves the accuracy of hot word recognition, and is conducive to improving the accuracy of subsequent semantic understanding and enhancing the voice interaction effect. Attached Figure Description

[0050] To more clearly illustrate the implementation methods in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0051] Figure 1 The diagram above exemplarily illustrates a system architecture diagram of a speech recognition device according to some embodiments;

[0052] Figure 2 The diagram illustrates, by way of example, a hardware configuration block diagram of a smart device 200 according to some embodiments;

[0053] Figure 3 The diagram illustrates, by way of example, a hardware configuration block diagram of a smart device 200 according to some embodiments;

[0054] Figure 4 The diagram illustrates, by way of example, the logical architecture of a smart TV 200-1 according to some embodiments;

[0055] Figure 5 The diagram above exemplarily illustrates the structure of a hot word recognition and verification system according to some embodiments;

[0056] Figure 6 The diagram illustrates a flowchart of a method for generating test text according to some embodiments.

[0057] Figure 7 The diagram above illustrates a flowchart of a method for calculating the success rate of hot word recognition according to some embodiments;

[0058] Figure 8 The diagram above illustrates a flowchart of a hot word weight correction algorithm according to some embodiments;

[0059] Figure 9 The diagram above illustrates a flowchart of a hot word recognition method according to some embodiments;

[0060] Figure 10 The diagram illustrates a path for speech recognition according to some embodiments;

[0061] Figure 11The diagram illustrates, for example, the architecture of a speech recognition system according to some embodiments. Detailed Implementation

[0062] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0063] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0064] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0065] Figure 1 An exemplary system architecture for which the speech recognition method and speech recognition device of this application can be applied is shown. Figure 1 As shown, 10 is a server and 200 is a terminal device, including (smart TV 200a, mobile device 200b, smart speaker 200c).

[0066] In this application, server 10 and terminal device 200 communicate data through various communication methods. Terminal device 200 can be connected via local area network (LAN), wireless local area network (WLAN), and other networks. Server 10 can provide various content and interactive features to terminal device 200. For example, terminal device 200 and server 10 can send and receive information, and receive software updates.

[0067] Server 10 can be a server that provides various services, such as a backend server that supports audio data collected by terminal device 200. The backend server can analyze and process the received audio and other data, and feed back the processing results (such as endpoint information) to the terminal device. Server 10 can be a server cluster or multiple server clusters, and can include one or more types of servers.

[0068] Terminal device 200 can be either hardware or software. When terminal device 200 is hardware, it can be various electronic devices with sound acquisition capabilities, including but not limited to smart speakers, smartphones, televisions, tablets, e-book readers, smartwatches, media players, computers, AI devices, robots, smart vehicles, etc. When terminal devices 200, 201, and 202 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., used to provide sound acquisition services) or as a single software program or software module. No specific limitations are made here.

[0069] In some embodiments, the hot word recognition method provided in this application can be executed by server 10.

[0070] Figure 2 A hardware configuration block diagram of a smart device 200 according to an exemplary embodiment is shown. For example... Figure 2 The smart device 200 shown includes at least one of the following: a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. The controller includes a central processing unit, an audio processor, RAM, ROM, and a first to an nth interface for input / output.

[0071] The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The smart device 200 can establish the transmission and reception of control signals and data signals through the communicator 220 and the server 10.

[0072] The user interface can be used to receive external control signals.

[0073] Detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0074] The sound acquisition device can be a microphone, also known as a "microphone" or "voice transducer," which can be used to receive the user's voice and convert the sound signal into an electrical signal. The smart device 200 can be equipped with at least one microphone. In some embodiments, the smart device 200 can be equipped with two microphones, which, in addition to acquiring sound signals, can also perform noise reduction. In other embodiments, the smart device 200 can also be equipped with three, four, or more microphones, enabling sound signal acquisition, noise reduction, sound source identification, and directional recording functions, etc.

[0075] Furthermore, the microphone can be built into the smart device 200, or it can be connected to the smart device 200 via wired or wireless means. Of course, this embodiment does not limit the location of the microphone on the smart device 200. Alternatively, the smart device 200 may not include a microphone, meaning the microphone is not located within the smart device 200. The smart device 200 can connect an external microphone (also called a microphone) via an interface (such as USB interface 130). This external microphone can be fixed to the smart device 200 using an external fastener (such as a camera bracket with a clip).

[0076] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the intelligent device 200.

[0077] For example, the controller includes at least one of a central processing unit (CPU), an audio processor, RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0078] In some examples, the smart device uses the Android operating system, such as... Figure 3 As shown, the smart TV 200-1 can be logically divided into an application layer (referred to as the "application layer") 21, a kernel layer 22, and a hardware layer 23.

[0079] Among them, such as Figure 3 As shown, the hardware layer may include Figure 2 The controller 250, communicator 220, detector 230, etc., are shown. Application layer 21 includes one or more applications. These applications can be system applications or third-party applications. For example, application layer 21 includes a voice recognition application, which can provide a voice interaction interface and services to enable the connection between the smart TV 200-1 and the server 10.

[0080] The kernel layer 22 serves as a software middleware between the hardware layer and the application layer 21, and is used to manage and control hardware and software resources.

[0081] In some examples, kernel layer 22 includes a detector driver that sends voice data collected by detector 230 to a speech recognition application. For instance, when the speech recognition application in smart device 200 starts and a communication connection is established between smart device 200 and server 10, the detector driver sends user-input voice data collected by detector 230 to the speech recognition application. The speech recognition application then sends query information containing this voice data to intent recognition module 202 in the server. Intent recognition module 202 inputs the voice data sent by smart device 200 into the intent recognition model.

[0082] To clearly illustrate the embodiments of this application, the following description is provided in conjunction with... Figure 4 This application describes a speech recognition network architecture provided in its embodiments.

[0083] See Figure 4 , Figure 4 This is a schematic diagram of a voice interaction network architecture provided in an embodiment of this application. Figure 4 In this system, the intelligent device receives input information and outputs the processing results of that information. The speech recognition module deploys a speech recognition service to convert audio into text; the semantic understanding module deploys a semantic understanding service to perform semantic parsing on the text; the business management module deploys a business instruction management service to provide business instructions; the language generation module deploys a language generation service (NLG) to convert instructions instructing the intelligent device to execute into text language; and the speech synthesis module deploys a text-to-speech (TTS) service to process the text language corresponding to the instructions and send it to a speaker for playback. In one embodiment, Figure 4 The architecture shown can contain multiple entity service devices with different business services deployed, or one or more entity service devices can combine one or more functional services.

[0084] In some embodiments, the following describes the basis Figure 4 The process of processing input information from smart devices in the architecture shown is illustrated with an example, taking a query statement input via voice as an example:

[0085] [Speech Recognition]

[0086] After receiving a query statement input via voice, the smart device can perform noise reduction and feature extraction on the audio of the query statement. The noise reduction process may include steps such as removing echoes and environmental noise.

[0087] [Semantic understanding]

[0088] Using acoustic and language models, natural language understanding is performed on the identified candidate text and associated contextual information. The text is parsed into structured, machine-readable information, including business domain, intent, slots, and other semantic information. An executable intent confidence score is obtained, and the semantic understanding module selects one or more candidate executable intents based on the determined intent confidence score.

[0089] [Business Management]

[0090] Based on the semantic parsing results of the query statement text, the semantic understanding module sends query instructions to the corresponding business management module to obtain the query results provided by the business service, as well as the actions required to "complete" the user's final request, and feeds back the device execution instructions corresponding to the query results.

[0091] [Language Generation]

[0092] Natural Language Generation (NLG) is configured to generate spoken text from information or instructions. Specifically, it can be categorized into casual conversation, task-oriented, knowledge-based question-answering, and recommendation-based systems. In casual conversation, NLG performs intent recognition and sentiment analysis based on context to generate open-ended responses. In task-oriented conversations, learned strategies are used to generate responses, typically including clarifying needs, guiding the user, asking questions, confirming, and closing remarks. In knowledge-based question-answering conversations, the required knowledge (knowledge, entities, fragments, etc.) is generated based on question type identification and classification, information retrieval, or text matching. In recommendation-based conversation systems, user interests are matched, candidate content is ranked, and then recommended content is generated for the user.

[0093] [Speech Synthesis]

[0094] The speech synthesis module is configured to present voice output to the user. It synthesizes voice output based on text provided by the digital assistant. For example, the generated dialogue response is in the form of a text string. The speech synthesis module then converts the text string into audible voice output.

[0095] It should be noted that, Figure 4 The architecture shown is merely an example and is not intended to limit the scope of protection of this application. Other architectures can also be used to achieve similar functions in the embodiments of this application. For example, all or part of the above process can be completed by a smart terminal, which will not be elaborated here.

[0096] In some embodiments, in order to accurately identify hot words in voice commands, this application constructs a hot word list and a hot word recognition verification system to identify and verify the hot words in the hot word list, and removes hot words with low recognition success rates to ensure the hot word recognition effect; this application also constructs a hot word weight generation system to set weights for the hot words in the hot word list, so that during speech recognition, hot word enhancement can be performed based on the weights to improve the hot word recognition efficiency.

[0097] See Figure 5 The image shows a schematic diagram of a hot word recognition and verification system provided in an embodiment of this application. Figure 5 As shown, the hot word recognition and verification system provided in this application embodiment includes a language generation module, a language synthesis module, an audio noise addition module, a speech recognition module, and a recognition and judgment module connected in sequence.

[0098] In some embodiments, the language generation module is used to generate test text containing hot words. The method for generating the test text can be found in [reference needed]. Figure 6 It includes the following steps:

[0099] Step S101: Obtain source data for generating test text.

[0100] In some embodiments, the source data may be the text corresponding to the voice command and the corresponding tag word pair sequence. During the voice interaction, the display device performs speech recognition on the voice command to obtain the user query, sends the user query to the server, the server identifies the tag word pair sequence from the user query, and generates an online log of this voice interaction. The online log includes the user query and the tag word pair sequence, which contains entity words and their tags in the user query.

[0101] For example, a user question might be: "I want to watch TV series B starring actor A." The tag pair sequence for this user question would be: [actor A:actor, TV series B:videoName], where actor A and TV series B are entity words in the user question, actor is the tag for actor A, and videoName is the tag for TV series B.

[0102] The language generation module can extract user questions and tag word pair sequences from online log requests by pulling a large number of online log requests.

[0103] Step S102: Construct a question template based on the source data.

[0104] In some embodiments, to enrich the test text, multiple question templates can be constructed based on the source data. These question templates are used to generate new user questions. The method for constructing question templates may include replacing entity words in the user questions with their corresponding tags.

[0105] For example, for the user's question: I want to watch TV series B with actor A, based on the entity word "actor A" with the tag "actor", the following question template is generated: I want to watch TV series B with actor A; based on the entity word "TV series B" with the tag "videoName", the following question template is generated: I want to watch videoname with actor A.

[0106] For the user's question: "Play the fifth episode of TV series C", based on the tag "videoName" for the entity word "TV series C", the following question template is generated: "Play the fifth episode of videoName".

[0107] Among them, "I want to watch actor A's videoname" and "Play the fifth episode of videoName" both match the tag videoName.

[0108] Step S103: Obtain the question templates for matching hot words in the hot word list, and generate test text for testing the recognition accuracy of hot words based on the hot words and question templates.

[0109] In some embodiments, each hot word in the hot word list is tagged, and a question version can be matched for each hot word based on the hot word's tag. After matching the question template, the hot word and the question template can be combined to form test text.

[0110] For example, the tag for the hot word "TV series D" in the hot word list is videoName. Based on videoName, the following question template can be matched: I want to watch actor A's videoname, play episode 5 of videoname. Replacing the tag in the question template with the hot word generates the following test text: I want to watch actor A's TV series D, play episode 5 of TV series D.

[0111] It should be noted that when generating test text, it is not necessary to judge the accuracy of the knowledge in the test text, as long as the semantic structure is correct. For example, it is not necessary to judge whether actor A has starred in TV series C, as long as the semantic structure of "I want to watch actor A's TV series C" is correct.

[0112] In some embodiments, the language generation module is used to convert test text into test audio. The language generation module can call the multi-speaker TTS API service to generate multiple audio clips from different speakers for each test text generated by the language generation module, resulting in multiple test audio clips.

[0113] In some embodiments, the audio noise-adding module is used to preprocess the test audio to generate more test audio tracks. The audio noise-adding module can use the Kaldi tool to process the test audio, such as adjusting the speech rate, volume, and noise level of the test audio. Adjusting the noise level of the test audio may include adding noise to the test audio. Through the above preprocessing, multiple different audio tracks can be generated based on a single test audio track.

[0114] In some embodiments, the speech recognition module is used to invoke a speech recognition service without hot word enhancement to perform speech recognition on the test audio and generate recognized text.

[0115] In some embodiments, the identification and judgment module is used to verify the hot word recognition effect of the test audio in a conventional speech recognition scenario.

[0116] After generating the text to be recognized, the speech recognition module can calculate the recognition success rate for each hot word in the hot word list. The steps for calculating the recognition success rate can be found in [link to documentation]. Figure 7 It includes the following steps:

[0117] Step S201: Calculate the text inclusion accuracy of each hot word in the hot word list.

[0118] In some embodiments, for a hot word, text inclusion accuracy refers to the ratio of the number of recognized texts containing that hot word to the number of test audios containing that hot word. Text inclusion accuracy can be used to determine the accuracy with which a hot word is recognized in test text, and its calculation formula is as follows:

[0119]

[0120] Where c_accn is the text inclusion accuracy, n is the number of test audios generated based on the hot word, and c is the number of recognized texts containing the hot word in the speech recognition results.

[0121] For example, there are 10 test audio clips corresponding to the hot word "TV series D". After performing speech recognition on these 10 test audio clips, 7 out of the 10 recognized texts contain the hot word "TV series D" and 3 do not contain the hot word. Therefore, the text inclusion accuracy of the hot word "TV series D" is 70%.

[0122] Step S202: Calculate the text equality accuracy of hot words.

[0123] In some embodiments, for a hot word, text equivalence accuracy refers to the ratio of the number of recognized texts containing that hot word that are exactly the same as the test texts containing that hot word to the number of test texts containing that hot word. Text equivalence accuracy can be used to determine whether test texts containing a hot word can be completely and accurately recognized, and its calculation formula is as follows:

[0124]

[0125] Where e_acc is the text equality accuracy, and e is the number of recognized texts containing the hot word that are exactly the same as the corresponding test text.

[0126] For example, the number of test audios corresponding to the hot word "TV series D" is 10. After performing speech recognition on these 10 test audios, 6 of the 10 recognized texts are exactly the same as the corresponding test text, 1 contains the hot word but the recognized text is different from the test text, and 3 do not contain the hot word. Therefore, the text equality accuracy of the hot word "TV series D" is 60%.

[0127] Step S203: Calculate the recognition success rate of hot words based on the text inclusion accuracy and text equality accuracy.

[0128] In some embodiments, for a hot word, the recognition success rate can be determined by combining text inclusion accuracy and text equality accuracy, and the calculation formula is as follows:

[0129] acc = α × c_acc + (1-α) × e_acc

[0130] Where acc is the recognition success rate, and α is a hyperparameter that can be set to a value greater than 0.8 and less than 1.

[0131] pass Figure 7 After obtaining the recognition success rate of each hot word in the hot word list using the method shown, hot words with recognition success rates lower than the success rate threshold can be deleted from the hot word list to ensure that the hot word list has a high overall recognition success rate.

[0132] In some embodiments, a hot word weighting generation system can be used to set weights for hot words in the hot word list, thereby enhancing hot words based on weights during speech recognition and improving the efficiency of hot word recognition.

[0133] In some embodiments, the CART tree regression algorithm, a machine learning method, can be used to fit the weights of hot words. The training data can be in libsvm format, such as: "TV series D#1:3,2:0.6,2:0.6,...,12:0.8#0.6", constructing multi-dimensional features for hot words to calculate their weights. The features include: hot word tag category, hot word length, hot word popularity, hot word timeliness, recognition success rate, hot word triggering degree, hot word perplexity degree, hot word TF-IDF value, and number of hot word segments.

[0134] In some embodiments, hot word tag categories may include categories such as film and television, automobiles, face recognition, counting, calorie calculation, plants, and animals. Different weights can be assigned to tags in different categories. For example, film and television and automobile hot words are frequently used in voice interaction, so film and television and automobile tags can be assigned relatively high weights, while other categories can be assigned relatively low weights. One-hot encoding can be used to set hot word tag categories, and the feature dimension of a hot word tag category is the number of tag categories.

[0135] In some embodiments, hot words with more characters are generally more difficult to fully identify. The feature dimension of hot word length can be set to 1. Based on the maximum hot word length of 10, the number of characters of each hot word in the hot word table is normalized to obtain the hot word length. For hot words with more than 10 characters, their hot word length is set to 1.

[0136] In some embodiments, the popularity of trending words can be obtained based on trending search data or ratings and popularity data from relevant websites. The feature dimension of trending word popularity can be set to two, including rating and popularity, both of which need to be normalized.

[0137] In some embodiments, the newer the trending words, the more attention they deserve. The number of days a trending word has appeared is discretized using 3 / 10 / 30 days, with a feature dimension of 4. Example: the feature vector for less than 3 days is 1,0,0,0, and the feature vector for more than 30 days is 0,0,0,1.

[0138] In some embodiments, the recognition success rate can be the recognition success rate of hot words output by the hot word recognition verification system, with a feature dimension of 1.

[0139] In some embodiments, the triggering frequency of hot words can be set according to the frequency of triggering hot words in the past 7 days, 15 days, 30 days, 60 days, 180 days and all historical hot words, with a feature dimension of 6, which focuses more on the weight update of existing hot words.

[0140] In some embodiments, the greater the perplexity of a hot word, the less coherent its semantics, the more difficult it is to identify, and the more it needs to be enhanced. Based on the output ppl of the pre-trained model, and discretized according to 5 / 10 / 20, the perplexity of the hot word can be obtained, with a feature dimension of 4.

[0141] In some embodiments, the tf-idf values ​​of hot words are the average tf-idf value, the sum of tf-idf values ​​after hot word segmentation, and the feature dimension of the hot word tf-idf value is 2.

[0142] In some embodiments, the number of hot word segments can be obtained by segmenting hot words using jieba and then normalizing them according to a maximum number of 5. The feature dimension of the number of hot word segments is 1.

[0143] In some embodiments, for each hot word in the hot word list, after setting the feature vector of the hot word according to the above hot word features, machine learning training is performed on the feature vector to obtain the hot word weight.

[0144] In some embodiments, to improve the accuracy of hot word recognition, the hot word weights can be corrected to improve the accuracy of hot word recognition based on the hot word weights. The correction method includes: replacing the speech recognition module used in the hot word recognition verification system with a speech recognition module loaded with the latest hot word list, re-performing speech recognition on the test audio, and correcting the hot word weights based on the recognition results. The hot words in the latest hot word list are assigned weights. The correction algorithm can be found in [reference needed]. Figure 8 It includes the following steps:

[0145] Step S301: Based on the text inclusion accuracy of the hot words, set the hot words in the hot word list to either the first category label or the second category label.

[0146] In some embodiments, based on the hot word recognition results of the hot word recognition verification system, hot words containing accuracy rates less than an accuracy threshold are assigned a first category label, and hot words containing accuracy rates greater than or equal to a preset accuracy rate are assigned a second category label. The accuracy threshold can be set according to actual needs; for example, the accuracy threshold can be set to 98%.

[0147] Step S302: Increase the weight of the first type of hot words by the first ratio, and decrease the weight of the second type of hot words by the second ratio.

[0148] In some embodiments, the first ratio and the second ratio may be the same, such as both being set to 5%.

[0149] In some embodiments, the first ratio and the second ratio may be different; for example, the first ratio may be greater than the second ratio or less than the second ratio.

[0150] Step S303: Recalculate the text inclusion accuracy of hot words and update the category labels of hot words based on the calculation results.

[0151] In some embodiments, after adjusting the weights of hot words, the text inclusion accuracy of the hot words can be recalculated, and it can be determined whether the text inclusion accuracy of the hot words is less than the accuracy threshold based on the calculation result, and the category label of the hot words can be updated. For example, after the weight adjustment, if the text inclusion accuracy of a hot word is updated from 95% to 98%, then the category label of the hot word is updated from the second category label to the first category label.

[0152] Step S304: Readjust the weight of hot words, wherein the adjustment ratio is reduced.

[0153] In some embodiments, to further improve the success rate of hot word recognition, the weight of hot words can be adjusted. To prevent the adjustment from being too large and causing a decrease in the success rate of hot word recognition, the adjustment ratio can be reduced.

[0154] In some embodiments, the first ratio and the second ratio may be attenuated at the same rate, for example, the first ratio and the second ratio may be attenuated at a rate of 0.5.

[0155] In some embodiments, the first ratio and the second ratio may be attenuated at different ratios, wherein the attenuation ratio of the first ratio may be greater than the attenuation ratio of the second ratio, or the attenuation ratio of the first ratio may be less than the attenuation ratio of the second ratio.

[0156] Step S305: When the text inclusion accuracy of hot words reaches the accuracy threshold according to the preset ratio, the weight correction ends.

[0157] In some embodiments, when the text inclusion accuracy of a preset proportion of hot words reaches an accuracy threshold, it indicates that the recognition results of the hot word list have met the recognition accuracy requirements, and the weight correction can be terminated. The preset proportion can be set according to actual needs. For example, the preset proportion can be set to 98%.

[0158] Based on the hot word recognition and verification system and hot word weight generation system in the above embodiments, after setting and updating weights for hot words in the hot word table, the system can enhance the recognition of user voice commands during voice interaction based on the weights of the hot words, thereby improving the accuracy of voice recognition. See also Figure 9 The hot word identification method may include the following steps:

[0159] Step S401: Obtain the hot word list, wherein the hot words in the hot word list are assigned weights.

[0160] In some embodiments, the original hot word table can be processed by the hot word recognition and verification system and the hot word weight generation system described above to obtain a hot word table in which each hot word is assigned a weight.

[0161] Step S402: Calculate the weight of each character in the hot words based on the weight of the hot words.

[0162] In some embodiments, after obtaining a list of hot words with weights, the weight of each character in the hot words can be calculated based on the weights of the hot words. The calculation formula is as follows:

[0163]

[0164] In the above formula, c i Let v be the weight of the i-th character in the hot word list. k For hot words w k The weight, For the containing word c i A collection of hot words, Represents a set The size of the set The number of hot words in the text.

[0165] For example, a hot word list is {Hisense:3, Neptune:2}, indicating that the hot word list contains two hot words: Hisense and Neptune. Hisense has a weight of 3 and Neptune has a weight of 2. According to the formula for calculating the weight of a single character, the following character weight list can be obtained: {Sea:2.5, Letter:3, King:2, Star:2}. This character weight list indicates that the character weight of "Sea" is 2.5, the character weight of "Letter" is 3, the character weight of "King" is 2, and the character weight of "Star" is 2.

[0166] Step S403: Construct a speech recognition path graph based on the weights of hot words and individual characters.

[0167] In some embodiments, after obtaining the weights of individual characters in the hot word list, a speech recognition path graph can be constructed, such as a WFST (weighted finite-state transducer) path graph. The construction method includes: obtaining all hot words whose starting character is the same Chinese character; for one of the obtained hot words, if the hot word contains two characters, setting a zero node and a first node, the zero node being the starting node and the ending node, the weight of the routing path from the zero node to the first node being the weight of the first character of the hot word, the weight of the backtracking path from the first node to the zero node being the negative of the weight of the first character of the hot word, and the weight of the routing path from the first node to the zero node being the weight of the second character of the hot word; if The hot word contains three characters. A zero node, a first node, and a second node are set. The zero node is the start node and the end node. The path weight from the zero node to the first node is the weight of the first character of the hot word. The weight of the route path from the first node to the second node is the weight of the second character of the hot word. The weight of the backtrack path from the second node to the zero node is the negative of the sum of the weights of the first and second characters of the hot word. The weight of the route path from the second node to the zero node is the weight of the third character of the hot word. And so on, until the path graph contains a synchronous hot word with the same starting character.

[0168] For example, taking the hot word list as {Hisense:3, Neptune:2}, the constructed path diagram can be found in [reference needed]. Figure 10 , Figure 10 In the diagram, 0 represents the zero node, 1 represents the first node, 2 represents the second node, and eps represents the weight of the backtrack path.

[0169] Step S404: Calculate the probability of candidate sequences of the input audio based on the path graph, wherein the candidate sequence includes multiple words.

[0170] In some embodiments, for an input audio, such as a voice command of a user, when performing speech recognition, multiple candidate sequences are output. Some of the candidate sequences may contain hotwords. By querying the path graph corresponding to the hotword table, the weight sum of each candidate sequence can be obtained. After normalizing the weight sum, the probability of the candidate sequence can be obtained.

[0171] Exemplarily, for an input audio, one candidate sequence is {Hai: 0.8, Xin: 0.6}, another candidate sequence is {Hai: 0.8, Wang: 0.5}, and another candidate sequence is {Hai: 0.8, Yang: 0.4}. Among them, the hotword table is {Hisense: 3, Neptune: 2}. It can be seen that "ocean" is not a hotword in the hotword table. By querying the path graph, the weight sum corresponding to {Hai: 0.8, Xin: 0.6} can be output as: 2.5 + 3 = 5.5, the weight sum corresponding to {Hai: 0.8, Wang: 0.5} is: 2.5 + 2 = 4.5, and the weight sum corresponding to {Hai: 0.8, Yang: 0.4} is: 2.5 - 2.5 = 0. After normalizing the above weight sums to the range of 0 to 1, the probability of {Hai: 0.8, Xin: 0.6} can be obtained as: 5.5 / (5.5 + 4.5) = 0.55, the probability of {Hai: 0.8, Wang: 0.5} is 4.5 / (5.5 + 4.5) = 0.45, and the probability of {Hai: 0.8, Yang: 0.4} is 0 / (5.5 + 4.5) = 0.

[0172] Step S405: Determine one of the candidate sequences as the text corresponding to the input audio according to the single - word probability in the candidate sequence and the probability of the candidate sequence.

[0173] In some embodiments, the single - word probability in the candidate sequence and the probability of the candidate sequence can be comprehensively considered to obtain the prediction probability of the final candidate sequence. The candidate sequence with the maximum prediction probability is determined as the text corresponding to the input audio, that is, the semantic text, realizing the accurate recognition of hotwords in the candidate sequence. After obtaining the semantic text, semantic understanding can be performed on the semantic text to obtain the semantic understanding result. The response result corresponding to the semantic understanding result is obtained through the service management module, and the display device responds according to the response result to achieve voice interaction with the user.

[0174] Among them, the method of comprehensively considering the single - word probability in the candidate sequence and the probability of the candidate sequence can be the attention - based rescoring method, or other methods, such as the average - taking method or the weighted calculation method, etc.

[0175] It should be noted that Figures 6 to 9 The method shown can be executed by the controller of the display device, or by the server, or by the display device and the server respectively executing some steps. The embodiments of the present application do not make specific limitations on this.

[0176] based on Figure 9 The hot word recognition method shown in this application also illustrates a speech recognition system, the architecture of which can be found in [reference needed]. Figure 11 It includes a feature extractor, a shared encoder, a CTC decoder, a hot word enhancement module, a rescoring module, and a text processing and output module connected in sequence.

[0177] In some embodiments, a feature extractor is used to extract features from the input audio to obtain a feature vector, wherein the input audio may be a user's voice command.

[0178] In some embodiments, a shared encoder is used to encode the feature vector output by the feature extractor to obtain an encoding result.

[0179] In some embodiments, the CTC decoder is used to decode the encoding result of the shared encoder to obtain a decoding result, which includes the aforementioned candidate sequence. The candidate sequence includes single characters and single character probabilities obtained sequentially after feature extraction, encoding, and decoding of the input audio.

[0180] In some embodiments, the hot word enhancement module is used to query the WFST path graph to obtain the probability of each candidate sequence.

[0181] In some embodiments, the rescoring module is used to rescore based on the probability of the candidate sequence and the probability of the single character in the candidate sequence to obtain the predicted probability of each candidate sequence.

[0182] In some embodiments, the text processing and output module is used to filter out the candidate sequence with the highest predicted probability, combine the individual characters in the candidate sequence into text, and obtain the output text of the input audio.

[0183] As can be seen from the above embodiments, this application embodiment effectively solves the problem that a unified fixed weight cannot flexibly handle the recognition of some hot words by customizing the weights in the hot word table and enhancing the hot words based on the custom weights, thereby improving the accuracy of hot word recognition; this application embodiment constructs a hot word recognition verification system, which improves the speech recognition system's ability to recognize newly added hot words; this application embodiment constructs an automated hot word weight generation system, which adds hot words in advance, solving the problem of not being able to perceive and process hot words in advance.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

[0185] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, include: monitor; The controller, which is communicatively connected to the display, is configured to: Receive voice commands input by the user; Obtain the candidate sequence corresponding to the voice command, wherein the candidate sequence contains multiple single characters and the probability of the single character; The probability of the candidate sequence is calculated based on the speech recognition path graph with custom weights. The semantic text corresponding to the voice command is determined based on the probability of the candidate sequence and the probability of the single character. Generate a response result based on the semantic text; Respond according to the stated response result.

2. The display device according to claim 1, characterized in that, The step of calculating the probability of the candidate sequence based on a speech recognition path graph with custom weights includes: Query the path graph corresponding to the hot word table to obtain the weight sum of each candidate sequence; The weights are then normalized to obtain the probability of the candidate sequence.

3. The display device according to claim 1, characterized in that, Determining the semantic text corresponding to the voice instruction based on the probability of the candidate sequence and the probability of the single character includes: Based on the probability of the candidate sequence and the probability of the single character, an attention-based rescoring is performed to obtain the predicted probability of the candidate sequence. The semantic text corresponding to the voice instruction is generated based on the candidate sequence with the highest predicted probability.

4. The display device according to claim 1, characterized in that, Before receiving the voice command, the controller is also configured to: Obtain the source data used to generate the test text; Construct a question template based on the source data; Obtain the question templates for matching hot words in the hot word list, and generate test text based on the hot words and question templates to test the recognition success rate of the hot words.

5. The display device according to claim 1, characterized in that, Before receiving the voice command, the controller is also configured to: For each hot word in the hot word list, the ratio of the number of recognized texts containing the hot word to the number of test audios containing the hot word is calculated to obtain the text inclusion accuracy. For the hot words, the ratio of the number of recognized texts containing the hot words that are exactly the same as the test texts containing the hot words to the number of test texts containing the hot words is calculated to obtain the text equality accuracy. The recognition success rate of the hot words is calculated based on the text inclusion accuracy and text equality accuracy. Remove hot words whose recognition success rate is lower than the success rate threshold from the hot word list.

6. The display device according to claim 1, characterized in that, Before receiving the voice command, the controller is also configured to: In the hot word list, hot words whose text contains an accuracy rate less than an accuracy rate threshold are assigned a first category label, and hot words whose text contains an accuracy rate greater than or equal to the accuracy rate threshold are assigned a second category label. The text containing accuracy rate is the ratio of the number of recognized texts containing the hot word in the hot word list to the number of test audios containing the hot word. The weight of the first category of hot words is increased by a first ratio, and the weight of the second category of hot words is decreased by a second ratio. The first category of hot words are hot words with the first category identifier, and the second category of hot words are hot words with the second category identifier. Recalculate the text inclusion accuracy of the hot words and update the category identifier of the hot words based on the calculation results; The weights of the hot words will be readjusted, with the adjustment ratio being reduced. The weight correction ends when the accuracy of the text inclusion of hot words reaches the accuracy threshold according to the preset ratio.

7. The display device according to claim 6, characterized in that, The weighting dimensions of the hot words include: hot word tag category, hot word length, hot word popularity, hot word timeliness, recognition success rate, hot word triggering degree, hot word perplexity degree, hot word TF-IDF value, and number of hot word segments.

8. The display device according to claim 6, characterized in that, Before receiving the voice command, the controller is also configured to: Obtain a hot word list, wherein the hot words in the hot word list are assigned weights; The weight of each character in the hot words is calculated based on the weight of the hot words; A speech recognition path graph is constructed based on the weights of the hot words and individual characters.

9. A hot word recognition method, characterized in that, include: Obtain a hot word list, wherein the hot words in the hot word list are assigned weights; The weight of each character in the hot words is calculated based on the weight of the hot words; A speech recognition path graph is constructed based on the weights of the hot words and individual characters; The probability of a candidate sequence for the input audio is calculated based on the path graph, wherein the candidate sequence includes multiple single words; Based on the probability of a single character in the candidate sequence and the probability of the candidate sequence itself, one of the candidate sequences is determined as the text corresponding to the input audio.

10. The hot word recognition method according to claim 9, characterized in that, The step of calculating the weight of each character in the hot words based on their weights includes: For each character in the hot word list, obtain the weight of the hot words containing that character; Calculate the average weight of the hot words containing the given character to obtain the weight of the given character.

Images