Speech recognition method and related device

Abstract

The application discloses a speech recognition method and related equipment, and relates to the technical field of data processing. The method comprises the following steps: in response to a speech recognition instruction, obtaining to-be-recognized speech and a preset hot word library; and using a preset ASR model to generate a speech recognition result based on the to-be-recognized speech and the preset hot word library. Before speech recognition, the application determines a topic distribution, a word distribution of a topic and a distinguishing degree corresponding to a word based on a preset LDA model and a preset IDF algorithm, and then generates high-quality hot words and the corresponding weight of each hot word. In the process of speech recognition, a more accurate speech recognition result is obtained based on the high-quality hot words.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to speech recognition methods and related equipment. Background Technology

[0002] In related technologies, Automatic Speech Recognition (ASR) technology has significantly improved its recognition accuracy in general domains. However, in specific business scenarios such as finance, healthcare, and government affairs, there are still problems with inaccurate recognition of professional terms, obscure words, and new words. Existing methods usually guide the recognition results by injecting "hot words" into the ASR model, but the quality of hot words directly affects the recognition effect. Current hot word generation methods mostly rely on manual experience or simple statistics, lacking effective cleaning of the corpus and semantic modeling. This can easily lead to hot word redundancy, weight imbalance, and even the introduction of noise, thereby affecting the recognition accuracy. Summary of the Invention

[0003] The main purpose of this application is to provide a speech recognition method and related equipment, which aims to solve the technical problem of how to improve the accuracy of hot words, thereby improving the accuracy of speech recognition.

[0004] To achieve the above objectives, this application proposes a speech recognition method, which includes:

[0005] In response to a speech recognition command, the system acquires the speech to be recognized and a preset hot word library. The preset hot word library contains multiple hot words and a weight corresponding to each hot word. The hot words and the weights are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. Based on the speech to be recognized and the preset hot word library, a preset ASR model is used to generate speech recognition results.

[0006] In one embodiment, before the step of acquiring the speech to be recognized and the preset hot word library, the method further includes: Obtain multiple corpus documents in different formats; The multiple corpus documents in different formats are preprocessed to obtain multiple sets of structured, standard text content; Based on the multiple sets of structured, standard text content, the preset LDA model, and the preset IDF algorithm, multiple hot words and the corresponding weights for each hot word are generated; Based on the hot words and the weights, a preset hot word library is constructed.

[0007] In one embodiment, the step of preprocessing the plurality of corpus documents in different formats to obtain multiple sets of structured, standard text content further includes: Based on file hashing, a document deduplication operation is performed on the multiple corpus documents of different formats to obtain multiple deduplicated corpus documents; Noise processing and text extraction operations are sequentially performed on the multiple deduplicated corpus documents to obtain multiple sets of text content; Based on MinHash and LSH, semantic deduplication is performed on the multiple sets of text content to obtain multiple sets of semantically deduplicated text content. Perform text standardization on the multiple sets of semantically deduplicated text content to obtain multiple sets of structured, standard text content.

[0008] In one embodiment, the step of performing semantic deduplication on the multiple sets of text content based on MinHash and LSH to obtain multiple sets of semantically deduplicated text content further includes: Based on the multiple sets of text content, a bag-of-words model is constructed; Based on the bag-of-words model, calculate the MinHash signature corresponding to each group of text content; Based on LSH and the MinHash signature, similar text content is identified among the multiple sets of text content, and the similar text content is removed to obtain multiple sets of semantically deduplicated text content.

[0009] In one embodiment, the step of generating multiple hot words and the corresponding weights for each hot word based on the multiple sets of structured, standard text content, the preset LDA model, and the preset IDF algorithm further includes: The pre-defined LDA model is used to process the multiple sets of structured, standard text content to obtain the topic distribution and the word distribution of the topics; Based on the aforementioned multiple sets of structured, standard text content, the discriminative power of each word is calculated using a preset IDF algorithm. Based on a preset adjustment coefficient, the discrimination score and the word distribution of the topic are weighted and summed to generate multiple hot words corresponding to different topics and the weight of each hot word.

[0010] In one embodiment, the step of processing the multiple sets of structured, standard text content using a preset LDA model to obtain topic distribution and topic word distribution further includes: Based on jieba and a preset custom dictionary, Chinese and English word segmentation, stemming, and word form restoration operations are performed on the multiple sets of structured and standard text content to construct a bag of words; Based on the bag of words, the preset LDA model is iteratively trained until the topic allocation corresponding to the preset LDA model tends to stabilize, thereby obtaining the topic distribution and the word distribution of the topic. The iterative training is implemented based on Gibbs sampling, which is used to converge the topic distribution results and the word distribution results of the topic.

[0011] In one embodiment, the step of calculating the discriminative power of words based on the multiple sets of structured, standard text content using a preset IDF algorithm further includes: Determine the frequency of occurrence of each word in the multiple sets of structured, standard text content; Based on the frequency of occurrence and the preset IDF algorithm, the distinguishability of the words is calculated.

[0012] Furthermore, to achieve the above objectives, this application also proposes a speech recognition device, the speech recognition device comprising: The acquisition module is used to acquire the speech to be recognized and a preset hot word library in response to a speech recognition command. The preset hot word library contains multiple hot words and the weight corresponding to each hot word. The hot words and the weight are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. A generation module is used to generate speech recognition results based on the speech to be recognized and the preset hot word library using a preset ASR model.

[0013] In addition, to achieve the above objectives, this application also proposes a speech recognition device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the speech recognition method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the speech recognition method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the speech recognition method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This application discloses a speech recognition method and related equipment, relating to the field of data processing technology. Among related technologies, Automatic Speech Recognition (ASR) technology has significantly improved its recognition accuracy in general fields. However, in specific business scenarios such as finance, healthcare, and government affairs, there are still problems with inaccurate recognition of professional terms, obscure words, and new words. Existing methods typically guide the recognition results by injecting "hot words" into the ASR model, but the quality of the hot words directly affects the recognition effect. Current methods for generating hot words often rely on manual experience or simple statistics, lacking effective corpus cleaning and semantic modeling. This can easily lead to redundant hot words, weight imbalances, and even the introduction of noise, thus affecting recognition accuracy. In contrast, in this application, firstly, in response to a speech recognition command, the speech to be recognized and a preset hot word library are acquired. The preset hot word library contains multiple hot words and the weight corresponding to each hot word. The hot words and the weights are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. Then, based on the speech to be recognized and the preset hot word library, a preset ASR model is used to generate speech recognition results.

[0017] Understandably, before performing speech recognition, this application determines the topic distribution, the word distribution of the topic, and the discriminative power of the corresponding words based on a preset LDA model and a preset IDF algorithm, thereby generating high-quality hot words and generating a weight for each hot word. During the speech recognition process, more accurate speech recognition results are obtained based on the high-quality hot words. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the speech recognition method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the speech recognition method of this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the speech recognition method of this application; Figure 4 This is a schematic diagram of the module structure of the speech recognition device according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the speech recognition method in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution in this application's embodiments is: In this embodiment, for ease of description, the following description uses a speech recognition device as the execution subject.

[0025] While Automatic Speech Recognition (ASR) technology has significantly improved accuracy in general domains, it still faces challenges in specific business scenarios such as finance, healthcare, and government, particularly in recognizing specialized terminology, obscure words, and novel terms. Existing methods typically guide recognition results by injecting "hot words" into the ASR model, but the quality of these hot words directly impacts recognition performance. Current hot word generation methods largely rely on manual experience or simple statistics, lacking effective corpus cleaning and semantic modeling. This can easily lead to hot word redundancy, weight imbalance, and even the introduction of noise, thus affecting recognition accuracy.

[0026] This application provides a solution in which: first, in response to a speech recognition command, the speech to be recognized and a preset hot word library are acquired. The preset hot word library contains multiple hot words and a weight corresponding to each hot word. The hot words and the weights are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. Then, based on the speech to be recognized and the preset hot word library, a preset ASR model is used to generate a speech recognition result.

[0027] Understandably, before performing speech recognition, this application determines the topic distribution, the word distribution of the topic, and the discriminative power of the corresponding words based on a preset LDA model and a preset IDF algorithm, thereby generating high-quality hot words and generating a weight for each hot word. During the speech recognition process, more accurate speech recognition results are obtained based on the high-quality hot words.

[0028] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or voice recognition device capable of performing the above functions. The following description uses a voice recognition device as an example to illustrate this embodiment and the subsequent embodiments.

[0029] Based on this, the embodiments of this application provide a speech recognition method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the speech recognition method of this application.

[0030] In this embodiment, the speech recognition method includes steps S10 to S40: Step S10: In response to a speech recognition command, obtain the speech to be recognized and a preset hot word library. The preset hot word library contains multiple hot words and the weight corresponding to each hot word. The hot words and the weight are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. It should be noted that a voice recognition command is a trigger signal that initiates the voice recognition process. It can be a command actively issued by the user (such as clicking a button or using a voice wake-up word) or a call command automatically generated by the system. Its function is to start the entire voice recognition process.

[0031] It should be noted that the speech to be recognized refers to the raw audio data input by the user, which contains the speech content that needs to be converted into text. It is the direct processing object of the speech recognition system.

[0032] It should be noted that the preset hot word library refers to a pre-built vocabulary library that stores hot words optimized for specific scenarios (such as finance, healthcare, etc.) and their corresponding weights. The hot word library is generated by cleaning, topic modeling, and weight calculation of domain corpora, and is used to assist ASR models in improving the recognition accuracy of key terms.

[0033] It should be noted that hot words refer to terms that frequently appear in specific business scenarios or professional fields and are easily misidentified by general ASR models. For example, "China Merchants Bank" and "wealth management products" in financial scenarios. The hot word generation process combines LDA topic segmentation and TF-IDF discriminant calculation to ensure that they are both relevant to the scenario and have discriminative power.

[0034] It's important to note that the weight refers to the numerical value corresponding to each hot word, representing the importance of that word in the ASR recognition process or the strength of its priority for replacement. The weight is calculated by weighting the LDA topic probability and the TF-IDF value. The higher the weight, the more likely the ASR model is to replace the hot word with similar-sounding results during recognition.

[0035] It should be noted that topic distribution refers to the probability distribution of each document (or corpus segment) across various preset topics in the LDA (Latent Dirichlet Allocation) model. It reflects the degree of correlation between the document content and each topic. It is estimated through iterative training of the LDA model and is used to assign scene labels to documents.

[0036] It should be noted that the word distribution of a topic refers to the probability distribution of each word under each topic, representing the semantic relevance of the words to that topic. For example, under the topic of "finance," words such as "transfer" and "balance" have higher probabilities. This is also estimated using the LDA model.

[0037] It's important to note that word discrimination refers to a word's ability to distinguish different documents within the entire corpus, typically measured by IDF (Inverse Document Frequency). The fewer documents a word appears in, the higher its IDF value and the stronger its discrimination. In hot word generation, IDF is used to filter words that are representative of a given context.

[0038] It should be noted that the pre-trained LDA model refers to a pre-trained LDA topic model used to extract topic distributions and topic word distributions from domain corpora. LDA is an unsupervised learning algorithm that estimates parameters through iterative methods such as Gibbs sampling, ultimately outputting document-topic distributions and topic-word distributions.

[0039] It should be noted that the preset IDF algorithm refers to a predefined IDF calculation formula used to calculate the inverse document frequency of words in a corpus.

[0040] In this embodiment, upon receiving a speech recognition command, the system acquires the speech signal to be recognized and loads a pre-built hot word library. This hot word library is not a static, general vocabulary, but rather dynamically optimized for specific business scenarios using LDA and IDF algorithms, containing scenario-related hot words and their weights. This provides prior knowledge for the accurate recognition of the subsequent ASR model.

[0041] Understandably, by using LDA to segment scenarios and IDF to calculate discriminative power, the generated hot words can more accurately reflect domain characteristics and avoid interference from general vocabulary. The hot word library can be customized for different business scenarios (such as customer service and healthcare), enabling the ASR model to perform better in specific scenarios. A high-quality hot word library directly impacts the ASR decoding process, laying the foundation for correcting misidentifications in general models.

[0042] Step S20: Based on the speech to be recognized and the preset hot word library, a preset ASR model is used to generate a speech recognition result.

[0043] It should be noted that the pre-trained ASR model refers to a pre-trained Automatic Speech Recognition (ASR) model that can convert input audio signals into corresponding text sequences. During the decoding process, this model can utilize hot words and their weights from a hot word library to dynamically adjust the recognition candidates (e.g., incorporating hot word weights into the language model or using them as bias terms), thereby improving the recognition accuracy of specific words.

[0044] It should be noted that the speech recognition result refers to the final output text content, that is, the natural language sentence generated by the ASR model based on the speech to be recognized and the hot word library. This result should more accurately include scene-related terms.

[0045] In this embodiment, the speech to be recognized is input into the ASR model, and the hot word weight information in the preset hot word library is used to guide the model to prioritize these words during decoding, ultimately generating an optimized text result. The intervention of the hot word library can correct errors that may be produced by the general model (such as homophone misrecognition and omission of low-frequency words).

[0046] Understandably, by leveraging hot word weights, ASR models can more accurately identify the pronunciations corresponding to hot words in audio as target words, especially for technical terms and new words. In specific business scenarios (such as voice search and intelligent customer service), users' key intentions are often reflected in hot words; accurately identifying these words can improve interaction fluency and satisfaction. The hot word library can be dynamically updated according to business needs, continuously improving recognition performance without retraining the ASR model.

[0047] This application discloses a speech recognition method and related equipment, relating to the field of data processing technology. Among related technologies, Automatic Speech Recognition (ASR) technology has significantly improved its recognition accuracy in general fields. However, in specific business scenarios such as finance, healthcare, and government affairs, there are still problems with inaccurate recognition of professional terms, obscure words, and new words. Existing methods typically guide the recognition results by injecting "hot words" into the ASR model, but the quality of the hot words directly affects the recognition effect. Current methods for generating hot words often rely on manual experience or simple statistics, lacking effective corpus cleaning and semantic modeling. This can easily lead to redundant hot words, weight imbalances, and even the introduction of noise, thus affecting recognition accuracy. In contrast, in this application, firstly, in response to a speech recognition command, the speech to be recognized and a preset hot word library are acquired. The preset hot word library contains multiple hot words and the weight corresponding to each hot word. The hot words and the weights are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. Then, based on the speech to be recognized and the preset hot word library, a preset ASR model is used to generate speech recognition results.

[0048] Understandably, before performing speech recognition, this application determines the topic distribution, the word distribution of the topic, and the discriminative power of the corresponding words based on a preset LDA model and a preset IDF algorithm, thereby generating high-quality hot words and generating a weight for each hot word. During the speech recognition process, more accurate speech recognition results are obtained based on the high-quality hot words.

[0049] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Before the step of obtaining the speech to be recognized and the preset hot word library, steps A100 to A400 are also included: Step A100: Obtain multiple corpus documents in different formats; It should be noted that the corpus documents refer to the original text materials used to generate hot words, which may include various formats such as HTML web page files, JSON data files, DOC / DOCX Word documents, etc. These documents come from specific business fields (such as finance, healthcare, customer service, etc.) and cover commonly used terminology, expressions, and knowledge content in that field.

[0050] Understandably, this step is the starting point of the hot word generation process, and its main task is to collect raw data. By obtaining corpus documents in multiple formats from different sources, a rich textual foundation is provided for subsequent hot word mining.

[0051] Understandably, multi-format documents can cover language expressions in different scenarios, such as web page text, business reports, and user dialogue records, ensuring that the generated hot words have broad domain representativeness. A rich corpus is a prerequisite for subsequent cleaning and modeling; the larger the data volume and the wider the coverage, the more accurately the generated hot words reflect real business needs.

[0052] Step A200: Preprocess the multiple corpus documents in different formats to obtain multiple sets of structured, standard text content; It should be noted that preprocessing refers to a series of cleaning and transformation operations performed on the original corpus documents to eliminate noise, redundancy, and format differences.

[0053] It should be noted that document deduplication refers to removing identical documents by calculating the file hash value (such as SHA256), thus avoiding duplicate data from taking up space and affecting computational efficiency.

[0054] It should be noted that noise removal and text extraction refer to removing non-text content such as images, links, tags, and metadata from documents of different formats (HTML, JSON, DOC, etc.) to extract the pure text, and then uniformly encoding it.

[0055] It should be noted that semantic deduplication refers to using MinHash and LSH (Local Sensitive Hash) algorithms to remove documents with the same content but different file hashes based on text content similarity, preventing the same content from being counted repeatedly and causing the weight of hot words to be artificially high.

[0056] It should be noted that text standardization refers to the process of unifying the format, encoding, and structuring the extracted text to generate standardized plain text data.

[0057] It should be noted that structured, standard text content refers to text data obtained after preprocessing, which has a uniform format (such as plain text, UTF-8 encoding), is free of noise and repetition, and has a clear content structure, and can be directly used as input for subsequent algorithms (LDA, TF-IDF).

[0058] Understandably, the core task of this step is data cleaning and standardization, transforming the messy and heterogeneous raw corpus into a high-quality, consistent text set, providing a reliable data source for subsequent hot word generation.

[0059] Understandably, removing duplicate documents and noisy content avoids interference from useless information, thus improving the accuracy of LDA topic modeling and TF-IDF calculation. Semantic deduplication eliminates duplicate counts of documents with the same content but different formats, ensuring that hot word weights accurately reflect the distribution of words in the corpus. Standardized text is easier for algorithms to process directly, reducing errors caused by formatting issues and improving processing efficiency.

[0060] Specifically, the step of preprocessing the multiple corpus documents of different formats to obtain multiple sets of structured, standard text content further includes steps A210 to A240: Step A210: Based on file hashing, perform document deduplication on the multiple corpus documents of different formats to obtain multiple deduplicated corpus documents; Understandably, this step is the first stage of preprocessing, which involves calculating the file-level hash value (such as SHA256) for each corpus document, comparing and removing documents with identical hash values, thus achieving initial deduplication.

[0061] Understandably, removing completely duplicate documents saves storage space and reduces the amount of data processed subsequently. It also prevents the same document from being counted multiple times, which could lead to inflated keyword weights and ensures the objectivity of keyword weights.

[0062] Step A220: Perform noise processing and text extraction operations sequentially on the multiple deduplicated corpus documents to obtain multiple sets of text content; Understandably, this step performs noise removal and text extraction operations for documents of different formats (such as HTML, JSON, DOC, etc.): removing non-text elements such as images, links, tags, and metadata, locating and extracting the main text content, and unifying the text encoding to finally obtain clean text data.

[0063] Understandably, eliminating formatting noise ensures that the extracted text contains only semantic content, providing high-quality input for subsequent semantic analysis. Customized processing workflows are employed for different document types to guarantee the effective utilization of corpora from various sources.

[0064] Step A230: Based on MinHash and LSH, perform semantic deduplication on the multiple sets of text content to obtain multiple sets of semantically deduplicated text content; It should be noted that MinHash is a hashing technique used to quickly estimate the similarity between two sets (such as the bag-of-words model of text). It approximates the similarity of the sets by generating fixed-length signatures and comparing the similarity between the signatures, thereby determining the similarity of the text content.

[0065] It's important to note that LSH (Locality Sensitive Hash) is a hashing method that maps similar items to the same "bucket" with a high probability, enabling fast nearest neighbor lookup. In semantic deduplication, LSH utilizes MinHash signatures to quickly cluster similar text, thereby efficiently identifying and removing documents with duplicate content.

[0066] Understandably, this step performs semantic-based secondary deduplication on the text content: first, a bag-of-words model of the text is constructed and the MinHash signature is calculated, and then the signature is bucketed using LSH to quickly find the set of texts with high similarity and mark and remove records with duplicate content (even if the file hashes are different).

[0067] Understandably, this step can detect documents with identical content but different metadata (such as format and timestamps), avoiding missed detections. It ensures that the same semantic content is not repeatedly calculated, allowing subsequent hot word weights to accurately reflect the distribution of words in the corpus. LSH's fast clustering mechanism significantly reduces the computational complexity of large-scale text deduplication.

[0068] Specifically, the step of performing semantic deduplication on the multiple sets of text content based on MinHash and LSH to obtain multiple sets of semantically deduplicated text content further includes steps A231 to A233: Step A231: Construct a bag-of-words model based on the multiple sets of text content; It should be noted that the bag-of-words model is a simplified model that represents text as numerical vectors. It ignores the word order and grammar in the text, and only counts the number of times (or frequency) each word appears in the document, transforming the text into a set representation of "word-frequency".

[0069] Understandably, this step converts the cleaned text content into a mathematical representation that can be processed by a computer: by word segmentation (such as using the jieba word segmenter) and counting word frequencies, a bag-of-words vector is constructed for each document, preparing data for subsequent similarity calculations.

[0070] It's understandable that converting unstructured natural language text into a structured vector form facilitates processing by subsequent algorithms (such as MinHash). Although word order information is lost, the frequency of word occurrence is preserved, which is sufficient to support the estimation of text similarity.

[0071] Step A232: Based on the bag-of-words model, calculate the MinHash signature corresponding to each group of text content; It should be noted that a MinHash signature refers to a fixed-length set of hash values ​​generated using the MinHash (minimum hashing) technique, used to approximate the characteristics of the original text set. It compresses the bag-of-words set of the text into a compact signature vector. The more similar the signatures of two texts, the higher the Jaccard similarity of their original content.

[0072] Understandably, this step applies the MinHash algorithm to the bag-of-words model for each document: it maps the elements in the bag-of-words set through multiple hash functions, takes the minimum hash value each time to form a signature, and thus compresses the high-dimensional bag-of-words space into a low-dimensional signature vector.

[0073] Understandably, compressing large-scale bag-of-words data into fixed-length signatures significantly reduces storage and computational overhead. Comparisons between signatures can approximate the similarity calculation of the original set, providing an efficient pre-screening method for large-scale text deduplication.

[0074] Step A233: Based on LSH and the MinHash signature, identify similar text content in the multiple sets of text content, remove the similar text content, and obtain multiple sets of semantically deduplicated text content.

[0075] Understandably, this step utilizes LSH to perform bucket clustering on MinHash signatures: documents with similar signatures are mapped to the same bucket, quickly identifying sets of texts with highly similar content, and then only one representative document is kept from each similar set, removing the remaining duplicates to achieve semantic deduplication.

[0076] Understandably, LSH's "bucketing" mechanism avoids global pairwise comparisons, making large-scale text similarity detection feasible and fast. Based on MinHash signatures, LSH can capture the semantic similarity of texts, effectively removing documents with essentially the same content but slightly different expressions (such as rewrites or formatting changes). By removing semantically repetitive text, it ensures that the same content is not counted multiple times during subsequent hot word generation, avoiding inflated weights.

[0077] Step A240: Perform text standardization on the multiple sets of semantically deduplicated text content to obtain multiple sets of structured, standard text content.

[0078] Understandably, this step involves standardizing the format of the semantically deduplicated text, including unifying character encoding, standardizing punctuation, removing redundant whitespace, and adjusting the text structure to generate structured standard text.

[0079] Understandably, by eliminating processing barriers caused by encoding and format differences, it ensures that subsequent LDA topic modeling and TF-IDF calculations can directly and efficiently process text. Standardized text is the foundation for stable algorithm operation, avoiding errors or deviations caused by format issues and improving the accuracy of hot word generation.

[0080] Step A300: Based on the multiple sets of structured, standard text content, the preset LDA model, and the preset IDF algorithm, generate multiple hot words and the corresponding weight for each hot word; It should be noted that this step is the core of hot word generation. By combining LDA and TF-IDF algorithms, hot words that are both relevant to the context and distinctive are extracted from standardized text, and they are assigned appropriate weights.

[0081] Understandably, LDA clusters documents by topic, ensuring that hot words have semantic relevance in specific scenarios and avoiding interference from general vocabulary. IDF filters out rare but crucial words in the corpus, ensuring that hot words highlight domain characteristics. The weighted calculation considers both the importance of words in the document (TF-IDF) and topic relevance (LDA probability), enabling hot words to more effectively guide recognition during ASR decoding.

[0082] Step A400: Based on the hot words and the weights, construct a preset hot word library.

[0083] Understandably, this step is the final step in the hot word generation process. It organizes the calculated hot words and weights into a usable format, builds a hot word library, and provides direct support for subsequent speech recognition optimization.

[0084] Understandably, a pre-built hot word library can be directly integrated into the ASR system without modifying the model itself, enabling rapid deployment and scenario adaptation. The hot word library supports dynamic addition, deletion, or adjustment of hot words and their weights, allowing for timely responses to business changes and continuous improvement in recognition accuracy. The ASR model can directly load the hot word library during decoding, quickly correcting misidentifications in general models through weight biasing, thereby improving the recognition rate of key terms.

[0085] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 The step of generating multiple hot words and their corresponding weights based on the multiple sets of structured, standard text content, the preset LDA model, and the preset IDF algorithm further includes steps B10 to B30: Step B10: Use a preset LDA model to process the multiple sets of structured, standard text content to obtain topic distribution and topic word distribution; Understandably, this step is one of the core components of hot word generation. The pre-processed, standardized text is input into a pre-trained LDA topic model. Through model inference (or iterative training), the topic distribution (document-topic probability) and the word distribution (topic-word probability) for each document are obtained. The topic distribution labels each text with a scene tag, while the topic word distribution reveals the keywords for each scene.

[0086] Understandably, by using LDA topic clustering to group the corpus according to latent semantic topics, subsequent hot word selection can be conducted independently for different scenarios, avoiding cross-scenario terminology interference. The distribution of topic words directly provides vocabulary strongly related to the topic, and these words themselves are scenario-representative, providing a candidate basis for hot word generation.

[0087] Specifically, the step of processing the multiple sets of structured, standard text content using a preset LDA model to obtain the topic distribution and the word distribution of the topics further includes steps B11-B12: Step B11: Based on jieba and a preset custom dictionary, perform Chinese and English word segmentation, stemming, and word form restoration operations on the multiple sets of structured and standard text content to construct a bag of words; It should be noted that jieba is a commonly used Chinese word segmentation tool (library) that can divide continuous Chinese text into independent word units.

[0088] It should be noted that the preset custom dictionary refers to a predefined domain-specific vocabulary, containing terms, proper nouns, or compound words specific to the business scenario. During word segmentation, the custom dictionary can guide jieba to correctly identify these words and avoid incorrect segmentation.

[0089] It should be noted that Chinese and English word segmentation refers to the process of dividing text (including mixed Chinese and English content) into independent word units. Chinese word segmentation needs to handle the problem of unclear word boundaries, while English word segmentation is based on spaces and punctuation.

[0090] It should be noted that stemming refers to the process of removing inflections from English words and restoring them to their stem form (e.g., restoring "running" and "ran" to "run"). Its purpose is to merge different forms of the same word, reducing vocabulary sparsity.

[0091] It should be noted that lemma reduction refers to the process of reducing a word to its lexical prototype (lemma), which is more accurate than stemming and takes into account part of speech and context (e.g., reducing "better" to "good"). Lemma reduction is also used for the normalization of English text.

[0092] Understandably, this step is a text preprocessing stage before LDA modeling: first, the jieba word segmentation tool (combined with a preset custom dictionary) is used to accurately segment the standardized text into Chinese and English words; then, stemming and word form restoration are performed on the English words in the segmentation results to unify the word forms; finally, a bag-of-words model is built based on the processed word set as the input of the LDA model.

[0093] Understandably, a pre-defined dictionary ensures that domain terms are not incorrectly segmented, preserving their complete semantics. Stemming and lemmatization merge different forms of the same word, reducing feature dimensionality and enabling the LDA model to more accurately capture the semantic relationships between words. High-quality bag-of-words input helps the LDA model generate a more stable and reasonable topic distribution and topic word distribution.

[0094] Step B12: Based on the bag of words, iteratively train the preset LDA model until the topic allocation corresponding to the preset LDA model tends to stabilize, and obtain the topic distribution and the word distribution of the topic. The iterative training is implemented based on Gibbs sampling, which is used to converge the topic distribution results and the word distribution results of the topic.

[0095] It's important to note that iterative training refers to the process in LDA models where the model parameters are gradually optimized by repeatedly updating the topic assignment for each word. Each iteration recalculates the document-topic and topic-word counts based on the current state, until the model converges.

[0096] It should be noted that topic assignment tends to stabilize when, after multiple iterations, the topic attribution of each word no longer changes significantly, or the numerical changes in document-topic distribution and topic-word distribution are minimal. At this point, the model is considered to have converged.

[0097] It should be noted that Gibbs sampling refers to a Markov Chain Monte Carlo (MCMC) method used to approximate sampling from a complex joint distribution. In LDA, Gibbs sampling fixes the topics of other words and resamples the topic of the current word according to the conditional probability distribution. After repeated iterations, the topic assignments of all words gradually approximate the true posterior distribution.

[0098] It should be noted that convergence means that after a sufficient number of iterations, the model parameters (such as topic assignment and topic distribution) essentially no longer change with each iteration, reaching a stable state. The output of the converged model is the final topic distribution and keyword distribution.

[0099] Understandably, this step executes the core training process of the LDA model: based on the constructed bag-of-words, the model is iteratively trained using the Gibbs sampling algorithm. In each iteration, the algorithm resamples the topic assignment for each word according to the current document-topic and topic-word count matrices, and updates the count matrix. After multiple iterations, when the topic assignments tend to stabilize, the model converges, and the output document-topic distribution and topic-word distribution are the final results.

[0100] It should be noted that the formula for the Gibbs sampling algorithm is as follows: in, Indicates the number of topics; Indicates that besides words The topic assignment of all words except those mentioned above; Indicates removal Then, the number of words in topic k in document d ( (If the original topic is k). Indicates removal Then, the frequency of word w in topic k ( (If the original topic is k). The hyperparameters of the document-topic Dirichlet distribution, which control the sparsity of the document and topic distributions, are represented by a K-dimensional vector. This represents the hyperparameters of the topic-word Dirichlet distribution, used to control the sparsity of the topic and word distributions; Document-topic distribution hyperparameters The k-th parameter corresponds to a component of topic k; Represents the topic-word distribution hyperparameters The w-th component corresponds to the component of word w; express; This represents the word distribution corresponding to topic k.

[0101] Understandably, Gibbs sampling can effectively approximate the posterior distribution of the LDA model, generating reliable document-topic and topic-word probabilities. Iterative sampling is suitable for processing large-scale text data, gradually optimizing topic allocation and avoiding local optima. By iterating until convergence, the output topic and topic word distributions are ensured to have statistical stability and semantic rationality, providing a high-quality foundation for subsequent hot word generation.

[0102] Step B20: Based on the multiple sets of structured, standard text content, calculate the distinguishability of each word using a preset IDF algorithm; Understandably, this step uses the IDF algorithm to calculate the inverse document frequency (IDF) of each word in the entire corpus, i.e., the word's discriminative power. The IDF value measures how many documents a word appears in; the fewer documents a word appears, the higher its discriminative power, indicating that the word is more representative of the uniqueness of a specific domain.

[0103] Understandably, IDF can effectively filter out words that are rare in the corpus but frequently appear in specific documents; these words are often technical terms or hot words in the field. LDA focuses on the co-occurrence relationship of words within a topic, while IDF assesses the rarity of words from a global perspective. Combining the two can provide a more comprehensive assessment of the importance of words.

[0104] Specifically, the step of calculating the discriminative power of words using a preset IDF algorithm based on the multiple sets of structured, standard text content further includes steps B21 to B22: Step B21: Determine the frequency of occurrence of each word in the multiple sets of structured, standard text content; It should be noted that frequency of occurrence refers to the number of times a word appears in a specific document, i.e., word frequency.

[0105] Understandably, this step involves performing word frequency statistics on the preprocessed standardized text: traversing each document, counting the raw number of times each word appears within the document, and forming a word frequency matrix or word frequency list. This is the foundational data for subsequent IDF calculations and TF-IDF weighted averages.

[0106] Understandably, word frequency reflects the importance of a word in a single document; higher frequency generally indicates a greater contribution of the word to the document's content. The raw term frequency is a direct input to the TF part of the TF-IDF algorithm, ensuring an accurate data foundation for subsequent weight calculations.

[0107] Step B22: Calculate the distinguishability of the word based on the occurrence frequency and the preset IDF algorithm.

[0108] Understandably, this step calculates the IDF value by first counting how many documents each word appears in (document frequency), and then using a predefined IDF formula to calculate the inverse document frequency (IVF) of each word, i.e., the discriminant power. A higher discriminant power indicates that the word is rarer in the corpus and more representative of a specific domain.

[0109] Understandably, IDF can effectively filter out words that are rare in the corpus but frequently appear in specific documents; these words are often technical terms or hot words in the field. Discrimination assesses the rarity of words from a global perspective, complementing LDA's thematic relevance and providing an important dimension for subsequent hot word weight fusion. IDF calculation based on global document frequency avoids interference from high-frequency but general words within local documents, making hot words more representative.

[0110] Step B30: Based on a preset adjustment coefficient, the distinguishability and the word distribution of the topic are weighted and summed to generate multiple hot words corresponding to different topics and the weight of each hot word.

[0111] It should be noted that the preset adjustment coefficient ( ), is an adjustable parameter used to balance the contributions of discrimination (IDF part) and topic relevance (LDA topic word probability) in the final weight. Through multiple experiments or empirical settings, the influence of topic probability on hot word weight can be controlled, making the generated hot words more focused on discrimination or topic relevance.

[0112] It should be noted that weighted summation refers to the operation of multiplying the two values ​​(discrimination score and keyword probability) by weight coefficients and then adding them together.

[0113] It should be noted that the default IDF algorithm is: Furthermore, the formula for weighted summation is: in, This indicates the number of times the word appears in the current document d; This represents the total number of words in document d; Indicates the current word / phrase; Words that belong to the current document; Indicates the current document; This represents the number of documents in the entire corpus (document collection); The number of documents in scenario k; This represents the adjustment coefficient, used to control the impact of topic probability on weights, and is adjusted multiple times. represents the probability distribution of word t in topic k; TF-IDF(t,d,D) represents the word's discriminative power; Dynamic-TF-IDF(t,k) represents the weight.

[0114] Understandably, this step integrates the topic word probabilities obtained from LDA with the discriminative power calculated by IDF through adjustment coefficients to generate the final weight of each word under different topics, and then selects a list of hot words under each topic.

[0115] Understandably, considering both the semantic importance (LDA) of words within the topic and their global rarity (IDF) simultaneously ensures that the generated hot words are both relevant to the context and representative. (Adjustment coefficient) Allows adjusting the impact of topic relevance based on business needs; for example, it can be increased in terminology-intensive scenarios. In general scenarios, it can reduce This enables adaptive optimization. The resulting hot words and their weights can be directly used for hot word bias in the ASR model, effectively improving speech recognition accuracy in specific scenarios.

[0116] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the speech recognition method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0117] This application also provides a voice recognition device, please refer to... Figure 4 The voice recognition device includes: The acquisition module 10 is used to acquire the speech to be recognized and a preset hot word library in response to a speech recognition command. The preset hot word library contains multiple hot words and the weight corresponding to each hot word. The hot words and the weight are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. The generation module 20 is used to generate speech recognition results based on the speech to be recognized and the preset hot word library using a preset ASR model.

[0118] In one embodiment, the speech recognition device further includes a construction module, which further includes: The acquisition unit is used to acquire multiple corpus documents in different formats; The preprocessing unit is used to preprocess the multiple corpus documents of different formats to obtain multiple sets of structured, standard text content; The generation unit is used to generate multiple hot words and the corresponding weight of each hot word based on the multiple sets of structured, standard text content, the preset LDA model and the preset IDF algorithm; The first construction unit is used to construct a preset hot word library based on the hot words and the weights.

[0119] In one embodiment, the building module further includes: The document deduplication unit is used to perform document deduplication operations on the multiple corpus documents of different formats based on file hashing, so as to obtain multiple deduplicated corpus documents; The noise processing and text extraction unit is used to sequentially perform noise processing and text extraction operations on the multiple deduplicated corpus documents to obtain multiple sets of text content. The semantic deduplication unit is used to perform semantic deduplication operations on the multiple sets of text content based on MinHash and LSH to obtain multiple sets of semantically deduplicated text content. The text standardization unit is used to perform text standardization operations on the multiple sets of semantically deduplicated text content to obtain multiple sets of structured, standard text content.

[0120] In one embodiment, the building module further includes: The second construction unit is used to construct a bag-of-words model based on the multiple sets of text content; The first calculation unit is used to calculate the MinHash signature corresponding to each group of text content based on the bag-of-words model; The first determining unit is used to determine similar text content among the multiple sets of text content based on LSH and the MinHash signature, remove the similar text content, and obtain multiple sets of semantically deduplicated text content.

[0121] In one embodiment, the building module further includes: The data processing unit is used to process the multiple sets of structured and standard text content using a preset LDA model to obtain the topic distribution and the word distribution of the topic; The second calculation unit is used to calculate the distinguishability of words based on the multiple sets of structured and standard text content using a preset IDF algorithm. The weighted summation unit is used to perform a weighted summation of the distinguishability and the word distribution of the topic based on a preset adjustment coefficient, so as to generate multiple hot words corresponding to different topics and the weight corresponding to each hot word.

[0122] In one embodiment, the building module further includes: The third construction unit is used to perform Chinese and English word segmentation, stemming, and word form restoration operations on the multiple sets of structured and standard text content based on jieba and a preset custom dictionary, so as to construct a bag of words; An iterative training unit is used to iteratively train the preset LDA model based on the bag of words until the topic allocation corresponding to the preset LDA model tends to stabilize, thereby obtaining the topic distribution and the word distribution of the topic. The iterative training is implemented based on Gibbs sampling, which is used to converge the topic distribution results and the word distribution results of the topic.

[0123] In one embodiment, the building module further includes: The second determining unit is used to determine the frequency of occurrence of each word in the multiple sets of structured, standard text content; The third calculation unit is used to calculate the distinguishability of a word based on its occurrence frequency and a preset IDF algorithm.

[0124] The speech recognition device provided in this application, employing the speech recognition method in the above embodiments, can solve the technical problems of speech recognition. Compared with related technologies, the beneficial effects of the speech recognition device provided in this application are the same as those of the speech recognition method provided in the above embodiments, and other technical features in the speech recognition device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0125] This application provides a speech recognition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the speech recognition method in Embodiment 1 above.

[0126] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a voice recognition device suitable for implementing embodiments of this application. The voice recognition device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The voice recognition device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0127] like Figure 5 As shown, the voice recognition device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the voice recognition device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the voice recognition device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show voice recognition devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0128] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0129] The speech recognition device provided in this application, employing the speech recognition method in the above embodiments, can solve the technical problems of speech recognition. Compared with related technologies, the beneficial effects of the speech recognition device provided in this application are the same as those of the speech recognition method provided in the above embodiments, and other technical features of the speech recognition device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0130] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0131] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0132] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the speech recognition method in the above embodiments.

[0133] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0134] The aforementioned computer-readable storage medium may be included in the speech recognition device; or it may exist independently and not be assembled into the speech recognition device.

[0135] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a speech recognition device, cause the speech recognition device to: In response to a speech recognition command, the system acquires the speech to be recognized and a preset hot word library. The preset hot word library contains multiple hot words and a weight corresponding to each hot word. The hot words and the weights are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. Based on the speech to be recognized and the preset hot word library, a preset ASR model is used to generate speech recognition results.

[0136] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0138] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0139] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described speech recognition method, thereby solving the technical problem of speech recognition. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the speech recognition method provided in the above embodiments, and will not be repeated here.

[0140] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the speech recognition method described above.

[0141] The computer program product provided in this application can solve the technical problem of speech recognition. Compared with related technologies, the beneficial effects of the computer program product provided in this application are the same as those of the speech recognition method provided in the above embodiments, and will not be repeated here.

[0142] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A speech recognition method, characterized in that, The speech recognition method includes: In response to a speech recognition command, the system acquires the speech to be recognized and a preset hot word library. The preset hot word library contains multiple hot words and a weight corresponding to each hot word. The hot words and the weights are generated based on topic distribution, word distribution of topics, and word discrimination. The topic distribution and word distribution of topics are estimated based on a preset LDA model, and the word discrimination is calculated based on a preset IDF algorithm. Based on the speech to be recognized and the preset hot word library, a preset ASR model is used to generate speech recognition results.

2. The speech recognition method as described in claim 1, characterized in that, Before the steps of obtaining the speech to be recognized and the preset hot word library, the method further includes: Obtain multiple corpus documents in different formats; The multiple corpus documents in different formats are preprocessed to obtain multiple sets of structured, standard text content; Based on the multiple sets of structured, standard text content, the preset LDA model, and the preset IDF algorithm, multiple hot words and the corresponding weights for each hot word are generated. Based on the hot words and the weights, a preset hot word library is constructed.

3. The speech recognition method as described in claim 2, characterized in that, The step of preprocessing the multiple corpus documents of different formats to obtain multiple sets of structured, standard text content further includes: Based on file hashing, a document deduplication operation is performed on the multiple corpus documents of different formats to obtain multiple deduplicated corpus documents; Noise processing and text extraction operations are sequentially performed on the multiple deduplicated corpus documents to obtain multiple sets of text content; Based on MinHash and LSH, semantic deduplication is performed on the multiple sets of text content to obtain multiple sets of semantically deduplicated text content. Perform text standardization on the multiple sets of semantically deduplicated text content to obtain multiple sets of structured, standard text content.

4. The speech recognition method as described in claim 3, characterized in that, The step of performing semantic deduplication on the multiple sets of text content based on MinHash and LSH to obtain multiple sets of semantically deduplicated text content further includes: Based on the multiple sets of text content, a bag-of-words model is constructed; Based on the bag-of-words model, calculate the MinHash signature corresponding to each group of text content; Based on LSH and the MinHash signature, similar text content is identified among the multiple sets of text content, and the similar text content is removed to obtain multiple sets of semantically deduplicated text content.

5. The speech recognition method as described in claim 2, characterized in that, The step of generating multiple hot words and their corresponding weights based on the multiple sets of structured, standard text content, the preset LDA model, and the preset IDF algorithm further includes: The pre-defined LDA model is used to process the multiple sets of structured, standard text content to obtain the topic distribution and the word distribution of the topics; Based on the aforementioned multiple sets of structured, standard text content, the discriminative power of each word is calculated using a preset IDF algorithm. Based on a preset adjustment coefficient, the discrimination score and the word distribution of the topic are weighted and summed to generate multiple hot words corresponding to different topics and the weight of each hot word.

6. The speech recognition method as described in claim 5, characterized in that, The step of processing the multiple sets of structured, standard text content using a preset LDA model to obtain topic distribution and topic word distribution further includes: Based on jieba and a preset custom dictionary, Chinese and English word segmentation, stemming, and word form restoration operations are performed on the multiple sets of structured and standard text content to construct a bag of words; Based on the bag of words, the preset LDA model is iteratively trained until the topic allocation corresponding to the preset LDA model tends to stabilize, thereby obtaining the topic distribution and the word distribution of the topic. The iterative training is implemented based on Gibbs sampling, which is used to converge the topic distribution results and the word distribution results of the topic.

7. The speech recognition method as described in claim 5, characterized in that, The step of calculating the discriminative power of words based on the multiple sets of structured, standard text content using a preset IDF algorithm further includes: Determine the frequency of occurrence of each word in the multiple sets of structured, standard text content; Based on the frequency of occurrence and the preset IDF algorithm, the distinguishability of the words is calculated.

8. A voice recognition device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the speech recognition method as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the speech recognition method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the speech recognition method as described in any one of claims 1 to 7.

Images