Speech recognition method, device and equipment, program product and storage medium

By obtaining and utilizing hot word key vectors and value vectors in the speech recognition model, and combining the attention mechanism to encode and filter speech information, the problem of poor hot word recognition effect in the existing technology is solved, and the accuracy of speech recognition is achieved. Hot word expansion, suitable for low-resource equipment.

CN119068868APending Publication Date: 2024-12-03TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310638081.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

When the existing speech recognition model deals with speech recognition containing a large number of hot words, there is a problem of poor recognition effect, especially when the training corpus is inconsistent with the actual scenario, hot words cannot be accurately recognized, which affects the system's usage effect.

Method used

By obtaining the hot word key vectors and value vectors in the initial hot word list, using the attention mechanism to encode and filter the recognized voice information, generate a target hot word list, and use these hot word lists during the decoding process to achieve the voice information Accurate recognition of hot words in it.

Benefits of technology

It improves the accuracy of speech recognition, supports the expansion of hot words and real-time addition, and does not affect the computing speed of the speech recognition model. It is suitable for low-resource equipment and reduces hardware costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068868A_ABST
    Figure CN119068868A_ABST
Patent Text Reader

Abstract

The invention provides a voice recognition method, which comprises the following steps: acquiring a first key vector and a first value vector of at least one hot word in an initial hot word list, acquiring to-be-recognized voice information, and encoding the to-be-recognized voice information to obtain a hidden layer output vector of the to-be-recognized voice information; based on an attention mechanism, filtering the initial hot word list by utilizing the hidden layer output vector, the first key vector and the first value vector to obtain a target hot word list; obtaining a second key vector and a second value vector of at least one target hot word in the target hot word list; and decoding the second key vector, the second value vector and the hidden layer output vector to obtain a voice recognition result corresponding to the to-be-recognized voice information, the voice recognition result comprising at least one target hot word in the target hot word list. The invention further provides a voice recognition device, an electronic equipment computer program product and a storage medium. The hot words in the voice information can be accurately recognized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to speech information processing technology, and in particular to speech recognition methods, devices, electronic devices, computer program products and storage media. Background Art

[0002] With the development of machine learning technology, machine learning can be implemented through the BERT (Bidirectional Encoder Representations from Transformers) mechanism. In the process of using the speech recognition model corresponding to BERT to recognize speech data, as the language usage environment develops and changes, the scenario corresponding to the corpus used in the training of the speech recognition model is likely to be inconsistent with the actual scenario, and the actual scenario may contain words that do not appear or appear very rarely in the training corpus. Therefore, the speech recognition model in the related technology cannot recognize or misrecognizes hot words in the speech information, causing the actual use effect of the speech recognition system to deteriorate, affecting the normal use of users. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a speech recognition method, device, electronic device, computer program product and storage medium. The technical solution of the embodiments of the present application is implemented as follows:

[0004] The present application provides a speech recognition method, the method comprising:

[0005] Obtain a first key vector and a first value vector of at least one hot word in the initial hot word list, wherein the hot word is used to represent a word that appears more than a threshold number of times within a preset time range;

[0006] Acquire speech information to be recognized, encode the speech information to be recognized, and obtain a hidden layer output vector of the speech information to be recognized;

[0007] Based on the attention mechanism, the initial hot word list is filtered using the hidden layer output vector, the first key vector and the first value vector to obtain a target hot word list;

[0008] Acquire a second key vector and a second value vector of at least one target hot word in the target hot word list;

[0009] The second key vector, the second value vector and the hidden layer output vector are decoded to obtain a speech recognition result corresponding to the speech information to be recognized, wherein the speech recognition result includes at least one target hot word in the target hot word list.

[0010] The embodiment of the present application also provides a speech information recognition device, characterized in that the device comprises:

[0011] An encoder module, configured to obtain a first key vector and a first value vector of at least one hot word in an initial hot word list, wherein the hot word is used to represent a word that appears more than a threshold number of times within a preset time range;

[0012] The encoder module is used to obtain the voice information to be recognized, encode the voice information to be recognized, and obtain the hidden layer output vector of the voice information to be recognized;

[0013] A hot word processing module, configured to filter the initial hot word list based on an attention mechanism by using the hidden layer output vector, the first key vector and the first value vector to obtain a target hot word list;

[0014] The hot word processing module is used to obtain a second key vector and a second value vector of at least one target hot word in the target hot word list;

[0015] A decoder module is used to decode the second key vector, the second value vector and the hidden layer output vector to obtain a speech recognition result corresponding to the speech information to be recognized, wherein the speech recognition result includes at least one target hot word in the target hot word list.

[0016] In some embodiments, the hot word processing module is used to use the hot word filtering module to perform a dot product process on the hidden layer output vector and the first key vector to obtain a first dot product result;

[0017] The hot word processing module is used to scale the first dot product result according to the dimension of the first key vector to obtain a first scaled dot product result;

[0018] The hot word processing module is used to perform normalization processing on the first scaling dot product result to obtain a first normalized dot product result;

[0019] The hot word processing module is used to determine the product of the first normalized dot product result and the first value vector as a reference value vector;

[0020] The hot word processing module is used to filter the initial hot word list according to the reference value vector to obtain the target hot word list, wherein the target hot word list includes the target hot words corresponding to the reference value vector.

[0021] In some embodiments, the hot word processing module is used to obtain an initial hot word value vector of at least one hot word in the initial hot word list;

[0022] The hot word processing module is used to compare the value vector of the at least one target hot word with the initial hot word value vector in sequence;

[0023] The hot word processing module is configured to add the hot word corresponding to the initial hot word value vector to the target hot word list when the value vector of the at least one target hot word is the same as the initial hot word value vector.

[0024] In some embodiments, the decoder module is used to decode the hidden layer output vector through a decoder network of the speech recognition model to obtain an initial speech recognition result corresponding to the speech information;

[0025] The decoder module is used to decode the second key vector and the second value vector through the decoder network of the speech recognition model to obtain a target hot word recognition result;

[0026] The decoder module is used to update the initial speech recognition result according to the target hot word recognition result to obtain the speech recognition result corresponding to the speech information to be recognized.

[0027] In some embodiments, the decoder network of the speech recognition model includes: a hot word import module and at least one decoding module;

[0028] The decoder module is used to obtain the historical text output feature vector of the decoder network according to the historical output result of the decoder network when the hot word import module is located in the first layer of the decoder network and is connected to a decoding layer;

[0029] The decoder module is used to use the hot word import module in the decoder network to perform a dot product process on the historical text output feature vector and the second key vector to obtain a second dot product result;

[0030] The decoder module is configured to scale the second dot product result according to the dimension of the second key vector to obtain a second scaled dot product result;

[0031] The decoder module is used to normalize the second scaling point product result to obtain a second normalized point product result;

[0032] The decoder module is configured to determine a product of the second normalized dot product result and the second value vector;

[0033] The decoder module is used to use the at least one decoding module to decode the product of the second normalized dot product result and the second value vector to obtain the target hot word recognition result.

[0034] In some embodiments, the decoder module is used to obtain the output vector of the decoding module of the nth layer when the hot word import module is located at the n+1th layer of the decoder network, where n is a positive integer greater than or equal to 1;

[0035] The decoder module is used to use the hot word import module in the decoder network to perform a dot product process on the output vector of the previous decoding layer of the hot word import module and the second key vector to obtain a third dot product result;

[0036] The decoder module is configured to scale the third dot product result according to the dimension of the second key vector to obtain a third scaled dot product result;

[0037] The decoder module is used to normalize the third scaling point product result to obtain a third normalized point product result;

[0038] The decoder module is configured to determine a product of the third normalized dot product result and the second value vector;

[0039] The decoder module is used for, when the hot word import module is the 2nth layer of the decoder network, using the hot word import module to decode the product of the third normalized dot product result and the second value vector to obtain the target hot word recognition result; or

[0040] The decoder module is used to decode the product of the third normalized dot product result and the second value vector using the decoding module of any layer from the 2nd layer to the 2n-1 layer connected to the hot word import module when the hot word import module is any layer from the 1st layer to the 2n-1 layer of the decoder network to obtain the target hot word recognition result.

[0041] In some embodiments, the decoder module is used to determine the position of at least one target hot word in the initial speech recognition result;

[0042] The decoder module is used to obtain at least one initial recognition vocabulary in the initial speech recognition result according to the position of the at least one target hot word;

[0043] The decoder module is used to perform consistency detection on at least one initial recognition word in the initial speech recognition result and the target hot word recognition result;

[0044] The decoder module is used to update all target hot words in the initial speech recognition result through the target hot word recognition result when any initial recognition vocabulary in the initial speech recognition result is inconsistent with the target hot word recognition result, so as to obtain the speech recognition result corresponding to the speech information to be recognized.

[0045] In some embodiments, the hot word processing module is used to obtain at least one hot word to be processed in the hot word list;

[0046] Performing voice expansion on the at least one hot word to be processed to obtain an extended vocabulary matching the at least one hot word to be processed, wherein the extended vocabulary includes at least one of the following:

[0047] Synonyms, antonyms, near synonyms and similar expressions;

[0048] The expanded vocabulary is added to the hot word list to obtain a target hot word list.

[0049] The present application also provides an electronic device, the electronic device comprising:

[0050] A memory for storing executable instructions;

[0051] The processor is used to implement the preceding speech recognition method when running the executable instructions stored in the memory.

[0052] The embodiment of the present application also provides a computer-readable storage medium storing executable instructions, which implement the above-mentioned speech recognition method when executed by a processor.

[0053] An embodiment of the present application also provides a computer program product, including a computer program or instructions, which implement the above-mentioned speech recognition method when executed by a processor.

[0054] The embodiments of the present application have the following beneficial effects:

[0055] 1) In an embodiment of the present application, a first key vector and a first value vector of at least one hot word in an initial hot word list are obtained, wherein the hot word is used to represent a word that appears more than a threshold number of times within a preset time range; voice information to be recognized is obtained, the voice information to be recognized is encoded, and a hidden layer output vector of the voice information to be recognized is obtained; based on an attention mechanism, the initial hot word list is filtered using the hidden layer output vector, the first key vector, and the first value vector to obtain a target hot word list; thus, the hidden layer output vector can be used to accurately recognize all hot words in the voice information, and at the same time, by using the initial hot word list, the hot words can be made highly extensible, not limited to fixed words, and support any customized hot words and their extensions. In addition, hot words can be added in real time at the sentence level without having to rebuild or add any modules to the speech recognition model, and the calculation speed of the speech recognition model is not affected.

[0056] 2) In the speech recognition stage, by obtaining the second key vector and the second value vector of at least one target hot word in the target hot word list; decoding the second key vector, the second value vector and the hidden layer output vector, a speech recognition result corresponding to the speech information to be recognized is obtained, wherein the speech recognition result includes at least one target hot word in the target hot word list. In this way, multiple hot words in the speech information can be accurately recognized, and the accuracy of the speech recognition result can be improved. At the same time, for the speech recognition of a large number of hot words, the use of the target hot word list can save the computing resources of the hardware equipment, speed up the speech recognition speed, reduce the deployment difficulty, and is suitable for use in low-resource devices, making the hardware cost of speech recognition lower. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a usage scenario of the speech recognition method provided in an embodiment of the present application;

[0058] Figure 2 An optional flowchart of the speech recognition method provided in the embodiment of the present application;

[0059] Figure 3a This is a schematic diagram of the working process of the encoder network in an embodiment of the present application;

[0060] Figure 3b Schematic diagram of the model structure of the hot word filtering module in the embodiment of the present application;

[0061] Figure 3c This is a schematic diagram of the working process of the decoder network in the embodiment of the present application;

[0062] Figure 4 An optional flowchart of the speech recognition method provided in the embodiment of the present application;

[0063] Figure 5 A schematic diagram of the structure of a decoder network in an embodiment of the present application;

[0064] Figure 6 A schematic diagram of the structure of a decoder network in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of the structure of a decoder network in an embodiment of the present application;

[0066] Figure 8 This is a process diagram of the speech recognition method in the embodiment of the present application;

[0067] Fig. 9 Schematic diagram of the model structure of the speech recognition model in the embodiment of the present application;

[0068] Fig.10 This is a schematic diagram of the structure of the hot word filtering network in the embodiment of the present application;

[0069] Fig.11 A schematic diagram of a speech recognition model with hot words imported into the network in an embodiment of the present application;

[0070] Fig.12 It is a structural diagram of an electronic device 100 provided in an embodiment of the present application for executing the speech recognition method provided in the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0072] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0073] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0074] 1) Speech Recognition (SR): Also known as Automatic Speech Recognition (ASR Automatic Speech Recognition), Computer Speech Recognition (CSR Computer Speech Recognition) or Speech To Text (STT Speech To Text), its goal is to use computers to automatically convert human speech content into corresponding text.

[0075] 2) Pre-trained language representation model (Bidirectional Encoder Representation from Transformers, BERT), a language model training method that uses massive text. This method is widely used in a variety of natural language processing tasks, such as text classification, text matching, machine reading comprehension, etc.

[0076] 3) Artificial Neural Network (ANN), also known as Neural Network (NN) for short, is a mathematical model or computational model that imitates the structure and function of biological neural networks (the central nervous system of animals, especially the brain) in the field of machine learning and cognitive science, and is used to estimate or approximate functions.

[0077] 4) Model parameters: It is a quantity that uses universal variables to establish the relationship between functions and variables. In artificial neural networks, model parameters are usually real number matrices.

[0078] 5) Word segmentation library: also known as word segmentation library, refers to a specific word segmentation method. Different word segmentation libraries have corresponding word dictionaries, and can perform word segmentation on the corresponding text information according to their corresponding word dictionaries.

[0079] 6) Model training, multi-classification learning of image data sets. The model can be built using deep learning frameworks such as Tensor Flow and torch, using multiple layers of neural network layers such as CNN to form a multi-classification model, and finally output the speech recognition results through algorithms such as softmax. During training, the model approaches the correct trend through objective functions such as cross entropy.

[0080] 7) Encoder-decoder structure: A network structure commonly used in machine translation technology. It consists of two parts: an encoder and a decoder. The encoder converts the input text into a series of context vectors that can express the characteristics of the input text. The decoder receives the output of the encoder as its own input and outputs the corresponding text sequence in another language.

[0081] 8) Hot words: When speech recognition is actually used, we often encounter some words that do not appear very often in the training set, such as place names, personal names, proprietary words, specific expressions, etc. Therefore, hot words are used to represent words that appear more than a threshold number of times within a preset time range (for example, words that appear more than or equal to 55 times in the target network within 24 hours).

[0082] 9) Token: Before any actual processing of the input text, it needs to be segmented into language units such as words, punctuation marks, numbers, or pure alphanumeric characters. These units are called tokens.

[0083] 10) Softmax: Normalized exponential function, which is a generalization of the logic function. It can "compress" a K-dimensional vector containing any real number into another K-dimensional real vector, so that each element is between [0, 1] and the sum of all elements is 1.

[0084] 11) Transformers: A new network structure that uses an attention mechanism to replace the traditional encoder-decoder model that must rely on other neural networks.

[0085] During the research, the inventors found that the speech recognition model provided in the related art has the following defects when performing speech recognition containing a large number of hot words:

[0086] Related technology: During the training process of the speech recognition model, the scenario corresponding to the corpus used in the speech recognition training is likely to be inconsistent with the actual scenario, and the actual scenario may contain words that do not appear or appear very rarely in the training corpus (such as hot words), resulting in poor actual use effect of the system.

[0087] Among them, hot words refer to words that are used frequently in a certain period of time. They often have the characteristics of the times and reflect the hot topics of a period. Hot words are usually closely related to social events or phenomena and become a tool for expressing public opinion and public opinion supervision. How to obtain hot words has become a challenging topic. In order to obtain hot words, hot word discovery technology came into being. Hot word discovery technology can be applied to many tasks such as Chinese word segmentation, Chinese information retrieval, input method, public opinion analysis, etc. When determining whether a word in a voice message is a hot word, a word that appears more than a threshold number of times within a preset time range can be determined as a hot word.

[0088] In related technologies, the main technical means to solve this problem is to organize specific scenes or specific words into a hot word list and increase its weight in the speech recognition model. The speech recognition model is a model used to describe the possibility of a sentence. When the weight of a specific word in the speech recognition model is increased, it is more likely to appear, thus compensating for the problem of hot words not being able to be recognized to a certain extent.

[0089] In the related art, the existing speech recognition models have defects in computational flexibility, computational speed and scalability when processing hot words in speech information. Specifically, 1) whenever a new hot word is added to the speech recognition model, the speech recognition model must be modified again and the decoder based on the speech recognition model must be recompiled, which is time-consuming and easily limited in real-time performance and flexibility; 2) when a large number of hot words are added, since there are not many hot words involved in the recognition of each sentence, but the weights of all hot words in the speech recognition model are increased, it will affect the recognition of non-hot words, and the scalability is seriously insufficient; in addition, new hot word types are limited to manual design, and there may be omissions or deficiencies, such as synonyms, antonyms, and similar expressions of these hot words, which are deficiencies in scalability.

[0090] In order to solve the above defects, the embodiment of the present application provides a speech recognition method, which can accurately recognize hot words in speech information, and at the same time, there is no need to adjust the sample weights in the speech recognition model training process. Figure 1 , Figure 1 A schematic diagram of a usage scenario of a voice information recognition method is provided for an embodiment of the present application. The electronic device 100 may include two different types of terminals or electronic devices, namely, terminal 10-1 and terminal 10-2. The target object may be selected according to the usage requirements of different voice recognition scenarios. Terminal 10-1 and terminal 10-2 are provided with a client or applet capable of performing a voice recognition function, such as an independent application running a voice recognition model, or an applet supporting a voice recognition model. Through the voice recognition device deployed in terminal 10-1 and terminal 10-2, the hot words in the voice information can be accurately recognized through the voice recognition model provided in the embodiment of the present application.

[0091] In some embodiments, a client that can set up voice recognition software is set on the terminal (including terminal 10-1 and terminal 10-2), and the target object can input the corresponding sentence to be voice recognized through the set voice recognition software client. In some embodiments, the artificial intelligence client (such as the voice assistant applet in the instant messaging client) can also send the voice information to be recognized to the server 200, and receive the voice recognition result sent by the server 200, and display the received voice recognition result to the target object. The target object can perform accuracy detection on the hot words in the voice recognition result. Terminals 10-1 and 10-2 are connected to server 200 via network 300. Network 300 can be a wide area network or a local area network, or a combination of the two. Wireless links are used to realize data transmission. The voice information recognition method provided in the embodiment of the present application can be used as a cloud service to serve various types of customers (packaged in a vehicle-mounted terminal or in different mobile electronic devices). The present application does not impose specific restrictions on this. When provided as a cloud service to corporate customers, the voice recognition model can be trained in the cloud server, and the hot word filtering module can be trained. The trained hot word filtering module can be used as a component of the voice recognition model, or it can be packaged as an independent process. The present application does not impose specific restrictions on this.

[0092] In some embodiments, the server 200 is used to deploy a speech recognition model and train the speech recognition model, and deploy the trained speech recognition model in the corresponding terminal 10-1 and / or 10-2, and use the trained speech recognition model through the terminal 10-1 and / or the terminal 10-2 to recognize voice instructions that may carry hot words, and execute voice instructions with hot words through the terminal 10-1 and / or the terminal 10-2, for example: the voice command "contact the address book VIP user", where the hot word is "vip" and the hot word indicates the address book user.

[0093] Among them, the speech information recognition method provided in the embodiment of the present application is based on artificial intelligence. Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0094] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0095] In the embodiments of the present application, the artificial intelligence software technologies mainly involved include the above-mentioned speech processing technology and machine learning. For example, it may involve speech recognition technology (Automatic Speech Recognition, ASR) in speech technology (Speech Technology), including speech signal preprocessing (Speech signal preprocessing), speech signal frequency domain analysis (Speech signal frequency analyzing), speech signal feature extraction (Speech signal feature extraction), speech signal feature matching / recognition (Speech signal feature matching / recognition), speech training (Speech training), etc.

[0096] For example, it may involve machine learning (ML), which is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning usually includes technologies such as deep learning. Deep learning includes artificial neural networks, such as convolutional neural networks (CNN), recurrent neural networks (RNN), deep neural networks (DNN), etc.

[0097] See also Figure 2 , Figure 2 A schematic diagram of an implementation flow of a speech recognition method provided for an embodiment of the present application, wherein the embodiment of the present application can be implemented in combination with cloud technology, which refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize data calculation, storage, processing, and sharing. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology, and application technology based on cloud computing business model applications. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites, so cloud technology needs to be supported by cloud computing.

[0098] It should be noted that cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called the "cloud". From the user's point of view, the resources in the "cloud" can be infinitely expanded, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. As a basic capability provider of cloud computing, a cloud computing resource pool platform will be established, referred to as a cloud platform, generally referred to as Infrastructure as a Service (IaaS), and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (which can be virtualized machines, including operating systems), storage devices and network devices. Different operation permissions are granted to the target objects that process data through cloud servers.

[0099] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of storage devices of various types (storage devices are also called storage nodes) in the network to work together through application software or application interfaces to provide data storage and business access functions to the outside world. In the related art, the storage method of the storage system is: create a logical volume. When creating a logical volume, a physical storage space is allocated to each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, the data is stored on the file system. The file system divides the data into many parts, each of which is an object. The object not only contains data but also contains additional information such as data identification (IDentity, ID). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object. Therefore, when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object. The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of the Redundant Array of Independent Disks (RAID), the physical storage space is divided into stripes in advance. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0100] Understandably, Figure 2 The execution subject of the steps shown can be implemented by a terminal running a voice recognition device alone, or by a server running a voice recognition device. Figure 2 The processing steps shown in the figure are explained by taking the terminal as the execution subject. Figure 2 The steps shown are explained.

[0101] Step 201: Obtain a first key vector and a first value vector of at least one hot word in an initial hot word list.

[0102] Among them, the speech recognition method provided in the embodiment of the present application can be implemented by improving the model of the speech recognition model in the related technology. For example, the encoder network of the speech recognition model can be derived from the encoder network provided by the Transformer / Conformer structure, and the decoder network of the speech recognition model can be derived from the decoder network provided by the Transformer structure. This application does not make specific restrictions. When using the speech recognition model to recognize speech information, the decoder network uses the attention mechanism to retrieve the hot word list. Considering the existence of hot words that are not related to the current recognition sentence, the recognition results of the encoder network of the speech recognition model are preliminarily screened and filtered to improve the accuracy of hot words.

[0103] In step 201, hot words are used to characterize words that appear more than a threshold number of times within a preset time range. The initial hot word list can come from a hot word expansion module, which can collect hot words that appear on the network. For example, when it is determined that the target word "XXXX" appears more than 100 times in the historical speech recognition results of the server within 24 hours, it can be determined that the type of the target word "XXXX" is a hot word, and the hot word expansion module can add the target word "XXXX" to the initial hot word list. Among them, the hot words in the initial hot word list can be words such as names of people in the user's address book, music names, place names, etc. When the initial hot word list is empty, an additional hot word list needs to be added. <nobias>The label indicates that there is no hot word in the initial hot word list. When there are two hot words "a" and "b" in the initial hot word list, the text information of hot words "a" and "b" is firstly characterized to obtain text feature vectors "A" and "B". Then, the key vector k of the text feature vector "A" is obtained by linearly transforming the text feature vectors. a Sum value vector v a , and the key vector k of the text feature vector "B" b Sum value vector v b Since both the first key vector and the first value vector can be represented by a vector group, the key vectors of the text feature vectors "A" and "B" can be combined to obtain the first key vector [k a , k b ], the value vectors of the text feature vectors "A" and "B" are combined to obtain the first value vector [v a , v b ].

[0104] As the target language changes, the target vocabulary "XXXX" may have different variations. In order to make the initial hot word list cover a sufficient number of hot words to ensure the accuracy of the speech recognition results, it is necessary to perform speech expansion on the hot words to be processed so that the initial hot word list can be richer.

[0105] In some embodiments, at least one hot word to be processed is voice expanded to obtain an extended vocabulary that matches the at least one hot word to be processed, wherein the extended vocabulary includes at least one of the following: synonyms, antonyms, near synonyms and similar expressions; the extended vocabulary is added to the hot word list to obtain a target hot word list. Among them, voice expansion of the target word "out of the circle" can obtain the synonym "famous", the antonym "dormant", the near synonym and similar expression "upper position". At this time, the number of hot words in the initial hot word list increases from 1 to 4, and the number of hot words increases by 4 times, which can make the hot words highly extensible, not limited to fixed vocabulary, and support any customized hot words and their extended words. At the same time, the voice expansion of hot words can realize the real-time addition of hot words at the sentence level, that is, each sentence can update the hot word list in a targeted and real-time manner, without rebuilding any module, and does not affect the calculation speed of the speech recognition model.

[0106] In some embodiments, semantic expansion is performed on at least one hot word to be processed to obtain an expanded vocabulary matching the at least one hot word to be processed, which can be achieved by:

[0107] 1) Performing manual semantic expansion on at least one hot word to be processed, and performing compliance detection on the result of the manual semantic expansion to obtain an extended vocabulary matching at least one hot word to be processed; during the manual semantic expansion, the extended vocabulary of the hot word to be processed may not comply with the legal and regulatory requirements in the application scenario of the speech recognition model. Therefore, by performing compliance detection, the speech recognition model can be effectively avoided from processing illegal vocabulary, so that the speech recognition result complies with the legal and regulatory requirements in the application scenario of the speech recognition model.

[0108] 2) Automatically semantically expand at least one hot word to be processed through a language processing model, and remove duplicates from the results of the automatic semantic expansion to obtain an extended vocabulary that matches at least one hot word to be processed. When performing automatic semantic expansion, the extended vocabulary that matches the hot word to be processed can be checked for compliance through a preset regular matching rule, but the result of the automatic semantic expansion may contain duplicate extended vocabulary. In order to avoid redundancy in the initial hot word list, it is necessary to remove duplicates from the results of the automatic semantic expansion to avoid the lengthy initial hot word list from affecting the processing speed of speech recognition during the speech recognition process.

[0109] Step 202: Acquire the speech information to be recognized, encode the speech information to be recognized, and obtain a hidden layer output vector of the speech information to be recognized.

[0110] In some embodiments, obtaining the voice information to be recognized can be achieved by:

[0111] According to the recognition environment of the target voice information, the voice parameter information carried by the target voice information is extracted, and the corresponding word segmentation library is triggered; the target voice information is segmented through the word dictionary of the triggered word segmentation library, and Chinese character text is extracted to form different word-level feature vectors; different word-level feature vectors are denoised to obtain a word-level feature vector set corresponding to the target voice information. In the preprocessing process, the Arabic numerals in the target voice information may not be converted into Chinese characters, and only conversions unrelated to the numbers are performed, such as retaining the original form of the Arabic numerals in the sentence, deleting irrelevant information such as the phonetic particles or modal particles of the target voice information, and at the same time, the abbreviations of the international system of units connected to the numbers, such as the units of weight g, kg, the units of length cm, m, and the units of time s, ms, etc., are not converted and retained in their original state. For Chinese text, it is necessary to segment the Chinese text accordingly, because only words can contain complete information in Chinese. Correspondingly, the Chinese word segmentation tool Jieba can be used to segment the Chinese text. Among them, "this thing happened in the year 2021", after word segmentation, becomes "this / thing / happened / in / two / zero / two / one / year". Among them, the so-called word segmentation has both verbal meaning and noun meaning; each word segmentation is a word or phrase, that is, the smallest semantic unit with a certain meaning; for different users or different use environments of speech information recognition models received, the smallest semantic units contained therein need to be divided differently, and adjustments need to be made in a timely manner. This process is called word segmentation, that is, word segmentation can refer to the above-mentioned process of dividing the smallest semantic unit; on the other hand, the smallest semantic unit obtained after division is often also called word segmentation, that is, the word obtained after the word segmentation operation is performed; sometimes in order to distinguish two meanings from each other, the smallest semantic unit referred to by the latter meaning is called a word segmentation object (Term); in this application, the name of word segmentation object is used; the word segmentation object corresponds to the keyword used as the index basis in the inverted list. For Chinese, since the word as the smallest semantic unit is often composed of different numbers of characters, there is no natural distinguishing mark in phonetic characters such as blank partitions between words. Therefore, for Chinese, accurate word segmentation to obtain a reasonable word segmentation object is an important step. During word segmentation processing, different types of word segmentation dictionaries such as medical, professional text, news, entertainment reading, live broadcast, and games can be triggered according to different usage environments to obtain more accurate word segmentation results, making the voice information to be recognized more concise.

[0112] In some embodiments, the speech information to be recognized is encoded by the encoder network of the speech recognition model, and finally output by the encoder network in the form of a hidden layer output vector, and processed by the hot word filtering module and the decoder network to obtain a speech recognition result carrying the hot words.

[0113] In some embodiments, when encoding the speech information to be recognized, reference is made to Figure 3a , Figure 3a Schematic diagram of the working process of the encoder network in the embodiment of the present application, wherein the encoder network includes 6 identical decoder layers ( Figure 3a 2 decoder layers are shown in FIG. 2 ). After the speech information entering the first encoding layer passes through the 6 encoder layers in the encoder network, the hidden layer output vector is output. Figure 3a As shown in the figure, x1 is processed by the attention mechanism to the state of z1, and x2 is processed by the attention mechanism to the state of z2. The tensor that has passed the attention mechanism needs to be processed by the residual network and linear transformation, and then enter the fully connected feedforward network. The feedforward network needs to perform the same operation, residual processing and linear transformation. Finally, the output tensor can enter the next encoding layer, and then this operation is iterated for 6 times to obtain the hidden layer output vector.

[0114] Step 203: Based on the attention mechanism, the initial hot word list is filtered using the hidden layer output vector, the first key vector and the first value vector to obtain a target hot word list.

[0115] In some embodiments, step 203 may be implemented by a hot word filtering module, which may be deployed in the terminal alone or as part of a speech recognition model deployed on the terminal.

[0116] refer to Figure 3b , Figure 3b This is a schematic diagram of the model structure of the hot word filtering module in the embodiment of the present application, wherein the hot word filtering module is based on the attention mechanism to filter the initial hot word list to obtain the target hot word list. The vectors input to the hot word filtering module include: hidden layer output vector (Query, abbreviated as Q), the first key vector (Key abbreviated as K) and the first value vector (Value abbreviated as V). Among them, the first key vector (Key) and the first value vector (Value) are both vector groups. In some embodiments, filtering the initial hot word list through the hot word filtering module to obtain the target hot word list can be achieved in the following ways:

[0117] Using the hot word filtering module, the hidden layer output vector (Query) and the first key vector (Key) are multiplied by points to obtain the first point product result; where Query multiplied by Key can be expressed as: Q*K T ;

[0118] According to the dimension of the first key vector, the first dot product result is scaled to obtain a first scaled dot product result; wherein the dimension of the first key vector is d k , scaling the first point multiplication result can be achieved by formula (1):

[0119]

[0120] Among them, dk represents the dimension of the first key vector, Q is the hidden layer output vector, K is the first key vector, K T is the transpose of the first key vector.

[0121] The first scaling dot product result is normalized to obtain a first normalized dot product result; the product of the first normalized dot product result and the first value vector (Value) is determined as a reference value vector; wherein the reference value vector is calculated with reference to formula 2:

[0122]

[0123] Among them, Attention(Q, K, V) represents the reference value vector, softmax() represents the normalization process, and d k represents the dimension of the first key vector, Q is the hidden layer output vector, K is the first key vector, K T is the transpose of the first key vector and V is the first value vector.

[0124] The initial hot word list is filtered according to the reference value vector to obtain a target hot word list, wherein the target hot word list includes the target hot words corresponding to the reference value vector.

[0125] In some embodiments, filtering the initial hot word list according to the reference value vector to obtain the target hot word list can be achieved in the following manner:

[0126] Obtain the initial hot word value vector of at least one hot word in the initial hot word list; compare the value vector of at least one target hot word with the initial hot word value vector in turn; when the value vector of at least one target hot word is the same as the initial hot word value vector, add the hot word corresponding to the initial hot word value vector to the target hot word list. Among them, by comparing the value vector of at least one target hot word with the initial hot word value vector in turn, the traversal of the initial hot word list is realized, avoiding the omission of hot words in the speech information to be recognized, so that the speech recognition result is more accurate. At the same time, the position of the hot word filtering module includes two cases: 1) The hot word filtering module does not need to be inserted into the trained speech recognition model. The beneficial technical effect of this method is that since the hot word filtering module does not need to be inserted into the trained speech recognition model, it can be independent of the existing speech recognition model. While improving the speech recognition performance, it saves the computing resources of the hardware equipment, speeds up the speech recognition speed, reduces the deployment difficulty, and is more suitable for low-resource equipment, making the hardware cost of speech recognition lower. 2) The hot word filtering module can be inserted between the encoder network and the decoder network of the trained speech recognition model. The beneficial technical effect of this approach is that by inserting the hot word filtering module into the speech recognition model, the speech recognition model can be more portable. At the same time, when the speech recognition model is deployed in a device with strong computing power, the speed of speech recognition is further improved.

[0127] Through the processing of the above steps 201 to 203, the hot words in the voice information to be recognized have been recognized by the hot word filtering module, and the voice information to be recognized needs to be further recognized through the decoder network of the voice recognition model to ensure that the recognition result of the voice information to be recognized is output completely and accurately.

[0128] Step 204: Obtain a second key vector and a second value vector of at least one target hot word in the target hot word list.

[0129] The hot words in the target hot word list can be the names of people in the user's address book, music names, place names, Internet buzzwords, etc. obtained after filtering the initial target hot word list. When the target hot word list is empty, an additional hot word list needs to be added. <nobias>The label indicates that the target hot word list is empty. When there are two hot words "c" and "d" in the target hot word list, the text information of the hot words "c" and "d" is firstly characterized to obtain the text feature vectors "C" and "D". Then, the key vector kc and value vector v of the text feature vector "C" are obtained by linearly transforming the text feature vectors. c , and the key vector k of the text feature vector "D" d Sum value vector v d Since both the second key vector and the second value vector can be represented in the form of vector groups, the key vector and value vector of the text feature vectors "C" and "D" can be combined to obtain the second key vector [k c , k d ] and the second value vector [v c , v d ].

[0130] Step 205: Decode the second key vector, the second value vector and the hidden layer output vector to obtain a speech recognition result corresponding to the speech information to be recognized, wherein the speech recognition result includes at least one target hot word in the target hot word list.

[0131] In some embodiments, see Figure 4 , Figure 4 An optional flow chart of the speech recognition method provided in the embodiment of the present application is provided. It can be understood that: Figure 4 The execution subject of the steps shown can be implemented by a terminal running a voice recognition device alone, or by a server running a voice recognition device. Figure 4 The processing steps shown in the figure are described by taking the terminal as the execution subject as an example. On the basis of executing steps 201 to 204, the following steps can be continued: Figure 4 Steps 2051 to 2053 are shown to obtain the speech recognition result corresponding to the speech information to be recognized. Figure 4 The steps shown are explained.

[0132] Step 2051: decode the hidden layer output vector through the decoder network of the speech recognition model to obtain the initial speech recognition result corresponding to the speech information.

[0133] refer to Figure 3c , Figure 3c Schematic diagram of the working process of the decoder network in the embodiment of the present application, wherein the input, output and decoding process of the decoder network can be expressed as:

[0134] Input: hidden layer output vector & the output of the decoder network corresponding to position i-1 (i.e. the historical output result of the decoder network); Output: the probability distribution of the output word at position i in the speech recognition result; the vector output by the last decoder layer of the decoder network will undergo linear processing and normalization. The role of the linear processing is to map the vector processed by the encoder layer into a logits vector, and then use the softmax function to transform the logits vector to obtain the corresponding probability value, and finally find the position of the maximum probability, which completes the output of the decoder network.

[0135] Step 2052: Decode the second key vector and the second value vector through the decoder network of the speech recognition model to obtain the target hot word recognition result.

[0136] In some embodiments, unlike the speech recognition model used in the related art, a hot word import module based on the attention mechanism needs to be inserted into the decoder network of the speech recognition model provided in the embodiment of the present application, which is used to import the target hot words in the target hot word list generated by the hot word filtering module into the decoder network for decoding processing. Therefore, the decoder network of the speech recognition model in the present application includes a hot word import module and at least one decoding module. Since the position of the hot word import module in the decoder network is not fixed, it specifically includes the following situations: 1) The hot word import module is located in the first layer of the decoder network; 2) The hot word import module is located in the middle layer of the decoder network; 3) The hot word import module is located in the 2nth layer of the decoder network, that is, the top layer of the decoder network.

[0137] Therefore, the working process of the decoder network of the speech recognition model is not exactly the same, which is explained below.

[0138] In some embodiments, reference Figure 5 , Figure 5 This is a structural diagram of the decoder network in the embodiment of the present application. When the hot word import module is located in the first layer of the decoder network and is connected to a decoding layer, the working process of the decoder network is as follows:

[0139] According to the historical output results of the decoder network, the historical text output feature vector of the decoder network is obtained, and the historical text output feature vector is used as Query2 (abbreviated as Q2); the hot word import module in the decoder network is used to perform dot multiplication on the historical text output feature vector and the second key vector Key2 (abbreviated as K2) to obtain the second dot multiplication result; it can be expressed as: Q2*K2 T , where K2 T is the transpose of the second key vector;

[0140] Then, the second dot product result is scaled according to the dimension of the second key vector to obtain a second scaled dot product result; wherein the dimension of the second key vector is d2, and the scaling of the second dot product result can be performed by (formula

[0141] Formula 3) realizes:

[0142]

[0143] Among them, Attention(Q, K, V) represents the reference value vector, softmax() represents the normalization process, and d 2k represents the dimension of the second key vector, Q2 is the historical text output feature vector, K2 is the second key vector, K2 T is the transpose of the second key vector and V2 is the second value vector.

[0144] The second scaled dot product result is normalized to obtain a second normalized dot product result; the product of the second normalized dot product result and the second value vector (Value2) is determined; and the product of the second normalized dot product result and the second value vector (Value2) is decoded using at least one decoding module to obtain a target hot word recognition result.

[0145] In some embodiments, reference Figure 6 , Figure 6 : is a structural diagram of a decoder network in an embodiment of the present application, wherein the decoder network has 2n layers, the 2n layer is the top layer of the decoder network, when the hot word import module is located at the n+1th layer of the decoder network, the output vector of the decoding module at the nth layer is obtained, n is a positive integer greater than or equal to 1, and the hot word import module is the 2nth layer of the decoder network, the working process of the decoder network is as follows:

[0146] Using the hot word import module in the decoder network, the output vector of the previous decoding layer of the hot word import module and the second key vector are processed by point multiplication to obtain a third point product result; at this time, the output vector of the previous decoding layer of the hot word import module can be used as Query2 in Formula 3; according to the dimension of the second key vector (Key2), the third dot product result is scaled to obtain a third scaled dot product result; the third scaled dot product result is normalized to obtain a third normalized dot product result; the product of the third normalized dot product result and the second value vector (Value2) is determined; the hot word import module is used to decode the product of the third normalized dot product result and the second value vector (Value2) to obtain a target hot word recognition result.

[0147] In some embodiments, reference Figure 7 , Figure 7 It is a structural diagram of the decoder network in the embodiment of the present application. When the hot word import module is located at the n+1th layer of the decoder network, the output vector of the decoding module of the nth layer is obtained, n is a positive integer greater than or equal to 1, and the hot word import module is any layer from the 2nd layer to the 2n-1th layer of the decoder network, including the following cases: 1) When the decoder layer is the 2nth layer (i.e., the top layer) of the decoder network, the hot word import module is located at the corresponding position from the 2nd layer to the 2n-1th layer of the decoder network. At this time, the input and output ends of the hot word import module are respectively connected to a decoder layer. For example, n is 2, at this time, the decoder network has 4 layers, the 4th layer is the decoder layer, the 3rd layer is the hot word import module, the 2nd layer is the decoder layer, and the 1st layer is the decoder layer; or, n is 2, at this time, the decoder network has 4 layers, the 4th layer is the decoder layer, the 3rd layer is the decoder layer, the 2nd layer is the hot word import module, and the 1st layer is the decoder layer.

[0148] 2) The hot word import module is located at the n+1th layer of the decoder network, the decoder layer is the nth layer of the decoder network, the decoder layer is the n+2th layer of the decoder network, and the decoder layer is the n+3th layer of the decoder network. At this time, the input and output ends of the hot word import module are respectively connected to a decoder layer (nth layer and n+2th layer), and the decoder layer located at the n+3th layer can continue to be connected to at least one decoder layer.

[0149] For different positions of the hot word import module above, the working process of the decoder network is as follows:

[0150] Using the hot word import module in the decoder network, the output vector of the previous decoding layer of the hot word import module and the second key vector are processed by point multiplication to obtain a third point product result; at this time, the output vector of the previous decoding layer of the hot word import module can be used as Query2 in Formula 3; then, according to the dimension of the second key vector (Key2), the third point product result is scaled to obtain a third scaled point product result; the third scaled point product result is normalized to obtain a third normalized point product result; the product of the third normalized point product result and the second value vector (Value2) is determined; the decoder layer located at the 2nth layer is used to decode the product of the third normalized point product result and the second value vector (Value2) to obtain the target hot word recognition result.

[0151] pass Figures 5 to 7 The model structures of the three decoder networks shown can all obtain target hot word recognition results, and then the initial speech recognition results need to be updated using the target hot word recognition results through step 2503.

[0152] Step 2053: Update the initial speech recognition result according to the target hot word recognition result to obtain the speech recognition result corresponding to the speech information to be recognized.

[0153] In some embodiments, updating the initial speech recognition result according to the target hot word list recognition result can be achieved in the following ways:

[0154] Determine the position of at least one target hot word in the initial speech recognition result; obtain at least one initial recognition vocabulary in the initial speech recognition result according to the position of at least one target hot word; wherein the initial speech recognition result includes multiple vocabulary positions, each vocabulary position may correspond to an initial recognition vocabulary, and the initial recognition vocabulary corresponding to each vocabulary position in the initial speech recognition result may include two situations, namely 1) the initial recognition vocabulary is the target hot word, and 2) the initial recognition vocabulary is a non-target hot word.

[0155] When the decoder network decodes the hidden layer output vector, each time the decoder network completes a round of decoding processing, it can find the decoding result with the maximum probability corresponding to the word position; in the decoding process, for the hidden layer output vector and the second key vector and the second value vector, when the second key vector and the second value vector are decoded, the position corresponding to the decoding result is the position of the target hot word when the hidden layer output vector is decoded. According to the position of the target hot word, the corresponding position identifier can be obtained, so as to use the position identifier to find the corresponding initial recognition vocabulary in the initial speech recognition result. For example: the second key vector of the target hot word "out of the circle" is [k 出圈 ] and the second value vector is [v 出圈 ], the decoding result is "out of the circle", in the initial speech recognition result "Zhang San (position i-1) / out of the circle (position i) / le (position i+1)" when decoding the hidden layer output vector, position i is the position of the target hot word, and the initial speech recognition result of position i is "out of the circle" (it may also be the initial speech recognition result "out of the punch" or "out of the roll").

[0156] Afterwards, at least one of the initial recognition words in the initial speech recognition result and the target hot word recognition result are checked for consistency; when any of the initial recognition words in the initial speech recognition result and the target hot word recognition result are inconsistent, all the target hot words in the initial speech recognition result are updated through the target hot word recognition result to obtain the speech recognition result corresponding to the speech information to be recognized. Among them, when any of the initial recognition words in the initial speech recognition result and the target hot word recognition result are inconsistent, it may include the following situations: 1) there is only one initial recognition word in the initial speech recognition result; 2) there are at least 2 identical initial recognition words in the initial speech recognition result; 3) there are at least 2 different initial recognition words in the initial speech recognition result, which are explained below respectively:

[0157] 1) There is only one initial recognition word in the initial speech recognition result.

[0158] For example: In the initial speech recognition result, the initial recognized word in the target hot word position AA"xx”BB is "generating test papers", and the recognized result of the target hot word is "going viral" (the pronunciation is the same, but the semantics are different). Therefore, the initial recognized word and the recognized result of the target hot word are inconsistent and need to be replaced.

[0159] 2) There are at least two identical initial recognized words in the initial speech recognition result.

[0160] For example: In the initial speech recognition result, a) the initial recognized word in the target hot word position AA"xx”BB is "generating test papers", and b) the initial recognized word in the target hot word position cc"xx”dd is "going viral"; the recognized result of the target hot word is "going viral" (the pronunciation is the same, but the semantics are different). Therefore, when the initial recognized word in the target hot word position AA"xx”BB is different from the recognized result of the target hot word, it can be directly replaced by the recognized result of the target hot word to save the time consumed by speech recognition.

[0161] 3) There are at least two different initial recognized words in the initial speech recognition result.

[0162] For example: In the initial speech recognition result, a) the initial recognized word in the target hot word position AA"xx”BB is "generating test papers", and b) the initial recognized word in the target hot word position cc"xx”dd is "lying flat"; the recognized results of the target hot words are "going viral" and "lying flat" (the pronunciation is the same, but the semantics are different). Therefore, when the initial recognized word is different from the recognized result of the target hot word, the initial recognized word can be replaced by the recognized result of the target hot word in sequence to improve the accuracy of the speech recognition result.

[0163] To better illustrate the working process of the speech recognition method provided by the embodiments of the present application, the following takes the usage environment of video conference speech recognition as an example to illustrate the processing process of the speech recognition method provided by the embodiments of the present application. First, the following describes the defects in hot word recognition in the speech recognition scenario in the related art discovered by the inventors. The inventors found that in the training process of the speech recognition model, the scenario corresponding to the corpus used in speech recognition training is very likely to be inconsistent with the actual scenario, and the actual scenario may contain words that do not appear or appear very rarely in the training corpus, resulting in a deterioration of the actual use effect of the system.

[0164] In the speech recognition scenario of professional conferences, with the development of professional technologies, many technical hotspots will emerge, thus forming hot words used to represent a certain technical hotspot. For example, the professional term "quantum entanglement", but the speech recognition model in the related art cannot accurately recognize the hot word "quantum entanglement" and often recognizes it as "Liangzi Winery", resulting in users being unable to obtain accurate speech recognition results.

[0165] In related technologies, the main technical means to solve this problem is to organize specific voice conference scenarios or specific vocabulary "quantum entanglement" into a hot word list and increase its weight in the speech recognition model. The speech recognition model is a model used to describe the possibility of a sentence. When the weight of specific vocabulary in the speech recognition model is increased, it is more likely to appear, thus compensating for the problem of hot words not being able to be recognized to a certain extent.

[0166] In the related art, the existing speech recognition models have defects in computational flexibility, computational speed and scalability when processing hot words in speech information. Specifically, 1) whenever a new hot word is added to the speech recognition model, the speech recognition model must be modified again and the decoder based on the speech recognition model must be recompiled, which is time-consuming and easily limited in real-time performance and flexibility; 2) when a large number of hot words are added, since there are not many hot words involved in the recognition of each sentence, but the weights of all hot words in the speech recognition model are increased, it will affect the recognition of non-hot words, and the scalability is seriously insufficient; in addition, new hot word types are limited to manual design, and there may be omissions or deficiencies, such as synonyms, antonyms, and similar expressions of these hot words, which are deficiencies in scalability.

[0167] In order to solve the above defects, refer to Figure 8 , Figure 8 This is a process diagram of the speech recognition method in the embodiment of the present application. It can be understood that: Figure 8 The execution subject of the steps shown can be implemented by a terminal running a voice recognition device alone, or by a server running a voice recognition device. Figure 8 The processing steps shown are described by taking the terminal as the execution subject as an example, and specifically include the following steps:

[0168] Step 801: Obtain a first key vector and a first value vector of at least one hot word in an initial hot word list.

[0169] refer to Fig. 9 , Fig. 9 This is a schematic diagram of the model structure of the speech recognition model in the embodiment of the present application. Different from the existing speech recognition models in the related art, the speech recognition model provided in the embodiment of the present application includes: a hot word filtering network based on the attention mechanism, a hot word importing network based on the attention mechanism, and a hot word expansion network, wherein 1) the hot word filtering network based on the attention mechanism can add any number of hot words while maintaining accuracy; 2) the hot word importing network based on the attention mechanism further extracts relevant information from the filtered hot word representation. 3) The hot word expansion network expands the associated vocabulary for a given hot word; the following is combined with the description of the processing steps to explain the working process of the hot word filtering network based on the attention mechanism, the hot word importing network based on the attention mechanism, and the hot word expansion network.

[0170] Step 802: Acquire voice information to be recognized.

[0171] Among them, the encoder network of the speech recognition model can be derived from the encoder network provided by the Transformer / Conformer structure, and the decoder network of the speech recognition model can be derived from the decoder network provided by the Transformer structure. This application does not make any specific restrictions on this.

[0172] Step 803: Encode the speech information to be recognized to obtain a hidden layer output vector of the speech information to be recognized.

[0173] Among them, the encoder network of the speech recognition model can encode the speech information to be recognized and output the text results of CTC recognition in the form of hidden layer output vectors. By comparing the text results recognized by CTC with the hot word list, the hot words related to recognition can be screened out. However, this method is limited by the accuracy of CTC, that is, when the CTC recognition results are inaccurate, the screening of hot words is insufficient and inaccurate, so combined with Fig. 9 As shown, a hot word screening network (i.e., a hot word filtering network based on an attention mechanism) is added to the speech recognition model of the present application, and the hot word list is filtered using the hidden layer output vector to obtain a filtered hot word list. In this process, since the hidden layer output vector retains all possibilities of the speech information to be recognized, accurate hot word filtering can be achieved.

[0174] Before executing step 804, it is necessary to use hot words to expand the network and expand the existing hot words. Hot words, that is, popular vocabulary, as a lexical phenomenon reflects the issues and things that people in a region generally pay attention to in a period of time. For example, "lying flat" and "involution" that appeared in 2022 are all hot words. When determining whether the searched keyword is a hot word, the number of times a certain word is searched over a period of time can be counted. The word "medal list" is searched very frequently within a few days, so it can be identified as a hot word.

[0175] When using hot words to expand the network and expand existing hot words, it can be achieved in the following ways:

[0176] Obtain at least one hot word to be processed in the hot word list; perform semantic expansion on at least one hot word to be processed to obtain an extended vocabulary that matches at least one hot word to be processed, wherein the extended vocabulary includes at least one of the following: synonyms, antonyms, near synonyms, and similar expressions; add the extended vocabulary to the hot word list to obtain a target hot word list. In some embodiments of the present application, semantic expansion is performed to obtain an extended vocabulary that matches at least one hot word to be processed, including: performing manual semantic expansion on at least one hot word to be processed, and performing compliance detection on the result of the manual semantic expansion to obtain an extended vocabulary that matches at least one hot word to be processed; or performing automatic semantic expansion on at least one hot word to be processed through a language processing model, and removing duplicates from the result of the automatic semantic expansion to obtain an extended vocabulary that matches at least one hot word to be processed. In this way, the accuracy and comprehensiveness of the hot words can be guaranteed, and at the same time, the requirements of legal provisions can be met. Based on the semantic expansion of hot words, that is, based on the semantic expansion means, the associated vocabulary of a given limited hot word, such as synonyms, antonyms, similar expressions, etc., is produced, which greatly enriches the coverage of hot words and provides more accurate speech recognition results.

[0177] Step 804: Based on the attention mechanism, the hot word list is filtered using the hidden layer output vector, the first key vector and the first value vector to obtain a target hot word list.

[0178] In some embodiments, reference Fig.10 , Fig.10 This is a schematic diagram of the structure of the hot word filtering network in the embodiment of the present application, wherein the dot product of the hidden layer output vector and the first key vector is calculated through the hot word filtering network; the dot product of the hidden layer output vector and the first key vector is scaled according to the dimension of the first key vector to obtain the scaled dot product of the hidden layer output vector and the first key vector; the scaled dot product of the hidden layer output vector and the first key vector is normalized to obtain a first normalized scaled dot product; the product of the first normalized scaled dot product and the first value vector is calculated to obtain at least one value vector; the hot word list is filtered according to at least one value vector to obtain a target hot word list, wherein the target hot word list includes at least one target hot word. In the above process, if Figure 3b As shown in the figure, Q (Query) comes from the hidden layer output vector; K (Key vector) and V (Value vector) both come from the hot word list, and each word has been converted into a vector (the mapping list is learned as a parameter in model training), and K and V are both vector group modes; the output K' dimension of the hot word filtering network is the same as the hidden layer output vector Q.

[0179] Step 805: Obtain a second key vector and a second value vector of at least one target hot word in the target hot word list.

[0180] refer to Fig.11 , Fig.11 This is a schematic diagram of a speech recognition model with a hot word import network in an embodiment of the present application. During the training phase, it is necessary to adjust the structure of the decoder network of the speech recognition model to enable the hot word import network to be added to the decoder network of the speech recognition model.

[0181] Step 806: Decode the second key vector, the second value vector and the hidden layer output vector to obtain a speech recognition result corresponding to the speech information to be recognized, wherein the speech recognition result includes at least one target hot word in the target hot word list.

[0182] Combination Fig.11 As shown, the decoding process of the speech recognition model is as follows: through the decoder network of the speech recognition model, the hidden layer output vector is decoded to obtain the initial speech recognition result corresponding to the speech information; the historical text output feature vector of the speech information to be recognized is obtained; through the hot word import network, the dot product of the historical text output feature vector and the second key vector is calculated; according to the dimension of the second key vector, the dot product of the historical text output feature vector and the second key vector is scaled to obtain the scaled dot product of the historical text output feature vector and the second key vector; the scaled dot product of the historical text output feature vector and the second key vector is normalized to obtain the second normalized scaled dot product; the product of the second normalized scaled dot product and the second value vector is calculated to obtain the recognition result of the target hot word; according to the recognition result of the target hot word, the initial speech recognition result is modified to obtain the speech recognition result corresponding to the speech information to be recognized. Combined with the previous embodiment, the decoder network carrying the hot word import network uses a conventional attention mechanism, Q comes from the output of the previous network of the decoder, K and V come from the input K' of the hot word filtering network, and the output dimension is the same as Q.

[0183] pass Fig.11 The speech recognition model with hot words imported into the network is shown, and the execution Figure 8 The speech recognition steps shown do not rely on additional language models when identifying hot words in sentences, which improves speech recognition performance while saving computing resources of hardware devices, speeding up speech recognition and reducing deployment difficulty. It is particularly suitable for low-resource devices such as end-side devices (such as personal mobile phones), making the hardware cost of speech recognition lower.

[0184] At the same time, using hot words to expand the network can make the hot words highly scalable, not limited to fixed vocabulary, support any customized hot words and their expansions, and realize real-time addition of hot words at the sentence level, that is, each sentence can update the hot word list in real time in a targeted manner without rebuilding any module and without affecting the calculation speed of the speech recognition model.

[0185] Finally, the hot word filtering network and the hot word importing network can be used to accurately identify multiple hot words in voice information, improve the accuracy of voice recognition results, and enhance the user experience of the voice recognition model.

[0186] See also Fig.12 , Fig.12 is a structural diagram of an electronic device 100 provided in an embodiment of the present application for executing the speech recognition method provided in the present application, Fig.12 The electronic device 100 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the electronic device 100 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Fig.12 Various buses are labeled as bus system 440 .

[0187] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0188] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0189] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0190] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0191] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0192] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0193] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0194] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0195] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0196] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Fig.12 The speech recognition device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, including the following software modules: an encoder module 4081, a hot word processing module 4082, and a decoder module 4083. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0197] The functions of the various software modules in the speech recognition device 455 are described below, wherein:

[0198] Encoder module 4081, used to obtain a first key vector and a first value vector of at least one hot word in the initial hot word list, wherein the hot word is used to represent a word that appears more than a threshold number of times within a preset time range;

[0199] The encoder module 4081 is used to obtain the speech information to be recognized, encode the speech information to be recognized, and obtain the hidden layer output vector of the speech information to be recognized;

[0200] A hot word processing module 4082 is used to filter the initial hot word list based on the attention mechanism using the hidden layer output vector, the first key vector and the first value vector to obtain a target hot word list;

[0201] A hot word processing module 4082, configured to obtain a second key vector and a second value vector of at least one target hot word in the target hot word list;

[0202] The decoder module 4083 is used to decode the second key vector, the second value vector and the hidden layer output vector to obtain a speech recognition result corresponding to the speech information to be recognized, wherein the speech recognition result includes at least one target hot word in the target hot word list.

[0203] In some embodiments, the hot word processing module 4082 is used to use the hot word filtering module to perform a dot product process on the hidden layer output vector and the first key vector to obtain a first dot product result;

[0204] A hot word processing module 4082 is used to scale the first dot product result according to the dimension of the first key vector to obtain a first scaled dot product result;

[0205] A hot word processing module 4082 is used to perform normalization processing on the first scaling dot product result to obtain a first normalized dot product result;

[0206] A hot word processing module 4082 is used to determine the product of the first normalized dot product result and the first value vector as a reference value vector;

[0207] The hot word processing module 4082 is used to filter the initial hot word list according to the reference value vector to obtain a target hot word list, wherein the target hot word list includes the target hot words corresponding to the reference value vector.

[0208] In some embodiments, the hot word processing module 4082 is used to obtain an initial hot word value vector of at least one hot word in the initial hot word list;

[0209] A hot word processing module 4082 is used to compare the value vector of at least one target hot word with the initial hot word value vector in sequence;

[0210] The hot word processing module 4082 is used to add the hot word corresponding to the initial hot word value vector to the target hot word list when the value vector of at least one target hot word is the same as the initial hot word value vector.

[0211] In some embodiments, the decoder module 4083 is used to decode the hidden layer output vector through the decoder network of the speech recognition model to obtain an initial speech recognition result corresponding to the speech information;

[0212] A decoder module 4083 is used to decode the second key vector and the second value vector through a decoder network of the speech recognition model to obtain a target hot word recognition result;

[0213] The decoder module 4083 is used to update the initial speech recognition result according to the target hot word recognition result to obtain the speech recognition result corresponding to the speech information to be recognized.

[0214] In some embodiments, a decoder network of a speech recognition model includes: a hot word import module and at least one decoding module;

[0215] The decoder module 4083 is used to obtain the historical text output feature vector of the decoder network according to the historical output result of the decoder network when the hot word import module is located in the first layer of the decoder network and is connected to a decoding layer;

[0216] A decoder module 4083, used to use the hot word import module in the decoder network to perform a dot product process on the historical text output feature vector and the second key vector to obtain a second dot product result;

[0217] A decoder module 4083, configured to scale the second dot product result according to the dimension of the second key vector to obtain a second scaled dot product result;

[0218] The decoder module 4083 is used to perform normalization processing on the second scaling point product result to obtain a second normalized point product result;

[0219] Decoder module 4083, used to determine the product of the second normalized dot product result and the second value vector;

[0220] The decoder module 4083 is used to use at least one decoding module to decode the product of the second normalized dot product result and the second value vector to obtain a target hot word recognition result.

[0221] In some embodiments, when the decoder network has 2n layers in total, the decoder module 4083 is used to obtain the output vector of the decoding module of the nth layer when the hot word import module is located at the n+1th layer of the decoder network, where n is a positive integer greater than or equal to 1;

[0222] The decoder module 4083 is used to use the hot word import module in the decoder network to perform a dot product process on the output vector of the previous decoding layer of the hot word import module and the second key vector to obtain a third dot product result;

[0223] A decoder module 4083, configured to scale the third dot product result according to the dimension of the second key vector to obtain a third scaled dot product result;

[0224] The decoder module 4083 is used to normalize the third scaling point product result to obtain a third normalized point product result;

[0225] A decoder module 4083, configured to determine a product of the third normalized dot product result and the second value vector;

[0226] The decoder module 4083 is used to decode the product of the third normalized dot product result and the second value vector using the hot word import module when the hot word import module is the 2nth layer of the decoder network to obtain a target hot word recognition result; or

[0227] The decoder module 4083 is used to decode the product of the third normalized dot product result and the second value vector using the corresponding decoding modules in the 3rd to 2nth layers connected to the hot word import module when the hot word import module is any layer from the 2nd layer to the 2n-1th layer of the decoder network to obtain the target hot word recognition result.

[0228] In some embodiments, the decoder module 4083 is used to determine the position of at least one target hot word in the initial speech recognition result;

[0229] A decoder module 4083, configured to obtain at least one initial recognition vocabulary in the initial speech recognition result according to the position of at least one target hot word;

[0230] The decoder module 4083 is used to perform consistency detection on at least one initial recognition word in the initial speech recognition result and the target hot word recognition result;

[0231] The decoder module 4083 is used to update all target hot words in the initial speech recognition result through the target hot word recognition result when any initial recognition vocabulary in the initial speech recognition result is inconsistent with the target hot word recognition result, so as to obtain the speech recognition result corresponding to the speech information to be recognized.

[0232] In some embodiments, the hot word processing module 4082 is used to obtain at least one hot word to be processed in the hot word list;

[0233] Performing voice expansion on at least one hot word to be processed to obtain an extended vocabulary matching the at least one hot word to be processed, wherein the extended vocabulary includes at least one of the following:

[0234] Synonyms, antonyms, near synonyms and similar expressions;

[0235] Add the expanded vocabulary to the hot word list to obtain the target hot word list.

[0236] The present application also provides an electronic device, the electronic device comprising:

[0237] A memory for storing executable instructions;

[0238] The processor is used to implement the preceding speech recognition method when running the executable instructions stored in the memory.

[0239] The embodiment of the present application also provides a computer-readable storage medium storing executable instructions, which implement the preceding speech recognition method when executed by a processor.

[0240] An embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the preceding speech recognition method.

[0241] according to Fig.12 The electronic device shown, in one aspect of the present application, the present application also provides a computer program product or a computer program, the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the different embodiments and combinations of embodiments provided in various optional implementations of the speech recognition method provided in the embodiments of the present application.

[0242] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the speech recognition method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0243] The above are merely embodiments of the present application and are not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.< / nobias> < / nobias>

Claims

1. A speech recognition method, characterized in that: The method comprises: Obtain a first key vector and a first value vector of at least one hot word in the initial hot word list, wherein the hot word is used to represent a word that appears more than a threshold number of times within a preset time range; Acquire speech information to be recognized, encode the speech information to be recognized, and obtain a hidden layer output vector of the speech information to be recognized; Based on the attention mechanism, the initial hot word list is filtered using the hidden layer output vector, the first key vector and the first value vector to obtain a target hot word list; Acquire a second key vector and a second value vector of at least one target hot word in the target hot word list; The second key vector, the second value vector and the hidden layer output vector are decoded to obtain a speech recognition result corresponding to the speech information to be recognized, wherein the speech recognition result includes at least one target hot word in the target hot word list.

2. The method according to claim 1, characterized in that: The method of filtering the initial hot word list by using the hidden layer output vector, the first key vector and the first value vector based on the attention mechanism to obtain a target hot word list includes: Using a hot word filtering module, performing a dot product process on the hidden layer output vector and the first key vector to obtain a first dot product result; Scaling the first dot product result according to the dimension of the first key vector to obtain a first scaled dot product result; Normalizing the first scaling dot product result to obtain a first normalized dot product result; Determine the product of the first normalized dot product result and the first value vector as a reference value vector; The initial hot word list is filtered according to the reference value vector to obtain the target hot word list, wherein the target hot word list includes the target hot words corresponding to the reference value vector.

3. The method according to claim 2, characterized in that Filtering the initial hot word list according to the value vector of the at least one target hot word to obtain the target hot word list includes: Obtaining an initial hot word value vector of at least one hot word in the initial hot word list; Comparing the value vector of the at least one target hot word with the initial hot word value vector in sequence; When the value vector of the at least one target hot word is the same as the initial hot word value vector, the hot word corresponding to the initial hot word value vector is added to the target hot word list.

4. The method according to claim 3, characterized in that The decoding of the second key vector, the second value vector and the hidden layer output vector to obtain a speech recognition result corresponding to the speech information to be recognized includes: Decoding the hidden layer output vector through a decoder network of the speech recognition model to obtain an initial speech recognition result corresponding to the speech information; Decoding the second key vector and the second value vector through a decoder network of the speech recognition model to obtain a target hot word recognition result; The initial speech recognition result is updated according to the target hot word recognition result to obtain a speech recognition result corresponding to the speech information to be recognized.

5. The method according to claim 4, characterized in that The decoder network of the speech recognition model includes: a hot word import module and at least one decoding module; Decoding the second key vector and the second value vector through the decoder network of the speech recognition model to obtain the recognition result of the target hot word list includes: When the hot word import module is located in the first layer of the decoder network and is connected to a decoding layer, a historical text output feature vector of the decoder network is obtained according to the historical output result of the decoder network; Using the hot word import module in the decoder network, performing a dot product process on the historical text output feature vector and the second key vector to obtain a second dot product result; Scaling the second dot product result according to the dimension of the second key vector to obtain a second scaled dot product result; Normalizing the second scaling dot product result to obtain a second normalized dot product result; Determine the product of the second normalized dot product result and the second value vector; The at least one decoding module is used to decode the product of the second normalized dot product result and the second value vector to obtain the target hot word recognition result.

6. The method according to claim 5, characterized in that When the decoder network has 2n layers, The method further comprises: When the hot word import module is located at the n+1th layer of the decoder network, an output vector of the decoding module at the nth layer is obtained, where n is a positive integer greater than or equal to 1; Using the hot word import module in the decoder network, perform dot product processing on the output vector of the previous decoding layer of the hot word import module and the second key vector to obtain a third dot product result; Scaling the third dot product result according to the dimension of the second key vector to obtain a third scaled dot product result; Normalizing the third scaling dot product result to obtain a third normalized dot product result; Determine the product of the third normalized dot product result and the second value vector; When the hot word import module is the 2nth layer of the decoder network, the hot word import module is used to decode the product of the third normalized dot product result and the second value vector to obtain the target hot word recognition result; or When the hot word import module is any layer from the 2nd layer to the 2n-1th layer of the decoder network, the corresponding decoding modules in the 3rd layer to the 2nth layer connected to the hot word import module are used to decode the product of the third normalized dot product result and the second value vector to obtain the target hot word recognition result.

7. The method according to claim 4, characterized in that The initial speech recognition result is updated according to the recognition result of the target hot word list to obtain a speech recognition result corresponding to the speech information to be recognized, including: Determine the position of at least one target hot word in the initial speech recognition result; Acquire at least one initial recognition vocabulary in the initial speech recognition result according to the position of the at least one target hot word; Performing consistency detection on at least one initial recognition word in the initial speech recognition result and the target hot word recognition result; When any initial recognition vocabulary in the initial speech recognition result is inconsistent with the target hot word recognition result, all target hot words in the initial speech recognition result are updated by the target hot word recognition result to obtain the speech recognition result corresponding to the speech information to be recognized.

8. The method according to claim 1, characterized in that The method further comprises: Obtain at least one hot word to be processed in the hot word list; Performing voice expansion on the at least one hot word to be processed to obtain an extended vocabulary matching the at least one hot word to be processed, wherein the extended vocabulary includes at least one of the following: Synonyms, antonyms, near synonyms and similar expressions; The expanded vocabulary is added to the hot word list to obtain a target hot word list.

9. A speech information recognition device, characterized in that: The device comprises: An encoder module, configured to obtain a first key vector and a first value vector of at least one hot word in an initial hot word list, wherein the hot word is used to represent a word that appears more than a threshold number of times within a preset time range; The encoder module is used to obtain the voice information to be recognized, encode the voice information to be recognized, and obtain the hidden layer output vector of the voice information to be recognized; A hot word processing module, configured to filter the initial hot word list based on an attention mechanism by using the hidden layer output vector, the first key vector and the first value vector to obtain a target hot word list; The hot word processing module is used to obtain a second key vector and a second value vector of at least one target hot word in the target hot word list; A decoder module is used to decode the second key vector, the second value vector and the hidden layer output vector to obtain a speech recognition result corresponding to the speech information to be recognized, wherein the speech recognition result includes at least one target hot word in the target hot word list.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; A processor, configured to implement the speech recognition method according to any one of claims 1 to 8 when running the executable instructions stored in the memory.

11. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the speech recognition method according to any one of claims 1 to 8 is implemented.

12. A computer-readable storage medium storing executable instructions, characterized in that: When the executable instructions are executed by a processor, the speech recognition method according to any one of claims 1 to 8 is implemented.