Speech recognition method and device, computer equipment and storage medium

Abstract

The invention relates to the field of voice processing, and discloses a voice recognition method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized voice in real time through adaptive sampling; preprocessing the to-be-recognized voice to obtain an optimized voice meeting a preset processing requirement; processing the optimized voice through a de-duplication voice recognition model to obtain a de-duplication text; the de-duplication speech recognition model is a speech recognition model with a de-duplication function after pruning processing. According to the method, the to-be-recognized voice is processed by using the de-duplication voice recognition model, and the de-duplication text is acquired, so that the problem of word or sentence repetition of the voice recognition model is solved, particularly under the scenes of long-time voice transcription, complex dialogue recording and the like, the word repetition phenomenon is remarkably reduced, and the model efficiency and experience are optimized.

Description

Technical Field

[0001] The present invention relates to the field of speech processing, is applicable to the field of financial technology, and in particular to a speech recognition method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of the FinTech industry, there is an increasing demand for automation and intelligence in business processes such as customer service, transaction processing and financial consulting. As an important branch of artificial intelligence, voice recognition technology has shown great potential in improving user experience and optimizing operational efficiency. Especially in the field of FinTech, voice interaction can not only simplify the operation process, but also provide users with a more convenient service experience.

[0003] Existing speech recognition models, such as Whisper (a multilingual speech recognition model developed by the open source artificial intelligence company OpenAI), have excellent performance in cross-language recognition and robustness, and are widely used in speech transcription and natural language processing tasks. However, when processing long text or complex speech input, speech recognition models are prone to word or sentence duplication, which affects recognition efficiency and accuracy, increases the difficulty of subsequent processing, and reduces user experience. Existing optimization methods such as parameter tuning and data enhancement have certain effects, but they cannot fundamentally solve the duplication phenomenon caused by the redundancy of the internal structure of the model. Summary of the invention

[0004] Based on this, it is necessary to provide a speech recognition method, apparatus, computer device and storage medium to address the above technical problems, so as to reduce the problem of repeated words or sentences in the speech recognition model.

[0005] A speech recognition method, comprising: Acquire the speech to be recognized in real time through adaptive sampling; Preprocess the speech to be recognized to obtain optimized speech that meets preset processing requirements; The optimized speech is processed by a deduplication speech recognition model to obtain deduplication text; the deduplication speech recognition model is a speech recognition model with deduplication function after pruning.

[0006] Optionally, before the step of processing the optimized speech through a deduplicated speech recognition model to obtain deduplicated text, the step further includes: Get pruning rules that match the pre-trained model; Pruning the pre-trained model according to the pruning rule to obtain a speech recognition model to be evaluated; The speech recognition model to be evaluated is optimized and verified to obtain the deduplicated speech recognition model.

[0007] Optionally, the acquiring of pruning rules matching the pre-trained model includes: Acquiring feature data associated with the pre-trained model; Extracting characteristic distribution of the characteristic data; The pruning rule corresponding to the feature distribution is obtained.

[0008] Optionally, pruning the pre-trained model according to the pruning rule to obtain the speech recognition model to be evaluated includes: Evaluating the weights of each linear layer in the pre-trained model; Determine a weight threshold according to the weights of each linear layer and the pruning rule; The linear layers whose absolute values ​​of weights in the pre-trained model are less than the weight threshold are pruned to obtain the speech recognition model to be evaluated.

[0009] Optionally, determining the weight threshold according to the weights of the respective linear layers and the pruning rule includes: Arrange the weights of each linear layer according to the weight value to form a weight sequence; Extract the global pruning ratio from the pruning rules; The weight threshold is determined according to the weight sequence and the global pruning ratio.

[0010] Optionally, the optimizing and verifying the speech recognition model to be evaluated to obtain the deduplicated speech recognition model includes: Partially pruning the encoder and decoder of the speech recognition model to be evaluated to obtain an optimized model; Evaluating the error word index of the optimization model; When the wrong word index meets the preset requirement, the optimization model is determined as the de-duplication speech recognition model.

[0011] Optionally, after the to-be-recognized speech is processed by the deduplication speech recognition model to obtain deduplication text, the method further includes: Extract key information from the deduplicated text; Summary data is generated according to the key information.

[0012] A speech recognition device, comprising: A voice acquisition module is used to acquire the voice to be recognized in real time through adaptive sampling; A preprocessing module, used to preprocess the speech to be recognized to obtain an optimized speech that meets the preset processing requirements; The model recognition module is used to process the optimized speech through a deduplication speech recognition model to obtain deduplication text; the deduplication speech recognition model is a speech recognition model with deduplication function after pruning.

[0013] A computer device comprises a memory, a processor and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-mentioned speech recognition method when executing the computer-readable instructions.

[0014] One or more readable storage media storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the speech recognition method as described above.

[0015] The above-mentioned speech recognition method, device, computer equipment and storage medium use a deduplicated speech recognition model to process the speech to be recognized and obtain deduplicated text, which solves the problem of word or sentence duplication in the speech recognition model, especially in scenarios such as long-term speech transcription and complex conversation recording, significantly reducing word duplication and optimizing model efficiency and experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0017] Figure 1 is a schematic diagram of an application environment of a speech recognition method in an embodiment of the present invention; Figure 2 is a flow chart of a speech recognition method according to an embodiment of the present invention; Figure 3 is a structural schematic diagram of a speech recognition device in one embodiment of the present invention; Figure 4 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The speech recognition method provided in this embodiment can be applied in Figure 1 In an application environment, a client communicates with a server. The client includes but is not limited to various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers.

[0020] In one embodiment, if Figure 2 As shown, a speech recognition method is provided, which is applied in Figure 1 The server in the example is used as an example to illustrate the following steps: S10, acquiring the speech to be recognized in real time through adaptive sampling; S20, preprocessing the speech to be recognized to obtain an optimized speech that meets preset processing requirements; S30. Process the optimized speech through a deduplication speech recognition model to obtain deduplication text; the deduplication speech recognition model is a speech recognition model with deduplication function after pruning.

[0021] Understandably, real-time acquisition of the speech to be recognized refers to the process of capturing and transmitting the speech to the recognition system for processing in real time through an audio input device. For example, in the telephone customer service system of a bank or e-commerce platform, the user's voice is transmitted to the server of the customer service center in real time through the telephone line. For another example, in a video conference, the voice of the participants is captured by a microphone. In some examples, the speech to be recognized can be acquired in real time through adaptive sampling. Adaptive sampling can dynamically adjust the sampling rate according to the characteristics of the input voice signal, thereby optimizing the resource utilization efficiency in the process of collecting the speech to be recognized and improving the performance of the speech recognition system. Whether there is speech and the activity of the speech can be determined based on voice activity detection (VAD). When speech is detected, the system can increase the sampling rate to capture more detailed speech information. During non-speech or background noise, the sampling rate is reduced to reduce unnecessary data processing and storage requirements.

[0022] The speech to be recognized can be preprocessed to obtain optimized speech that meets the preset processing requirements. The speech to be recognized is preprocessed, including but not limited to noise reduction processing, endpoint detection, sampling rate conversion, frequency response adjustment, and format conversion. The preset processing requirements can be defined according to the actual application scenario. For example, in an intelligent customer service system, the preset processing requirements include but are not limited to the response time being less than the preset time threshold to achieve a quick response; the accuracy rate is higher than the preset accuracy threshold to achieve high accuracy; noise reduction to effectively remove background noise to ensure that the customer's voice can be clearly recognized in a noisy environment. For another example, in a smart medical system, the preset processing requirements include but are not limited to privacy protection, strict compliance with data protection regulations, and ensuring the security and confidentiality of patient information; medical terminology recognition to enhance the ability to understand medical terms and professional conversations; long-term recording processing, suitable for long-term consultation records, and can work stably and maintain a high-quality recognition rate. After preprocessing, optimized speech can be obtained. The advantages of optimized speech include but are not limited to clear voice quality, appropriate length, unified format and standard, stable energy and dynamic range, good time alignment, effective feature representation, and high signal-to-noise ratio.

[0023] The deduplication speech recognition model is a speech recognition system that can recognize and eliminate duplicate content. This model can reduce insertion errors when processing the speech to be recognized, and output text with a higher accuracy rate, thereby improving the neatness and readability of the text. The deduplication speech recognition model provided in this embodiment is a speech recognition model with deduplication function after pruning. Pruning is a model optimization technology that can reduce the size and computational complexity of the model while maintaining the performance of the model. In this embodiment, the deduplication speech recognition model can not only recognize speech, but also remove duplicate speech content, and the size and running speed of the model are also optimized.

[0024] After testing, the deduplication speech recognition model performed well on various downstream datasets and did not require fine-tuning for each dataset. It is suitable for applications in scenarios with multiple languages ​​(such as foreign languages, dialects, etc.).

[0025] In one example, the true value of a test meeting is: You are in the back kitchen, this side, we are going to change this wall directly to this glass. The result of the pre-trained model recognition is: You are in the back kitchen, this side, we are going to change this wall directly to this glass. In another example, the true value is: When approved by the board of directors of XX company. The result of the pre-trained model recognition is: When the board of directors of XX company executes, executes, executes, executes... The recognition result of the de-duplication speech recognition model is: When approved by the board of directors of XX company.

[0026] The speech recognition method provided in this embodiment uses a deduplicated speech recognition model to process the speech to be recognized and obtains deduplicated text, thereby solving the problem of repeated words or sentences in the speech recognition model. In particular, in scenarios such as long-term speech transcription and complex conversation records, it significantly reduces word repetition and optimizes model efficiency and experience.

[0027] Optionally, before step S30, that is, before processing the optimized speech by the deduplication speech recognition model to obtain the deduplication text, the method further includes: S31, obtaining pruning rules matching the pre-trained model; S32, pruning the pre-trained model according to the pruning rule to obtain a speech recognition model to be evaluated; S33, optimizing and verifying the speech recognition model to be evaluated to obtain the deduplicated speech recognition model.

[0028] Understandably, a pre-trained model refers to a model that is obtained through training with a large amount of corpus data and can perform preliminary processing on speech signals. During the training process, the pre-trained model learns the underlying features of speech signals, such as phonemes, syllables, etc., as well as the combination relationship between them. It is necessary to obtain pruning rules that match the pre-trained model. When pruning, the differences in the characteristics of data in different modalities are taken into account. Data in different modalities have unique feature distributions and complexities, so pruning rules that match the pre-trained model are obtained. In order to minimize model parameters while maintaining the stability and accuracy of model performance, it is necessary to design highly adaptable pruning rules for different modalities. Here, the pruning rules that match the pre-trained model are pruning rules for processing speech modal data.

[0029] After obtaining the appropriate pruning rules, the pre-trained model can be pruned according to the pruning rules to obtain the speech recognition model to be evaluated. After obtaining the speech recognition model to be evaluated, it is necessary to further optimize and verify the speech recognition model to be evaluated to ensure that the obtained deduplication speech recognition model does not sacrifice the accuracy and robustness of the overall model while achieving the deduplication effect.

[0030] This embodiment uses matching pruning rules to obtain the speech recognition model to be evaluated, and then obtains the deduplicated speech recognition model through optimization and verification. This multi-level verification mechanism provides a guarantee for the comprehensive optimization of the model and avoids the negative impact of single task optimization.

[0031] Optionally, step S31, i.e., obtaining pruning rules matching the pre-trained model, includes: S311, acquiring feature data associated with the pre-trained model; S312, extracting characteristic distribution of the characteristic data; S313: Obtain the pruning rule corresponding to the feature distribution.

[0032] Understandably, in order to obtain pruning rules that match the pre-trained model, the feature data associated with the pre-trained model can be obtained first, such as the corpus data used in the pre-training process. The feature distribution of these feature data is extracted, such as calculating the minimum, maximum, mean, standard deviation and other statistical information of the weight or activation data, and using methods such as histograms and kernel density estimation to observe the distribution of feature values. For neural networks, this may include analyzing the activation data of each layer to understand their distribution characteristics.

[0033] Finally, obtain the pruning rules corresponding to the feature distribution. In some examples, pruning rules can be defined based on the analysis results of the feature distribution. Thresholds can be set based on the feature distribution, for example, connections with absolute weights less than a certain threshold can be removed. Define statistical rules, such as removing features that contribute the least to the model output. Apply structured pruning or unstructured pruning strategies and choose the appropriate pruning method based on specific needs.

[0034] This embodiment can effectively prune the pre-trained model by matching the feature distribution pruning rules to reduce the complexity of the model, while maintaining the performance of the model as much as possible, and reducing the model's dependence on training data. Optionally, step S32, that is, pruning the pre-trained model according to the pruning rules to obtain the speech recognition model to be evaluated, includes: S321, evaluating the weights of each linear layer in the pre-trained model; S322, determining a weight threshold according to the weights of each linear layer and the pruning rule; S323. Prune the linear layers in the pre-trained model whose absolute values ​​of weights are less than the weight threshold to obtain the speech recognition model to be evaluated.

[0035] Understandably, the pre-trained model does not need to be fine-tuned. The weights (magnitude) of all linear layers in each transformer block of the pre-trained model can be evaluated, and then the weight threshold can be selected by combining the pruning rules and the weights at the evaluation point. Prune the linear layers in the pre-trained model whose absolute values ​​of weights are less than the weight threshold, that is, reset the weights whose absolute values ​​are less than the weight threshold to zero. A specific sparse matrix memory can be used to efficiently store the speech recognition model to be evaluated, thereby accelerating the model inference speed.

[0036] Specifically, you can evaluate the weights of each linear layer in the pre-trained model, check and record the weight values ​​of each linear layer (such as the fully connected layer) in the pre-trained model. By accessing the model architecture, you can extract the weight matrices of all linear layers. For each weight matrix, you can calculate its statistical properties (such as the mean, standard deviation, etc.) to obtain the specific weight value.

[0037] The weight threshold can be determined based on the weights of each linear layer and the pruning rules. Set a criterion to decide which weights should be retained and which should be removed. By defining pruning rules, such as based on absolute value size, L1 norm, L2 norm, etc. According to the selected rule, a suitable threshold is calculated from the weights of all linear layers. This threshold may be a fixed value or a percentage relative to the weight distribution (such as removing the smallest 10% of weights). The choice of threshold usually requires balancing model performance and sparsity after pruning. The optimal threshold can be determined by experimentally verifying the impact of different thresholds on model accuracy.

[0038] The linear layers in the pre-trained model whose absolute weights are less than the weight threshold are pruned to obtain the speech recognition model to be evaluated, in order to create a more streamlined but still valid version of the model. By traversing all linear layers and comparing each weight with its corresponding threshold. For those weights whose absolute values ​​are less than the threshold, they are set to zero or directly removed from the model (depending on the implementation). It should be noted that this does not mean "pruning" the entire linear layer, but rather pruning specific weights within the layer. After pruning, a new model structure is obtained, which is the speech recognition model to be evaluated.

[0039] This embodiment provides a pruning process for a pre-trained model, which can help optimize the speech recognition model, not only reducing the number of model parameters, but also reducing computing costs and storage requirements, while maintaining the predictive performance of the model as much as possible.

[0040] Optionally, step S322, i.e., determining a weight threshold according to the weights of each linear layer and the pruning rule, includes: S3221, arranging the weights of the linear layers according to the weight values ​​to form a weight sequence; S3222, extracting a global pruning ratio from pruning rules; S3223. Determine the weight threshold according to the weight sequence and the global pruning ratio.

[0041] Understandably, when determining the weight threshold, the weights of each linear layer can be arranged according to the weight value to form a weight sequence, such as arranging from large to small. By forming a weight sequence, a unified ordering is created for the weights in all linear layers for global pruning. The global pruning ratio is then extracted from the pruning rule to determine the proportion of the weights to be removed to the total weights. The global pruning ratio can be between 10% and 90%. The specific ratio depends on the complexity of the model, the size and quality of the data set, and the choice of pruning algorithm. The global pruning ratio based on experimental data and performance evaluation can ensure that the model complexity is reduced without significantly affecting the accuracy of the model.

[0042] After determining the weight sequence and global pruning ratio, the weight threshold can be calculated. For example, if the weight sequence is 5, 4, 3, 2, 1 and the global pruning ratio is 20%, the weight threshold can be set to 1.5. All weights whose absolute value is less than or equal to this threshold will be marked as objects to be pruned.

[0043] This embodiment provides a method for determining a weight threshold, which can ensure that the pruning process is fair and globally consistent, rather than performing local optimization on each layer individually, which helps maintain the overall structure of the model while achieving effective parameter reduction.

[0044] Optionally, step S33, namely, optimizing and verifying the speech recognition model to be evaluated to obtain the deduplicated speech recognition model, includes: S331, locally pruning the encoder and decoder of the speech recognition model to be evaluated to obtain an optimized model; S332, evaluating the wrong word index of the optimization model; S333. When the wrong word index meets the preset requirement, the optimization model is determined as the deduplication speech recognition model.

[0045] Understandably, in order to ensure the high efficiency of improving the model performance, the speech recognition model to be evaluated is not further fine-tuned after pruning, and the encoder and decoder of the speech recognition model to be evaluated are locally pruned. The pruning ratio of different structures can be set according to actual needs. In one example, the pruning ratio of the decoder is set to twice that of the encoder. By locally pruning the encoder and decoder of the speech recognition model to be evaluated, an optimized model can be obtained, and unimportant weights in the encoder and decoder can be reduced to reduce model complexity and improve reasoning efficiency. For each layer in the encoder and decoder, pruning operations are applied independently. This means that each part will be optimized according to its own characteristics and needs. In each local pruning process, a suitable threshold is determined, and weights whose absolute value is less than the threshold are removed. For the encoder and decoder, this threshold can be the same or different, depending on their respective importance distributions.

[0046] In order to verify the performance of the optimized model, the model recognition results can be calculated on Chinese and English data sets, and the corresponding word error indicators can be calculated. Evaluating the word error indicators of the optimized model can measure the performance of the optimized model, especially its accuracy and reliability. In one example, the word error indicator can be the word error rate (WER). A test set can be prepared to evaluate the model using a test set containing real speech data and its correctly transcribed text. The word error rate is calculated by comparing the transcription results output by the model with the actual text. The word error rate is a key indicator for measuring the performance of a speech recognition system, which reflects the proportion of words that are incorrectly predicted by the model. After analysis, the number of insertion errors, deletion errors, and substitution errors of the single recognition and overall recognition results of the deduplication speech recognition model was analyzed, and it was found that the reduction in the word error rate was mainly attributed to the reduction in insertion errors.

[0047] When the error word index meets the preset requirements, the optimized model is determined as the deduplication speech recognition model to ensure that the final model is both effectively optimized and does not significantly sacrifice recognition accuracy. An acceptable range of error word rates or other performance indicator requirements can be predefined. For example, if the WER of the original model is 5%, the optimized model should remain near this level or better. Once the WER of the optimized model meets the preset requirements, it can be confirmed as the final deduplication speech recognition model.

[0048] This embodiment optimizes and verifies the speech recognition model to be evaluated to ensure that the deduplication speech recognition model achieves the deduplication effect without sacrificing the accuracy and robustness of the overall model.

[0049] Optionally, after step S30, that is, after the optimized speech is processed by the deduplication speech recognition model to obtain the deduplication text, the method further includes: S40, extracting key information from the deduplicated text; S50: Generate summary data according to the key information.

[0050] Understandably, after obtaining the deduplicated text, key information can be extracted from the deduplicated text. First, the deduplicated text can be preprocessed by splitting it into independent sentences, further splitting each sentence into words or phrases, removing common stop words that do not contain key information, such as "de", "he", "shi", etc., and performing part-of-speech tagging on each word to identify nouns, verbs, adjectives, etc. Then, at least the following key information can be obtained: counting the occurrence frequency of each word, as high-frequency words often contain more information; using TF-IDF (Term Frequency-Inverse Document Frequency) or other algorithms to assign an importance score to each word; identifying sentences that contain key information, which may contain high-frequency words or specific keywords; using named entity recognition (NER) technology to extract entity information such as names of people, locations, organizations, etc.

[0051] After obtaining the key information, summary data can be generated according to a preset generation method, such as an extractive summary or an abstractive summary. In a meeting scenario, generating summary data for the identified deduplicated text can provide clear meeting points for participants.

[0052] In this embodiment, by further processing the deduplicated text to generate summary data, the efficiency and quality of information processing can be improved, providing better services for users.

[0053] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution is prior or posterior. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0054] In one embodiment, a voice recognition device is provided, which corresponds one-to-one with the voice recognition method in the above embodiment. As Figure 3 shown, the voice recognition device includes a voice acquisition module 10 and a model recognition module 20. The detailed description of each functional module is as follows: The voice acquisition module 10 is used to acquire the voice to be recognized in real time through adaptive sampling; The preprocessing module 20 is used to preprocess the voice to be recognized to obtain an optimized voice that meets the preset processing requirements; The model recognition module 30 is used to process the optimized voice through a deduplicated voice recognition model to obtain deduplicated text; the deduplicated voice recognition model is a voice recognition model with a deduplication function after pruning processing.

[0055] Optionally, the voice recognition device further includes a model construction module, and the model construction module includes: A pruning rule acquisition unit, which is used to acquire pruning rules that match the pre-trained model; A pruning unit, used for pruning the pre-trained model according to the pruning rule to obtain a speech recognition model to be evaluated; The optimization and verification unit is used to optimize and verify the speech recognition model to be evaluated to obtain the deduplicated speech recognition model.

[0056] Optionally, the obtaining pruning rule unit includes: A feature data acquisition unit, used to acquire feature data associated with the pre-trained model; A feature distribution extraction unit, used for extracting feature distribution of the feature data; A matching rule unit is used to obtain the pruning rule corresponding to the feature distribution.

[0057] Optionally, the pruning unit includes: A weight evaluation unit, used to evaluate the weights of each linear layer in the pre-trained model; A weight threshold determination unit, used to determine a weight threshold according to the weights of each linear layer and the pruning rule; A model-to-be-evaluated unit is generated, which is used to remove the linear layers whose absolute values ​​of weights in the pre-trained model are less than the weight threshold, so as to obtain the speech recognition model to be evaluated.

[0058] Optionally, the weight threshold determination unit includes: A sorting unit, used to arrange the weights of each linear layer according to the weight value to form a weight sequence; A pruning ratio extraction unit, used for extracting a global pruning ratio from a pruning rule; A weight threshold generating unit is used to determine the weight threshold according to the weight sequence and the global pruning ratio.

[0059] Optionally, the optimization verification unit includes: A local pruning unit, used to locally prune the encoder and decoder of the speech recognition model to be evaluated to obtain an optimized model; An index evaluation unit, used to evaluate the wrong word index of the optimization model; A de-duplication model determination unit is used to determine the optimization model as the de-duplication speech recognition model when the error word index meets the preset requirements.

[0060] Optionally, the speech recognition device further includes: A key information extraction module, used to extract key information from the deduplicated text; The summary data generating module is used to generate summary data according to the key information.

[0061] For the specific definition of the speech recognition device, please refer to the definition of the speech recognition method above, which will not be repeated here. Each module in the above-mentioned speech recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0062] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer-readable instructions and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database of the computer device is used to store data involved in the speech recognition method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a speech recognition method is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0063] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein when the processor executes the computer-readable instructions, the following steps are implemented: Acquire the speech to be recognized in real time through adaptive sampling; Preprocess the speech to be recognized to obtain optimized speech that meets preset processing requirements; The optimized speech is processed by a deduplication speech recognition model to obtain deduplication text; the deduplication speech recognition model is a speech recognition model with deduplication function after pruning.

[0064] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The readable storage media store computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the following steps are implemented: Acquire the speech to be recognized in real time through adaptive sampling; Preprocess the speech to be recognized to obtain optimized speech that meets preset processing requirements; The optimized speech is processed by a deduplication speech recognition model to obtain deduplication text; the deduplication speech recognition model is a speech recognition model with deduplication function after pruning.

[0065] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0066] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0067] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A speech recognition method, characterized in that: include: Acquire the speech to be recognized in real time through adaptive sampling; Preprocess the speech to be recognized to obtain optimized speech that meets preset processing requirements; Processing the optimized speech through a deduplication speech recognition model to obtain deduplication text; The deduplication speech recognition model is a speech recognition model with deduplication function after pruning.

2. The speech recognition method according to claim 1, characterized in that: Before the optimized speech is processed by the deduplication speech recognition model to obtain the deduplication text, the method further includes: Get pruning rules that match the pre-trained model; Pruning the pre-trained model according to the pruning rule to obtain a speech recognition model to be evaluated; The speech recognition model to be evaluated is optimized and verified to obtain the deduplicated speech recognition model.

3. The speech recognition method according to claim 2, characterized in that: The obtaining of pruning rules matching the pre-trained model includes: Acquiring feature data associated with the pre-trained model; Extracting characteristic distribution of the characteristic data; The pruning rule corresponding to the feature distribution is obtained.

4. The speech recognition method according to claim 2, characterized in that: The step of pruning the pre-trained model according to the pruning rule to obtain a speech recognition model to be evaluated includes: Evaluating the weights of each linear layer in the pre-trained model; Determine a weight threshold according to the weights of each linear layer and the pruning rule; The linear layers whose absolute values ​​of weights in the pre-trained model are less than the weight threshold are pruned to obtain the speech recognition model to be evaluated.

5. The speech recognition method according to claim 4, characterized in that: The determining the weight threshold according to the weights of the respective linear layers and the pruning rule comprises: Arrange the weights of each linear layer according to the weight value to form a weight sequence; Extract the global pruning ratio from the pruning rules; The weight threshold is determined according to the weight sequence and the global pruning ratio.

6. The speech recognition method according to claim 2, characterized in that: The step of optimizing and verifying the speech recognition model to be evaluated to obtain the deduplicated speech recognition model includes: Partially pruning the encoder and decoder of the speech recognition model to be evaluated to obtain an optimized model; Evaluating the error word index of the optimization model; When the wrong word index meets the preset requirement, the optimization model is determined as the de-duplication speech recognition model.

7. The speech recognition method according to claim 1, wherein: After the optimized speech is processed by the deduplication speech recognition model to obtain the deduplication text, the method further includes: Extract key information from the deduplicated text; Summary data is generated according to the key information.

8. A speech recognition device, characterized in that: include: A voice acquisition module is used to acquire the voice to be recognized in real time through adaptive sampling; A preprocessing module, used to preprocess the speech to be recognized to obtain an optimized speech that meets the preset processing requirements; A model recognition module, used for processing the optimized speech through a deduplication speech recognition model to obtain deduplication text; The deduplication speech recognition model is a speech recognition model with deduplication function after pruning.

9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executed on the processor, characterized in that: When the processor executes the computer-readable instructions, the speech recognition method according to any one of claims 1 to 7 is implemented.

10. One or more readable storage media storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the speech recognition method according to any one of claims 1 to 7.

Images