Voice data processing methods, devices, electronic equipment and storage media

Abstract

This application discloses a speech data processing method, apparatus, electronic device, and storage medium. The method includes: acquiring speech data to be recognized; converting the speech data into text data based on Automatic Speech Recognition (ASR) technology; using the text data to determine first information and second information corresponding to the speech data, wherein the first information includes scene information corresponding to the speech data, and the second information includes contextual association information and prefix information related to words in the speech data; and using the text data, the first information, and the second information to determine keywords of the speech data.

Description

Technical Field

[0001] This application relates to the field of speech recognition, and more particularly to a speech data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the widespread application of artificial intelligence technology, Automatic Speech Recognition (ASR) technology, as a convenient human-computer interaction technology, can convert voice data into text data. For example, in commercial applications, businesses use ASR technology to automatically convert telephone conversations, allowing customers' voices to be recorded in text form, facilitating the categorization of customer needs and the retrieval of complaint information. Another example is in hospital settings, where, in addition to outpatient consultations, there are also examination scenarios such as ultrasound and gastroscopy. In these examination scenarios, medical staff typically need to hold medical equipment to examine patients, making it inconvenient to manually record the problems found during the examination. In such cases, ASR technology can convert the medical staff's examination results into text and output them in real time.

[0003] However, in related technologies, there is a problem with low accuracy in recognizing keywords such as professional terms when recognizing voice data. Summary of the Invention

[0004] To address the related technical issues, embodiments of this application provide a voice data processing method, apparatus, electronic device, and storage medium.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a voice data processing method, including:

[0007] Acquire the speech data to be recognized;

[0008] Based on ASR technology, the voice data is converted into text data;

[0009] Using the text data, determine the first information and the second information corresponding to the voice data. The first information includes the scene information corresponding to the voice data, and the second information includes the association information and prefix information related to the context of the words in the voice data.

[0010] Using the text data, the first information, and the second information, keywords of the voice data are determined.

[0011] In the above scheme, determining the first information corresponding to the voice data using the text data includes:

[0012] The text data is input into the first model to obtain the scene information corresponding to the voice data. The first model is used to determine the scene information of the input text data.

[0013] In the above scheme, the first model includes a model trained using a multi-task learning approach.

[0014] In the above scheme, determining the second information corresponding to the voice data using the text data includes:

[0015] The text data is input into the second model to obtain contextual association information and prefix information related to the words in the speech data. The second model is used to determine the contextual association information and prefix information related to the words in the input text data.

[0016] In the above scheme, the second model includes a model trained based on Bidirectional Encoder Representations from Transformer (BERT).

[0017] In the above scheme, the conversion of the voice data into text data based on ASR technology includes:

[0018] The voice data is preprocessed;

[0019] Based on ASR technology, the preprocessed speech data is converted into text data.

[0020] In the above scheme, the keywords of the voice data are determined using the text data, the first information, and the second information, based on a cluster search algorithm.

[0021] This application also provides a voice data processing method, including:

[0022] Acquisition unit, used to acquire the speech data to be recognized;

[0023] A conversion unit is used to convert the voice data into text data based on ASR technology;

[0024] The first processing unit is used to use the text data to determine first information and second information corresponding to the voice data. The first information includes scene information corresponding to the voice data, and the second information includes contextual association information and prefix information related to the words in the voice data.

[0025] The second processing unit is used to determine the keywords of the voice data using the text data, the first information, and the second information.

[0026] This application also provides an electronic device, including: a processor and a communication interface; wherein,

[0027] The processor is configured to acquire speech data to be recognized; convert the speech data into text data based on ASR technology; use the text data to determine first information and second information corresponding to the speech data, wherein the first information includes scene information corresponding to the speech data, and the second information includes contextual association information and prefix information related to the words in the speech data; and use the text data, the first information, and the second information to determine the keywords of the speech data.

[0028] This application also provides an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor.

[0029] Wherein, when the processor runs the computer program, it executes any of the steps of the voice data processing method.

[0030] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the aforementioned voice data processing methods.

[0031] The speech data processing method, apparatus, electronic device, and storage medium provided in this application embodiment acquire speech data to be recognized; based on ASR technology, the speech data is converted into text data; using the text data, first information and second information corresponding to the speech data are determined, the first information including scene information corresponding to the speech data, and the second information including contextual association information and prefix information related to the words in the speech data; using the text data, the first information, and the second information, the keywords of the speech data are determined. The solution provided in this application embodiment, based on general recognition methods in related technologies, can determine whether the application scenario of the current speech data is a domain scenario (such as the medical field) or a non-domain scenario through the scene information of the speech data; further, by combining the prefixes of hot words (i.e., keywords) in the speech data, it can distinguish between hot words in domain scenarios and general vocabulary in non-domain scenarios, thereby improving the accuracy of hot word recognition. Attached Figure Description

[0032] Figure 1 This is a schematic flowchart of a voice data processing method according to an embodiment of this application;

[0033] Figure 2 This is a schematic diagram of the structure of the identification system used in this application.

[0034] Figure 3This is a schematic diagram of the voice data processing device according to an embodiment of this application;

[0035] Figure 4 This is a schematic diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0036] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0037] In related technologies, voice data can correspond to different scenarios (such as teachers lecturing, doctors treating patients, business meetings, etc.). New words, new terms, or keywords that customers require to be recognized (such as professional terms in a specific field) will be different in different scenarios, making it difficult for voice recognition technology to accurately identify keywords in different scenarios, i.e., there is a problem of low recognition accuracy.

[0038] Based on this, in various embodiments of this application, based on relevant speech recognition technology, the scene of the speech input data can be determined to be a domain scene or a non-domain scene by using scene information; further, by combining the prefixes of hot words in the speech data, accurate recognition of hot words in speech input data under different scenes can be achieved.

[0039] This application provides a voice data processing method, such as... Figure 1 As shown, applied to electronic devices, the method includes:

[0040] Step 101: Obtain the speech data to be recognized;

[0041] Step 102: Based on ASR technology, convert the voice data into text data;

[0042] Step 103: Using the text data, determine the first information and the second information corresponding to the voice data. The first information includes the scene information corresponding to the voice data, and the second information includes the contextual association information and prefix information related to the words in the voice data.

[0043] Step 104: Using the text data, the first information, and the second information, determine the keywords of the voice data.

[0044] In practical applications, the electronic device may include a device capable of performing steps 101 to 104, such as a server, laptop computer, tablet computer, or desktop computer. This application embodiment does not limit this.

[0045] Here, the prefix information can be understood as the text content before the keywords in the text data; the keywords can be set according to the needs of the scenario, such as professional terms or domain terms in a specific field, such as disease names in the medical field.

[0046] In practical applications, in step 101, the electronic device can acquire the voice data to be recognized in real time through a voice acquisition device, such as a microphone; that is, while the user inputs voice data into the electronic device through the voice acquisition device, the electronic device recognizes the input voice data; this recognition scenario can be called a streaming voice recognition scenario. Alternatively, the electronic device can also retrieve stored voice data to be recognized from a database; that is, the electronic device will only recognize the input voice data after the user has input a segment of voice; this recognition scenario can be called a non-streaming voice recognition scenario. This application embodiment does not limit the method of acquiring voice data.

[0047] In practical applications, in order to ensure the accuracy of the conversion from voice data to text data, the voice data needs to be preprocessed before the electronic device converts the voice data.

[0048] Based on this, in one embodiment, the specific implementation of step 102 may include:

[0049] The voice data is preprocessed;

[0050] Based on ASR technology, the preprocessed speech data is converted into text data.

[0051] The electronic device can preprocess the voice data according to preset parameters, so that the parameters of the preprocessed voice data meet the preset parameters, which may include sampling rate, bit rate, and number of channels. This application embodiment does not limit the method of preprocessing by the electronic device.

[0052] For the preprocessed speech data, the electronic device can use ASR technology to convert the preprocessed speech data. ASR technology can be understood as a technique that uses an ASR encoder and a neural network-based Connectionist Temporal Classification (CTC) decoder to convert speech data into text data. Specifically, the ASR encoder is used to segment speech frames based on the speech data and obtain speech features; the CTC decoder is used to determine the mapping relationship between speech features and recognized text, thereby converting speech features into text information. The network structures of the ASR encoder and the CTC decoder can be the same as or different from those in related technologies. This application does not limit the network structures of the ASR encoder and the CTC decoder, as long as the functions of the ASR encoder and the CTC decoder are implemented.

[0053] Specifically, the ASR encoder divides the preprocessed speech data into multiple speech frames. By extracting features from each speech frame and encoding these features, the ASR encoder obtains an encoded feature vector, which is then input into the CTC decoder. Upon receiving the encoded feature vector, the CTC decoder, based on a pre-trained acoustic and language model, converts the encoded feature vector into text data. The acoustic model converts the feature vector into phonemes (e.g., initials or finals), and the language model converts the phonemes into characters or letters. For Chinese, the smallest unit in the text data is a character; for English, the smallest unit is a word or an English letter.

[0054] In practical applications, for the text data, the electronic device can determine the scene information corresponding to the text data through a model used to predict the scene.

[0055] Based on this, in one embodiment, the text data is input into a first model to obtain scene information corresponding to the voice data, and the first model is used to determine the scene information of the input text data.

[0056] In practical applications, the first model may also be called a scene bias model, a scene bias encoder, or a scene bias encoding module, etc. This application embodiment does not limit this, as long as its function is implemented.

[0057] Here, before using the first model, it is necessary to train the first model so that the scene information of the text data can be determined through the trained first model; wherein, the first model can be trained by the electronic device or by other devices, and the electronic device uses the trained first model. This application embodiment does not limit the training method of the first model.

[0058] For example, the pre-training process may include: the electronic device acquiring sample data carrying scene labels, the sample data including classification label text data or intent label text data; based on the lexicon, the electronic device can segment the sample data carrying scene labels to obtain segmented sample data; using the segmented sample data, the corresponding classification labels and intent labels, and employing a Transformer as an encoder and Adaptive Moment Estimation (AdaM) as a learning optimizer, the electronic device can train the first model to obtain the pre-trained first model.

[0059] In practical applications, a large amount of real-world data is often required as sample data to train the first model. However, in related technologies, due to the strong confidentiality of data in specific domain scenarios, it is difficult to obtain a large amount of real-world data. For example, in a business meeting scenario, it is not easy to obtain relevant real-world data. Due to the limited amount of sample data, the recognition accuracy of the pre-trained model may be low. In the above situation, the first model can be further trained based on relevant models.

[0060] Accordingly, in one embodiment, the first model includes a model trained using a multi-task learning approach.

[0061] In practical applications, the multi-task learning method can be understood as using information from relevant models to help train the first model, such as using word information and / or scene information involved in the training process of the ASR encoder and CTC decoder to help train the first model.

[0062] It should be noted that, for the first model, the electronic device can divide the network structure of the first model into shallow network layers and deep network layers. The shallow network layers are used for basic recognition of the scene information, and the deep network layers are used for deep recognition of the scene information. Since the shallow network layers can perform basic recognition of the scene information, the parameters of the shallow network layers are considered fundamental parameters. When using information from related models to help train the first model, the parameters of the shallow network layers can be frozen, thereby focusing on training the deep network layers to improve the recognition performance of the first model.

[0063] For example, by dividing the network structure of the first model, the bottleneck layer and softmax layer are designated as deep network layers, and the other network layers are designated as shallow network layers; the parameters of the shallow network layers are frozen or set to extremely small learning parameters (e.g., set to 5×e). -8 The first model is trained using the word information and / or scene information involved in the training process of the ASR encoder and CTC decoder.

[0064] In this way, in addition to the domain-specific scene information obtained from application information, the first model can be trained to obtain the regular scene information that users need to perform speech recognition through multi-task learning, thereby expanding the scene information. The domain-specific scene information can include scene information obtained from navigation applications using location as a keyword, scene information obtained from telephone applications using contact as a keyword for dialing, or scene information obtained from multimedia applications using movie titles as keywords. The regular scene information can include scene information from education, finance, beauty, or gaming, etc. In other words, by expanding the scene information of the first model through related models, the accuracy of the trained first model in recognizing scene information can be improved.

[0065] After training, the electronic device can determine the scene information corresponding to the voice data by inputting the text data into the first model, such as field scenes like medical scenes, financial scenes, or non-field scenes like shopping and sales.

[0066] In practical applications, for the text data, the electronic device can use a model for recognizing prefixes in the text data to determine the prefix information and contextual association information in the text data.

[0067] Based on this, in one embodiment, determining the second information corresponding to the voice data using the text data includes:

[0068] The text data is input into the second model to obtain contextual association information and prefix information related to the words in the speech data. The second model is used to determine the contextual association information and prefix information related to the words in the input text data.

[0069] The second model can also be called a prefix encoding model, a prefix encoder, or a prefix encoding module, etc. This application does not limit this, as long as its function is implemented.

[0070] Here, before using the second model, it is necessary to train the second model so that the contextual association information and prefix information of words in the text data can be determined by the trained second model; wherein, the second model can be trained by the electronic device or by other devices, and the electronic device uses the trained second model. This application embodiment does not limit the training method of the second model.

[0071] In one embodiment, the second model comprises a model trained based on BERT.

[0072] In practical applications, BERT refers to a pre-trained language model in related technologies that can provide sample datasets, keywords, prefix information, or keyword weights after training. In other words, a network structure can be built on BERT to obtain the second model, and the second model can be trained.

[0073] In practical applications, during the training process, when the amount of sample data is small, the electronic device can use masking strategies and BERT to make full use of the word meaning or word class information in the existing sample dataset (such as the sample dataset obtained from BERT) to obtain information on words with similar contexts, so as to effectively expand the sample data.

[0074] For example, assuming the sample dataset obtained from BERT contains the phrase "I think blueberries are an excellent ingredient for making jam" and the keyword "blueberry," the electronic device can use a mask to obtain a new word, "raspberry," that is similar in context to the keyword "blueberry." Employing information sharing, the electronic device can determine the weight of the new word "raspberry" based on the weight of the keyword "blueberry" obtained from BERT, thus sharing weights for words with similar contexts. Similarly, for prefix information, the electronic device also uses information sharing, leveraging the sample dataset obtained from BERT for information transfer. In this way, the sample data is expanded.

[0075] In practical applications, to achieve accurate keyword recognition, the electronic device assigns weights to keywords in the sample data. However, if the training process solely emphasizes keywords within the domain context, the trained second model will suffer recognition loss for speech content outside the domain, resulting in low recognition accuracy. For example, if the weight of the medical keyword "hepatitis C" is excessively emphasized, the trained second model might incorrectly identify "biscuit" as "hepatitis C" in a food context, leading to low recognition rates in non-domain scenarios.

[0076] To address the aforementioned issues, during training, the electronic device segments the sample data into keywords and prefixes. These segments, along with scene labels, are then input into the second model for training. The trained second model can then determine the association between prefixes and contextual information in the text data. This allows for the determination of the association weights between prefixes and keywords in different scenarios, avoiding the loss of recognition in non-domain scenarios caused by uniformly increasing keyword weights. Specifically, in domain-specific scenarios, the association weight between prefixes and keywords is typically higher, such as 100; in non-domain-specific scenarios, the association weight is typically lower, such as 1.

[0077] In practical applications, after obtaining the first information, the second information, and the text data, the electronic device can accurately identify the keywords in the text data based on the above three types of information.

[0078] Specifically, in one embodiment, the keywords of the voice data are determined using the text data, the first information, and the second information, based on a beam search algorithm.

[0079] In practical applications, the electronic device, based on the beam algorithm, uses the first and second information to determine the association weight between prefix information and keywords in the text data in the current scenario. Using this association weight, the electronic device can effectively identify keywords in the text data and obtain at least one identification result and corresponding score for the text data. The scores of the identification results are arranged in descending order of value. The electronic device can start by selecting N identification results as the final identification result, where N can be set as needed and is an integer greater than or equal to 1.

[0080] Here, after obtaining the final recognition result, the electronic device can also send the final recognition result to relevant devices for subsequent operations, such as summary extraction and intent analysis.

[0081] The speech data processing method provided in this application embodiment acquires speech data to be recognized; converts the speech data into text data based on ASR technology; uses the text data to determine first information and second information corresponding to the speech data, wherein the first information includes scene information corresponding to the speech data, and the second information includes contextual association information and prefix information related to the words in the speech data; and uses the text data, the first information, and the second information to determine the keywords of the speech data. The solution provided in this application embodiment, based on general recognition methods in related technologies, can determine whether the application scenario of the current speech data is a domain scenario (such as the medical field) or a non-domain scenario through the scene information of the speech data; further, by combining the prefixes of hot words (i.e., keywords) in the speech data, it can distinguish between hot words in domain scenarios and general vocabulary in non-domain scenarios, thereby improving the accuracy of hot word recognition.

[0082] The present application will be further described in detail below with reference to application examples.

[0083] This application example presents a hot word recognition system, such as... Figure 2 As shown, based on the relevant recognition system architecture, a scene bias encoder (i.e., the first model mentioned above) and a BERT-based prefix encoder (i.e., the second model mentioned above) are added; wherein, the scene bias encoder is used to determine whether the scene corresponding to the speech input data is a domain scene, and the BERT-based prefix encoder is used to determine the hot word prefixes and their corresponding weights in the speech input data.

[0084] In practical applications, the speech recognition method based on the above-mentioned hot word recognition system includes the following steps:

[0085] Step 1: The user voice data module of the hot word recognition system obtains user voice data (i.e., the voice data mentioned above) from relevant devices and preprocesses the obtained user voice data so that the sampling rate, bit rate or channel data and other parameters of the preprocessed user voice data meet the parameter requirements of the hot word recognition system; the preprocessed user voice data is then sent to the recognition encoding module (ASR Encoder) of the hot word recognition system.

[0086] Step 2: After receiving the preprocessed user speech data, the ASR Encoder identifies the inherent connections between speech features based on the preprocessed speech data;

[0087] Specifically, the ASR Encoder divides the preprocessed user speech data into multiple speech frames; extracts features from multiple speech frames and encodes the extracted features to obtain encoded feature vectors; sends the encoded feature vectors to the CTC Decoder module of the hot word recognition system, and then executes step 3.

[0088] Step 3: After receiving the encoded feature vector, the CTC Decoder determines the correspondence between the user's voice data and the recognized text based on the encoded feature vector; that is, the CTC Decoder can convert the user's voice data into the corresponding recognized text (i.e., text data) based on the encoded feature vector.

[0089] For example, using a trained acoustic model, the CTC Decoder can integrate feature vectors from adjacent speech frames to obtain phonemes (such as initials or finals); using a trained speech model, the CTC Decoder can integrate multiple phonemes to obtain English letters or Chinese characters. Thus, the CTC Decoder can achieve the conversion from feature vectors to recognized text.

[0090] Next, the CTC Decoder inputs the recognized text into the Scene bias Encoder, the BERT-based Prefixes Encoder, and the Search Module of the hot word recognition system.

[0091] Step 4: After receiving the recognized text sent by the CTC Decoder, the Scene bias Encoder predicts the scene of the recognized text;

[0092] Specifically, the Scene bias Encoder uses the recognized text as embedded word information and employs an attention mechanism to effectively identify hot words from the recognized text. Then, based on the contextual information of the hot words, it predicts the scene of the recognized text and obtains the scene bias information. For example, bias information 1 represents the scene of the recognized text as an educational scene, and bias information 2 represents the scene of the recognized text as a gaming scene, etc.

[0093] Step 5: The BERT-based prefix encoding module can output lower association weights between prefix words and hot words in non-domain scenarios and higher association weights between prefix words and hot words in domain scenarios based on the prefix words in the identified text; the association weights between prefix words and hot words in different scenarios are sent to the Search Module.

[0094] In practical applications, during training, on the one hand, by employing BERT and Mask training strategies, the system can identify words with contextual similarities to hot words in existing sample data when the amount of sample data is insufficient. Based on existing hot word weights, information is shared between these contextually similar words, effectively supplementing the hot word sample data. Similarly, for prefix words, the hot word recognition system also draws on information from existing prefix words to effectively supplement the prefix word sample data. On the other hand, by considering prefix words and combining them with scene bias information, the hot word recognition system can train to obtain the association weights between prefix words and hot words in different scenarios.

[0095] It should be noted that the execution order of steps 4 and 5 is not important.

[0096] Step 6: The Search Module receives information from the BERT-based prefix encoding module, the Scene bias Encoder, and the CTC decoder, and performs a search based on beams, enabling accurate recognition of the text.

[0097] Here, because the trained BERT-based prefix coding module can output the association weights between prefix words and hot words in different scenarios, combined with scenario bias information, the Search Module can determine lower association weights between prefix words and hot words in non-domain scenarios and higher association weights in domain scenarios. This avoids uniformly increasing hot word weights and causing recognition loss to other text content while ensuring hot word recognition accuracy. This is because, when using beam search, the Search Module scores different recognition results and selects the N highest-scoring results as the final recognition result. Without considering prefix words, if hot words in the domain scenario are given higher weights, it will lead to recognition loss to other text content in non-domain scenarios. Therefore, by considering the impact of prefix words on recognition rate in domain and non-domain scenarios and combining them with scenario bias information, hot words in the domain scenario and common vocabulary in the non-domain scenario can be separated to assign different association weights to hot words in different scenarios, thus improving the recognition accuracy of speech data.

[0098] Then, the Search Module can return the final recognition results to the device that interacts with the hot word recognition system, so that the device can perform related operations such as summary extraction and intent analysis.

[0099] The hot word recognition system proposed in this application example, based on a general speech recognition system, additionally incorporates a Scene bias Encoder and a BERT-based prefix encoder. The Scene bias Encoder's generalization reasoning of text expressions yields encoding predictions for different scenarios. The prefix encoding module considers the impact of prefix words on recognition accuracy in both domain-specific and non-domain-specific scenarios. This approach avoids the loss of recognition accuracy in non-domain-specific dialogue content caused by uniformly increasing hot word weights, while maintaining high hot word recognition rates, thus establishing a more robust hot word recognition system.

[0100] Secondly, when using the hot word recognition system in this application example for speech recognition, the need for sample data from different scenarios can be reduced. That is, it is not necessary to use sample data from the current scenario as a prerequisite for building the hot word recognition system. Instead, training strategies using BERT and masks can be employed to extract context-similar information from a large amount of existing data, thus achieving information sharing. This not only avoids limitations imposed by the amount and distribution of sample data but also reduces the consumption of memory and computing resources required for recognition.

[0101] In addition, since this application example does not require calculating word frequency thresholds to determine hot words in different scenarios, the hot word recognition system is not limited by the size of the recognized text or the existing hot word database when performing speech recognition, meaning that the hot word recognition system has high robustness.

[0102] To implement the method of the embodiments of this application, the embodiments of this application also provide a voice data processing device, which is installed on an electronic device, such as... Figure 3 As shown, the device includes:

[0103] Acquisition unit 301 is used to acquire the speech data to be recognized;

[0104] The conversion unit 302 is used to convert the voice data into text data based on ASR technology;

[0105] The first processing unit 303 is used to use the text data to determine first information and second information corresponding to the voice data. The first information includes scene information corresponding to the voice data, and the second information includes association information and prefix information related to the context of words in the voice data.

[0106] The second processing unit 304 is used to determine the keywords of the voice data using the text data, the first information and the second information.

[0107] In one embodiment, the first processing unit 303 is used to input the text data into a first model to obtain scene information corresponding to the voice data, and the first model is used to determine the scene information of the input text data.

[0108] In one embodiment, the first model includes a model trained using a multi-task learning approach.

[0109] In one embodiment, the first processing unit 303 is used to input the text data into a second model to obtain contextual association information and prefix information related to the words in the speech data. The second model is used to determine the contextual association information and prefix information related to the words in the input text data.

[0110] In one embodiment, the second model comprises a model trained based on BERT.

[0111] In one embodiment, the first processing unit 303 is configured to:

[0112] The voice data is preprocessed;

[0113] Based on ASR technology, the preprocessed speech data is converted into text data.

[0114] In one embodiment, the keywords of the voice data are determined using the text data, the first information, and the second information, based on a cluster search algorithm.

[0115] In practical applications, the acquisition unit 301 can be implemented by a processor in the voice data processing device combined with a communication interface; the conversion unit 302, the first processing unit 303 and the second processing unit 304 can be implemented by a processor in the voice data processing device.

[0116] It should be noted that the voice data processing device provided in the above embodiments is only illustrated by the division of the above-described program modules when performing voice data processing. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the voice data processing device and the voice data processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0117] Based on the hardware implementation of the above program modules, and in order to implement the method on the electronic device side of the embodiments of this application, the embodiments of this application also provide an electronic device, such as... Figure 4 As shown, the electronic device 400 includes:

[0118] Communication interface 401 enables interaction with other devices;

[0119] The processor 402 is connected to the communication interface 401 to enable interaction with other devices and to execute the methods provided by one or more of the above-mentioned technical solutions when running a computer program;

[0120] The computer program is stored in memory 403.

[0121] Specifically, the processor 402 is used for:

[0122] Acquire the speech data to be recognized;

[0123] Based on ASR technology, the voice data is converted into text data;

[0124] Using the text data, determine the first information and the second information corresponding to the voice data. The first information includes the scene information corresponding to the voice data, and the second information includes the association information and prefix information related to the context of the words in the voice data.

[0125] Using the text data, the first information, and the second information, keywords of the voice data are determined.

[0126] In practical applications, the processor 402 can acquire the voice data through the communication interface 401.

[0127] In one embodiment, the processor 402 is used to input the text data into a first model to obtain scene information corresponding to the voice data, wherein the first model is used to determine the scene information of the input text data.

[0128] In one embodiment, the first model includes a model trained using a multi-task learning approach.

[0129] In one embodiment, the processor 402 is used to input the text data into a second model to obtain contextual association information and prefix information related to the words in the speech data. The second model is used to determine the contextual association information and prefix information related to the words in the input text data.

[0130] In one embodiment, the second model comprises a model trained based on BERT.

[0131] In one embodiment, the processor 402 is configured to:

[0132] The voice data is preprocessed;

[0133] Based on ASR technology, the preprocessed speech data is converted into text data.

[0134] In one embodiment, the keywords of the voice data are determined using the text data, the first information, and the second information, based on a cluster search algorithm.

[0135] It should be noted that the specific processing procedure of processor 402 can be understood by referring to the above method.

[0136] Of course, in practical applications, the various components in electronic device 400 are coupled together through bus system 404. It can be understood that bus system 404 is used to realize the connection and communication between these components. In addition to a data bus, bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 4 The general designated all buses as Bus System 404.

[0137] The memory 403 in this embodiment is used to store various types of data to support the operation of the electronic device 400. Examples of such data include any computer program used to operate on the electronic device 400.

[0138] The methods disclosed in the embodiments of this application can be applied to the processor 402, or implemented by the processor 402. The processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 402 or by instructions in the form of software. The processor 402 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 402 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 403. The processor 402 reads the information in the memory 403 and completes the steps of the aforementioned method in combination with its hardware.

[0139] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0140] It is understood that the memory 403 in this embodiment can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0141] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 403 storing a computer program, which can be executed by the processor 402 of the electronic device 400 to complete the steps of the aforementioned voice data processing method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0142] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0143] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0144] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A voice data processing method, characterized in that, include: Acquire the speech data to be recognized; Based on Automatic Speech Recognition (ASR) technology, the speech data is converted into text data; Using the text data, determine the first information and the second information corresponding to the voice data. The first information includes the scene information corresponding to the voice data, and the second information includes the association information and prefix information related to the context of the words in the voice data. The prefix information is the text content before the keywords in the text data, and the association information is the information of words with similar contexts and their corresponding weights. Using the text data, the first information, and the second information, a cluster search algorithm is used to determine the association weight between the prefix information and the keyword; wherein, in a domain-specific scenario, the association weight is large; in a non-domain-specific scenario, the association weight is small. The keywords of the voice data are determined using the association weights.

2. The method according to claim 1, characterized in that, The step of using the text data to determine the first information corresponding to the voice data includes: The text data is input into the first model to obtain the scene information corresponding to the voice data. The first model is used to determine the scene information of the input text data.

3. The method according to claim 2, characterized in that, The first model includes a model trained using a multi-task learning approach.

4. The method according to claim 1, characterized in that, The step of using the text data to determine the second information corresponding to the voice data includes: The text data is input into the second model to obtain contextual association information and prefix information related to the words in the speech data. The second model is used to determine the contextual association information and prefix information related to the words in the input text data.

5. The method according to claim 4, characterized in that, The second model includes a model trained based on BERT.

6. The method according to claim 1, characterized in that, The process of converting the voice data into text data based on ASR technology includes: The voice data is preprocessed; Based on ASR technology, the preprocessed speech data is converted into text data.

7. A voice data processing device, characterized in that, include: Acquisition unit, used to acquire the speech data to be recognized; A conversion unit is used to convert the voice data into text data based on ASR technology; The first processing unit is used to use the text data to determine the first information and the second information corresponding to the voice data. The first information includes the scene information corresponding to the voice data, and the second information includes the association information and prefix information related to the context of the words in the voice data. The prefix information is the text content before the keywords in the text data, and the association information is the information of words with similar contexts and their corresponding weights. The second processing unit is used to determine the association weight between the prefix information and the keyword based on the bundle search algorithm using the text data, the first information, and the second information; wherein, in a domain scenario, the association weight is large; in a non-domain scenario, the association weight is small; and the keyword of the voice data is determined using the association weight.

8. An electronic device, characterized in that, include: Processor and communication interface; among which, The processor is configured to acquire speech data to be recognized; convert the speech data into text data based on ASR technology; use the text data to determine first information and second information corresponding to the speech data, wherein the first information includes scene information corresponding to the speech data, and the second information includes contextual association information and prefix information related to the words in the speech data, wherein the prefix information is the text content preceding the keyword in the text data, and the association information is the information and corresponding weight of words with similar contexts; and use the text data, the first information, and the second information, based on a cluster search algorithm, to determine the association weight between the prefix information and the keyword; wherein, in a domain-specific scenario, the association weight is large; in a non-domain-specific scenario, the association weight is small. The keywords of the voice data are determined using the association weights.

9. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 6.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images