Speech recognition classification method and system based on machine learning

Abstract

The invention provides a speech recognition classification method and system based on machine learning, and is applied to the technical field of data processing. The method comprises the steps of performing field feature extraction on application scene information of target voice, and generating a classification task influence factor; performing time-frequency analysis processing on the target voice stream information to generate a standardized voice feature matrix; performing feature dimension screening on the standardized voice feature matrix based on classification demand information of a target user to generate a target acoustic feature vector; performing signal characteristic analysis on the acquisition equipment information of the target voice to generate an equipment compensation characteristic vector; carrying out transfer learning on a preset voice classification model based on the acoustic training set to generate an adaptive voice classification model; and inputting the classification task influence factor, the target acoustic feature vector and the equipment compensation feature vector into an adaptive voice classification model, and performing feature fusion through an attention mechanism to generate voice classification result information with confidence.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a speech recognition classification method and system based on machine learning. Background Art

[0002] The speech signal itself is highly complex. In actual application scenarios, the signal-to-noise ratio of speech signals fluctuates greatly. For example, in noisy streets, shopping malls and other environments, speech will be mixed with a lot of background noise, resulting in a significant reduction in the signal-to-noise ratio, which makes it extremely difficult to accurately extract effective features from speech. At the same time, speaker characteristics vary greatly. The pronunciation habits, accent characteristics and unique vocalization methods of different individuals all increase the difficulty of speech feature extraction. In addition, changes in speech speed will also interfere with the accuracy of feature extraction. Fast speech speed may cause some speech information to be lost, while slow speech speed may reduce the efficiency of feature extraction. Existing speech recognition and classification methods often find it difficult to extract features comprehensively and accurately when dealing with these complex and changeable speech signals, which in turn has a serious impact on the accuracy of subsequent classification.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of this application is to provide a speech recognition classification method and system based on machine learning, which at least overcomes the problems existing in the prior art to a certain extent, by extracting domain features from application scenario information, generating classification task influencing factors including domain-specific vocabulary weights, etc.; processing speech stream information through time-frequency analysis, generating a standardized speech feature matrix; screening matrix features according to user classification requirements, obtaining a target acoustic feature vector including rhythm, phoneme probability distribution and context-related features; analyzing acquisition device information, generating a device compensation feature vector including sampling rate compensation coefficients, etc. The classification task influencing factors, target acoustic feature vectors and device compensation feature vectors are input into an adaptive speech classification model, and features are fused through an attention mechanism to generate target feature templates and attention weights, thereby obtaining a number of speech classification results and their confidence values, and finally generating speech classification result information with confidence, so as to achieve accurate classification of the target speech.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present invention.

[0006] According to one aspect of the present application, a speech recognition classification method based on machine learning is provided, comprising: obtaining speech stream information of a target speech in a preset scenario, application scenario information of the target speech, classification requirement information of a target user, acquisition device information of the target speech, a preset speech classification model and an acoustic training set, wherein the speech stream information of the target speech in the preset scenario comprises audio data under different signal-to-noise ratios, speaker characteristics and speech speed conditions; performing domain feature extraction on the application scenario information of the target speech to generate a classification task influencing factor, wherein the classification task influencing factor comprises a domain-specific vocabulary weight, a real-time requirement parameter and a fault tolerance threshold indicator; performing time-frequency analysis on the target speech stream information to generate a standardized speech feature matrix; and performing classification based on the classification of the target user. The standardized speech feature matrix is ​​screened in feature dimensions according to the class requirement information to generate a target acoustic feature vector, wherein the target acoustic feature vector includes rhythmic features, phoneme probability distribution features and contextual association features; the acquisition device information of the target speech is analyzed for signal characteristics to generate a device compensation feature vector, wherein the device compensation feature vector includes a sampling rate compensation coefficient, a frequency response curve correction parameter and an environmental noise baseline value; transfer learning is performed on the preset speech classification model based on the acoustic training set to generate an adaptive speech classification model; the classification task influencing factor, the target acoustic feature vector and the device compensation feature vector are input into the adaptive speech classification model, feature fusion is performed through the attention mechanism, and speech classification result information with confidence is generated.

[0007] Another aspect of the present application is a speech recognition and classification device based on machine learning, characterized in that it includes: an acquisition module, used to obtain speech stream information of a target speech in a preset scenario, application scenario information of the target speech, classification requirement information of a target user, acquisition device information of the target speech, a preset speech classification model and an acoustic training set, wherein the speech stream information of the target speech in the preset scenario includes audio data under different signal-to-noise ratios, speaker characteristics and speech speed conditions; a processing module, used to extract domain features from the application scenario information of the target speech and generate a classification task influencing factor, wherein the classification task influencing factor includes a domain-specific vocabulary weight, a real-time requirement parameter and a fault tolerance threshold indicator; time-frequency analysis is performed on the target speech stream information to generate a standardized speech feature matrix; based on The target user's classification requirement information is used to screen the feature dimensions of the standardized speech feature matrix to generate a target acoustic feature vector, wherein the target acoustic feature vector includes prosodic features, phoneme probability distribution features and context-related features; the target speech acquisition device information is analyzed for signal characteristics to generate a device compensation feature vector, wherein the device compensation feature vector includes a sampling rate compensation coefficient, a frequency response curve correction parameter and an environmental noise baseline value; transfer learning is performed on the preset speech classification model based on the acoustic training set to generate an adaptive speech classification model; the classification task influencing factor, the target acoustic feature vector and the device compensation feature vector are input into the adaptive speech classification model, and feature fusion is performed through an attention mechanism to generate speech classification result information with confidence.

[0008] According to another aspect of the present application, an electronic device is characterized in that it includes: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the above-mentioned machine learning-based speech recognition classification method by executing the executable instructions.

[0009] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a second processor, the above-mentioned machine learning-based speech recognition classification method is implemented.

[0010] The present application provides a speech recognition classification method and system based on machine learning. The server extracts domain features from application scenario information to generate classification task influencing factors including domain-specific vocabulary weights, etc.; processes speech stream information through time-frequency analysis to generate a standardized speech feature matrix; screens matrix features based on user classification requirements to obtain a target acoustic feature vector including rhythm, phoneme probability distribution and context-related features; and analyzes acquisition device information to generate a device compensation feature vector including a sampling rate compensation coefficient, etc.

[0011] The preset speech classification model is transferred and learned using the acoustic training set. By acquiring data features, generating sampling ratios and sampling features, multiple data group training models are constructed. Based on the test results of the test sample set, if there are risk factors that affect speech recognition classification, the trained model is determined as an adaptive speech classification model using a multi-task learning framework. The classification task influencing factors, target acoustic feature vectors, and device compensation feature vectors are input into the adaptive speech classification model. The features are fused through the attention mechanism to generate target feature templates and attention weights, and then several speech classification results and their confidence values ​​are obtained. Finally, speech classification result information with confidence is generated to achieve accurate classification of the target speech.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flow chart showing a method for speech recognition classification based on machine learning provided in one embodiment of the present application; Figure 2 A structural schematic diagram of a speech recognition and classification device based on machine learning provided in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0014] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0015] Combine the following Figure 1 To describe the speech recognition classification method based on machine learning according to the exemplary embodiment of the present application. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application are applied to any applicable scenario.

[0016] In one implementation, the present application also proposes a speech recognition classification method and system based on machine learning. Figure 1 The flowchart of a speech recognition classification method based on machine learning according to an embodiment of the present application is schematically shown. Figure 1 As shown, the method is applied to a server, comprising: S101, obtaining voice stream information of a target voice in a preset scenario, application scenario information of the target voice, classification requirement information of a target user, acquisition device information of the target voice, a preset voice classification model and an acoustic training set.

[0017] In one implementation, the preset scenario is that a customer calls a customer service hotline for consultation. The voice stream information of the target voice will contain different situations. For example, customer A calls a customer service hotline on a noisy street. At this time, the signal-to-noise ratio of the voice stream information is low, and the background noise may include the sound of vehicles driving, the noise of the crowd, etc.; customer B speaks with a strong local accent, which reflects the characteristics of the speaker; customer C speaks very fast when consulting a question, while customer D speaks slowly. The audio data under different signal-to-noise ratios, speaker characteristics, and speech speed conditions contained in these voice stream information will be acquired by the system. For example, customer A said: "Hey, your product... (background noise interference) always has problems when using it, how do I solve it?" There is both unclear speech caused by the low signal-to-noise ratio and customer A's unique speaking style.

[0018] In this intelligent customer service system, the application scenario information may be that the customer calls the customer service hotline for electronic products. This means that the voice content is likely to revolve around the purchase, use, and after-sales of electronic products. For example, the customer may be consulting about the battery life of the mobile phone, the software installation problem of the computer, etc. This information helps the system to preliminarily determine the field and possible category to which the voice belongs. The system developers set classification requirements and hope to classify customer voice into categories such as product consultation, complaint suggestions, and after-sales repair applications. For example, when the system detects that keywords such as "the product is not easy to use" and "the experience is very poor" are mentioned in the customer's voice, and there is dissatisfaction, it tends to be classified as a complaint suggestion; if the customer asks about the function and usage of the product, it is classified as a product consultation. This is based on the target user (here refers to the user of the system, that is, the customer service department) to quickly classify and process customer voices and improve service efficiency.

[0019] When customers call customer service, they may use different devices. Some customers use Apple phones, whose microphones have specific audio collection characteristics; some use Android phones, and different brands and models of Android phones also have differences in audio collection. For example, some phones may be better at collecting low-frequency sounds, while others are better at collecting high-frequency sounds. Some customers may also use landlines to call, and the audio collection frequency and noise reduction capabilities of landlines are also different from those of mobile phones. The system obtains information about these collection devices and can understand the characteristics of the source device of the voice data, providing a reference for subsequent processing.

[0020] A multi-task learning framework is used to simultaneously optimize the acoustic model and language model as the preset speech classification model. The model as a whole is a deep learning model, which combines convolutional neural network (CNN), recurrent neural network (RNN) and its variants (such as long short-term memory network LSTM, gated recurrent unit GRU) and attention mechanism (Attention Mechanism) and other technologies to achieve efficient speech recognition and classification tasks.

[0021] Regarding the acoustic model part, it includes the input layer, feature extraction layer and sequence modeling layer, as follows: Input layer: receives the classification task influencing factors, target acoustic feature vectors, and device compensation feature vectors as input. After preprocessing, these vectors are input into the model in a specific format to provide the model with information about the speech scene, acoustic characteristics, and acquisition equipment.

[0022] Feature extraction layer: Convolutional neural network (CNN) is used. CNN has the characteristics of local receptive field and weight sharing, and can effectively extract acoustic features from the input speech feature vector. For example, by performing convolution operations in the time and frequency dimensions with convolution kernels of different sizes, the time-frequency features of speech are extracted, and local patterns in speech signals, such as the characteristics of phonemes, are captured. Assuming that 3x3 and 5x5 convolution kernels are used, multiple convolutions are performed on the speech features, and the original speech feature vectors are mapped to different feature spaces to obtain more representative acoustic features.

[0023] Sequence modeling layer: Use a variant of the recurrent neural network (RNN), such as the long short-term memory network (LSTM) or the gated recurrent unit (GRU). This layer can process the temporal information of the speech signal, solve the gradient vanishing or gradient exploding problems of the RNN, and better capture the long-distance dependencies in the speech. Taking LSTM as an example, each LSTM unit contains an input gate, a forget gate, and an output gate. The gating mechanism controls the inflow and outflow of information, thereby remembering long-term information. When processing speech sequences, LSTM can effectively learn information such as the rhythmic features of speech and the sequential relationship of phonemes.

[0024] Regarding the language model part, it includes the embedding layer, recurrent neural network layer, multi-task learning fusion layer, attention mechanism layer and output layer: Embedding layer: Convert words or characters in speech into low-dimensional vector representations, namely word embeddings. For example, using pre-trained word vectors (such as Word2Vec or GloVe), each word is mapped to a vector space of fixed dimension, so that words with similar semantics are close in the vector space, thus providing an effective semantic representation for the language model.

[0025] Recurrent neural network layer: RNN or its variants (such as LSTM, GRU) are also used to model language information. This layer can learn the contextual relationship between words and predict the probability distribution of the next word, thereby understanding the semantic and grammatical structure of speech. By training on a large amount of text data, the language model can capture the statistical laws of language and improve the ability to understand speech content.

[0026] Multi-task learning fusion layer: The acoustic model and language model are fused by sharing some network layers and parameters. For example, in the hidden layer, some neurons receive inputs of acoustic features and language features at the same time, and feature fusion is achieved through weight adjustment. At the same time, multi-task learning is used to set different loss functions for the acoustic model and language model. For example, the loss function of the acoustic model can be the cross entropy loss between the predicted phonemes and the real phonemes, and the loss function of the language model can be the cross entropy loss between the predicted vocabulary and the real vocabulary. Through the back-propagation algorithm, the parameters of the acoustic model and the language model are optimized at the same time, so that the model can learn the language features while learning the acoustic features, thereby improving the overall performance of the model.

[0027] Attention mechanism layer: The attention mechanism is introduced in the output part of the model. The attention mechanism can dynamically assign weights so that the model can pay more attention to the features related to the current task when generating speech classification results. For example, when integrating the classification task influencing factors, target acoustic feature vectors, and device compensation feature vectors, the attention mechanism can automatically adjust the degree of attention to each feature vector according to different speech content and application scenarios, highlight important features, and suppress the influence of irrelevant features, thereby improving the accuracy of classification.

[0028] Output layer: Based on the fused features, the fully connected layer and the Softmax function are used to output the results of speech classification and the confidence value of each classification result. The fully connected layer maps the output of the previous layer to the dimension of the number of classification categories, and the Softmax function converts these outputs into probability distributions, where each probability value represents the probability that the speech belongs to the corresponding category.

[0029] In addition, in the CNN layer, parameters such as the size of the convolution kernel (such as 3x3, 5x5, etc.), step size, and padding method determine the effect of the convolution operation on the extraction of speech features. Smaller convolution kernels can capture local detail features, while larger convolution kernels can obtain more extensive contextual information. The step size determines the distance that the convolution operation slides on the input feature map, and the padding method affects the size of the feature map after convolution. The weight matrix (such as input weight, forget weight, output weight, etc.) and bias term in the LSTM or GRU unit are important parameters for model learning. These parameters determine the way the LSTM / GRU unit processes input information and its memory capacity. For example, the input weight determines the degree of influence of the input information on the current state, and the forget weight controls the degree of retention of the information at the previous moment. The dimension of the word embedding vector determines the representation capacity of the vocabulary in the vector space. Generally speaking, a higher dimension can more accurately represent the semantic information of the vocabulary, but it also increases the computational complexity. For example, common word embedding dimensions are 100, 200, 300, etc.

[0030] The weight matrix and bias term in the fully connected layer map the fused features to the classification category space. The size of the weight matrix is ​​determined by the output dimension and the number of classification categories of the previous layer, and the bias term is used to adjust the baseline value of the output. In the multi-task learning fusion layer, the weight parameters used to balance the loss of the acoustic model and the language model are important hyperparameters. These weight parameters determine the relative importance of the acoustic model and the language model in the overall training process. By adjusting these weights, the performance of the model on different tasks can be optimized.

[0031] The acoustic training set contains speech data of various scenarios, different speakers, and different contents, as well as their corresponding classification labels. For example, the training set contains a large number of speech samples from electronic product customer service scenarios, covering various types of speech such as product consultation, complaint suggestions, and after-sales repair applications, and is annotated with accurate category information. Through these training data, the preset speech classification model learns the relationship between different speech features and classifications, providing a basis for classifying new target speech in subsequent practical applications.

[0032] S102, extracting domain features from application scenario information of the target speech to generate a classification task influencing factor.

[0033] In one implementation, domain features are extracted from the application scenario information of the target speech to generate scene information features described in natural language, wherein the scene information features described in natural language are used to characterize the scene adaptation information of the target speech. When a customer calls the electronic product customer service hotline, the target speech obtained by the system is such as "The tablet computer I bought has been charging very slowly recently. Is there a problem with the battery?" The system extracts domain features from the application scenario information of this speech and generates scene information features described in natural language. For example, key information such as "tablet computer", "slow charging" and "battery problem" are extracted from the speech. This information constitutes the scene information features described in natural language, which are used to characterize the adaptation information of the target speech and the after-sales consultation scenario of electronic products. It shows that this speech is likely to be a consultation on after-sales problems of electronic products (tablet computers) (slow charging and suspected battery failure), which preliminarily reflects the association between speech and specific scenarios.

[0034] Construct a domain knowledge ontology library, which includes the categories of various electronic products (such as tablets, mobile phones, computers, etc.), the components of each product (such as batteries, screens, motherboards, etc.), common problems (such as charging problems, abnormal screen display, etc.), and the relationships between them. For example, it is clear that tablets and batteries are related, and slow charging belongs to charging problems. This ontology library provides a structured knowledge framework for subsequent processing. Based on the domain knowledge ontology library, the scene information features described in natural language are mapped and processed to generate structured features. "Tablet computer" is mapped to the tablet computer category under the electronic product category in the ontology library, "slow charging" is mapped to the charging problem in the common problems, and "battery problem" is mapped to the battery-related problems of tablet computer components. Through this mapping, the unstructured information described in natural language is converted into structured features, which is convenient for subsequent quantification and analysis.

[0035] The structured features are processed to generate domain adaptation index, real-time requirement parameters and fault tolerance threshold indicators.

[0036] The method includes a calculation formula for obtaining a domain adaptation index, and the calculation formula is: ; ; in, Represents the domain adaptation index, Represents the number of occurrences of domain terms, Represents the length of the speech segment, represents the contextual association strength, represents the average time span of the context window, Represents a positive number, used to prevent the denominator of the logarithmic term from being zero. represents the i-th domain term appearing in the target speech, Represents the contextual information of the target speech.

[0037] Suppose that within a certain time window, the number of occurrences of domain terms (such as "tablet" and "battery") in this speech is counted. 2 times, the length of the speech segment It is calculated to be 5 seconds (assuming the total duration of the speech is 5 seconds). By analyzing the context information of the speech, using the formula Calculate the context association strength, assuming that the attention mechanism is used to calculate is 0.8 (the attention mechanism calculates the score based on the degree of association between each domain term and the context), and the average time span of the context window is Set to 3 seconds. To prevent the denominator of the logarithmic term from being zero, take a positive number. is 0.01. Then the domain adaptation index The higher this value is, the more adaptable the voice is to the field of electronic products.

[0038] The domain adaptation index, real-time requirement parameters, and fault tolerance threshold indicators are processed to generate the classification task impact factor; because it is a customer service hotline scenario, customers hope that the problem can be solved as soon as possible, so the real-time requirement is high. It is assumed that the real-time requirement parameters are set according to experience =0.8 (the value range can be determined based on actual business needs and experience, between 0 and 1, and the larger the value, the higher the real-time requirement). Considering that in customer service scenarios, occasional recognition errors may not have a serious impact on the overall service, but too high an error rate cannot be tolerated, it is assumed that the fault tolerance threshold indicator is set The method includes a calculation formula for obtaining the impact factor of the classification task, and the calculation formula is: .in, Represents the domain adaptation index, represents the domain adaptation weight coefficient, Represents the real-time requirement parameters, Represents real-time adjustment parameters, Represents the fault tolerance threshold indicator, Represents a positive number.

[0039] Assume domain adaptation weight coefficient is 1.2 (based on the importance of the domain adaptation index), the real-time adjustment parameter t is 2 (based on the specific quantitative setting of the business for real-time), a positive number Still 0.01, assuming the function is the Sigmoid function. First calculate , and then calculated by Sigmoid function, This classification task impact factor comprehensively considers the domain adaptation degree, real-time requirements, and fault tolerance ability, and is used for the subsequent input adaptive speech classification model, which affects the result of speech classification. It indicates that in this customer service scenario, the comprehensive matching degree of this speech with the current business scenario is relatively high, and the model should give higher weight consideration during classification.

[0040] S103. Perform time-frequency analysis processing on the target speech stream information to generate a standardized speech feature matrix.

[0041] In one implementation, perform time-frequency analysis processing on the target speech stream information to generate a number of speech frames, where the speech frames are used to represent the frame information generated by the speech information in the time dimension. When the customer says the speech "The tablet computer I bought has been charging very slowly recently. Is there a problem with the battery?", the system first performs time-frequency analysis processing on the target speech stream information. Assume that the system sets every 10 milliseconds as a frame to divide the speech. This 5-second-long speech will be divided into 500 frames (5 seconds = 5000 milliseconds, 5000÷10 = 500). These speech frames discretize the speech in the time dimension, and each frame carries part of the speech information in that time period. For example, in a certain frame, it may contain the starting part of the pronunciation of the word "tablet", and another frame contains the middle or ending part of its pronunciation. Through these frames, the characteristics of the speech can be analyzed segment by segment.

[0042] Perform fast Fourier transform and conversion to Mel scale processing on a number of speech frames to generate a Mel spectrum feature sequence, where the Mel spectrum feature sequence includes the frequency components and energy distribution information of the speech, and is used to represent the acoustic characteristics of the speech. Perform fast Fourier transform (FFT) on each of the generated 500 speech frames. Taking one frame as an example, FFT converts this frame of speech from the time domain to the frequency domain, obtaining the amplitude and phase information of this frame of speech at different frequencies. For example, in a certain frame, after FFT, it is found that the speech has a high energy concentration in the 200 - 500 Hz frequency band, which may be related to the pronunciation of the final sound of the word "ping" in "tablet" because different speech phonemes have different energy distribution characteristics in the frequency domain. Convert the frequency domain information after FFT processing to the Mel scale. The human ear's perception of sounds at different frequencies is not linear, and the Mel scale is more in line with the human ear's perception characteristics. Under the Mel scale, remap the speech frequencies. For example, the frequency originally at 1000 Hz may correspond to a specific Mel frequency value under the Mel scale. After performing such a conversion on each frame of speech, a Mel spectrum feature sequence is generated. This sequence contains the frequency components and energy distribution information of the speech, and can better represent the acoustic characteristics of the speech. For example, the word "charge" in "charging" will show a unique frequency component and energy distribution pattern in the Mel spectrum feature sequence, differentiating it from other words.

[0043] The Mel spectrum feature sequence is processed by dynamic time warping to generate a standardized speech feature matrix, in which each row of the standardized speech feature matrix represents the feature vector of a frame of speech, and each column represents the feature dimension, so that the speech features have a unified format in terms of time and feature dimension. Since different customers speak at different speeds, even if the speech has the same content, the length of their Mel spectrum feature sequence on the time axis will be different. For example, customer A speaks faster and customer B speaks slower. They say the same thing "the tablet charges slowly", and the number of frames of the Mel spectrum feature sequence is different. In this case, it is necessary to perform dynamic time warping (DTW) on the Mel spectrum feature sequence. The DTW algorithm will find the optimal alignment path between Mel spectrum feature sequences of different lengths. Assuming that the Mel spectrum feature sequence of customer A's speech has 400 frames after processing, and that of customer B has 600 frames, DTW will warp them to the same length, for example, to 500 frames. After DTW processing, a standardized speech feature matrix is ​​generated. Each row of the matrix represents the feature vector of a frame of speech, and each column represents the feature dimension. For example, the value in the first row and the first column may represent the energy value of the first frame of speech at a specific Mel frequency, the value in the first row and the second column may represent the energy value of the first frame of speech at another Mel frequency, and so on. In this way, regardless of whether the speech comes from a fast-speaking or slow-speaking customer, it can be presented in a unified format, making it easier to filter feature dimensions based on the target user's classification requirements and input them into the adaptive speech classification model for processing.

[0044] S104, screening the standardized speech feature matrix by feature dimension based on the classification requirement information of the target user to generate a target acoustic feature vector.

[0045] In one implementation, the standardized speech feature matrix is ​​screened for feature dimensions based on the target user's classification requirement information to generate prosodic features, phoneme probability distribution features, and context-related features. The target user's classification requirement is to classify customer voice into categories such as product consultation, complaint suggestions, and after-sales repair applications. The system screens the standardized speech feature matrix based on this requirement. From the sentence spoken by the customer, the system identifies prosodic features, phoneme probability distribution features, and context-related features related to the classification. For example, when the customer speaks, his tone becomes heavier and his speech speed is faster. These prosodic manifestations suggest that he may be dissatisfied and are related to the complaint suggestion category; and the specific phonemes contained in the voice, such as "mobile phone", "charging", "slow", etc., constitute the phoneme probability distribution features; at the same time, the context of the entire sentence revolves around the problem of slow charging of mobile phones, reflecting the context-related features.

[0046] Process the prosodic features to generate the mean fundamental frequency and the variance of the fundamental frequency. The mean fundamental frequency and the variance of the fundamental frequency are used to characterize the pitch and the variation range of the intonation, as well as the speech rate value. The system processes the recognized prosodic features. By analyzing the change of the frequency of the speech signal over time, the mean fundamental frequency and the variance of the fundamental frequency are calculated. Suppose that after calculation, the mean fundamental frequency of this segment of speech is relatively high, which means that the overall intonation of the customer's speech is relatively high, reflecting that the customer may be more excited; the larger variance of the fundamental frequency indicates a large variation range of the intonation. Considering the content, it is very likely that the customer is expressing dissatisfaction, further verifying the possibility of a complaint suggestion. In addition, by analyzing the speech duration and the number of syllables, the speech rate value can also be calculated, and it is found that the customer's speech rate is slightly faster than the normal speech rate, which also reflects from the side that the customer's mood is relatively urgent.

[0047] Process the phoneme probability distribution features to generate the probability values of each phoneme appearing in the speech. In this sentence, phonemes such as "shou", "ji", "chong", "dian", "man" appear with a relatively high frequency. The system counts the number of times each phoneme appears in the speech and combines it with the total number of phonemes in the speech to calculate the occurrence probability of each phoneme. For example, the word "shouji" appears frequently, and the phoneme probabilities of "shou" and "ji" are relatively high, indicating that the speech content is related to mobile phones. These phoneme probability distribution values help the system further judge the field and category to which the speech belongs.

[0048] Process the context association features to generate a context association score. The context association score is used to characterize the semantic association degree between the current speech segment and the context before and after. If in the customer service system, the customer also mentioned "this situation has never occurred before" before, then the current speech segment "charging is extremely slow" has a strong semantic association with the previous text, indicating that the problem has occurred recently. The system uses natural language processing technology and semantic analysis algorithms to evaluate this association degree and gives a context association score. Suppose this score is relatively high, indicating that the current speech segment is closely related to the context before and after, and can better assist the system in understanding the customer's intention. For example, it can more clearly judge that this is a feedback about the mobile phone charging problem, rather than other irrelevant topics.

[0049] Process the mean fundamental frequency, the variance of the fundamental frequency, the probability values of each phoneme appearing in the speech, and the context association score to generate a target acoustic feature vector. These values are combined into a vector according to a certain order and weight. Suppose the vector form is [mean fundamental frequency, variance of the fundamental frequency, probability of phoneme 1, probability of phoneme 2,..., context association score]. This target acoustic feature vector contains key information in multiple aspects such as the prosody, phonemes, and context of the speech, and serves as important data for the subsequent input into the adaptive speech classification model, helping the model more accurately classify the speech. In this example, this segment of speech is classified into the complaint suggestion category.

[0050] S105, performing signal characteristic analysis on the acquisition device information of the target speech to generate a device compensation feature vector.

[0051] In one implementation, the sampling rate of the acquisition device information of the target voice is analyzed and processed to generate a sampling rate compensation coefficient. The system obtains the sampling rate information of the mobile phone used by customer A. It is assumed that the sampling rate of the mobile phone is 44.1kHz, and the system default standard sampling rate is 16kHz. In order to make the subsequent voice processing based on a unified standard, the system performs sampling rate analysis and processing on the acquisition device information of the target voice. The sampling rate compensation coefficient is generated by calculating the proportional relationship between the actual sampling rate and the standard sampling rate. In this example, the sampling rate compensation coefficient = 16kHz / 44.1kHz≈0.363. This coefficient is used to subsequently adjust the sampling rate of the voice signal to ensure the consistency and comparability of the voice features.

[0052] The system obtains the frequency response curve information of the acquisition device and the noise signal of the acquisition device when there is no voice input. The system obtains the frequency response curve information of the Android phone, which is a curve that describes the response ability of the mobile phone microphone to sound signals at different frequencies. At the same time, the system records the noise signal of the mobile phone when there is no voice input. For example, in a quiet environment, before customer A calls the customer service number, the mobile phone microphone collects a signal of a short period of silence, which is the noise signal when there is no voice input. The frequency response curve information is processed to generate frequency response curve correction parameters, where the frequency response curve correction parameters are used to characterize the gain deviation of the acquisition device in different frequency bands. The system processes the obtained frequency response curve information. Assume that through analysis, it is found that the signal gain of the mobile phone in the low frequency band (20-200Hz) is 5dB higher than that of the standard device, and the signal gain in the high frequency band (5000-8000Hz) is 3dB lower than that of the standard device. These gain deviations will affect the frequency components of the voice signal, resulting in changes in voice characteristics. In order to correct these deviations, the system generates frequency response curve correction parameters. For example, a -5dB correction parameter is generated for the low frequency band, and a +3dB correction parameter is generated for the high frequency band. These correction parameters are used to adjust the frequency response of the speech signal in the future, so that the frequency characteristics of the speech are closer to the standard state, thereby improving the accuracy of speech recognition.

[0053] The noise signal of the acquisition device is processed when there is no voice input to generate an environmental noise baseline value, where the environmental noise baseline value includes the average power and root mean square value of the noise. The environmental noise baseline value is determined by calculating the average power and root mean square value of the noise signal. Assume that after calculation, the average power of the noise signal is 0.001W and the root mean square value is 0.032V. These values ​​represent the noise intensity level collected by the mobile phone microphone in the current acquisition environment. The environmental noise baseline value is used as a reference for subsequent noise reduction processing of the voice signal to help the system distinguish between voice signals and noise and improve the quality of voice signals.

[0054] The sampling rate compensation coefficient, frequency response curve correction parameter and environmental noise baseline value are processed to generate a device compensation feature vector. Assume that these parameters are combined into a vector form: [sampling rate compensation coefficient, low frequency band frequency response curve correction parameter, high frequency band frequency response curve correction parameter, noise average power, noise root mean square value], that is, [0.363, -5dB, +3dB, 0.001W, 0.032V]. This device compensation feature vector contains key information about the sampling rate, frequency response and noise characteristics of the acquisition device, and is input into the subsequent adaptive speech classification model to compensate for the impact of differences in acquisition devices on the speech signal and improve the accuracy of speech recognition classification. For example, when classifying the voice of customer A, the model will adjust the speech signal according to this device compensation feature vector to avoid classification errors caused by equipment factors and more accurately determine the category of the customer's voice, such as determining whether the customer is consulting about product issues or making complaints and suggestions.

[0055] S106, performing transfer learning on the preset speech classification model based on the acoustic training set to generate an adaptive speech classification model.

[0056] In one implementation, any number of data features in an acoustic training set are obtained. Assume that the acoustic training set contains a large amount of voice data from different customers calling the electronic product customer service hotline, as well as corresponding classification labels (such as product consultation, complaint suggestions, after-sales repair applications, etc.). Randomly select a portion of data features from this training set, such as selecting 1,000 voice data and their corresponding classification labels. These voice data cover different problem consultations, complaints, and after-sales related content of various electronic products (mobile phones, computers, tablets, etc.).

[0057] Based on the number of data features in the acoustic training set, a sampling ratio is generated, and the acoustic training set is processed based on the sampling ratio to generate a preset number of sampling features. Among the 1000 selected data, the number of each type of data features is counted. Assume that there are 500 product consultation voices, 300 complaints and suggestions, and 200 after-sales maintenance applications. In order to ensure that each type of data has a suitable ratio in subsequent training, a sampling ratio is generated according to the number of data. For example, the sampling ratio of product consultation is set to 500 / 1000=0.5, the sampling ratio of complaint and suggestion is set to 300 / 1000=0.3, and the sampling ratio of after-sales maintenance application is set to 200 / 1000=0.2. These sampling ratios are used to control the proportion of each type of data extracted from the training set, avoid a certain type of data dominating the training process, and ensure that the model can learn the characteristics of each type of data. Assume that 500 sampling features are preset to be generated. Data is extracted from the training set according to the above sampling ratio. For product consultation, 500×0.5=250 voice data are extracted; for complaints and suggestions, 500×0.3=150 are extracted; and for after-sales maintenance applications, 500×0.2=100 are extracted. These extracted data constitute a preset number of sampling features, which not only retain the proportional relationship of various types of data in the original training set, but also reduce the amount of data, making it easier to conduct model training in the future.

[0058] Based on any data feature and each sampling feature, multiple data groups are generated, wherein each data group contains a target number of data samples, and at least one data sample includes identification information. Select any data feature from the 1,000 data originally selected, such as a product consultation voice data about the battery life of a mobile phone. Combine this data with the 500 sampling features generated previously. Generate multiple data groups with a certain number as a group (assuming that each group contains 10 data samples). In each data group, at least one data sample has identification information (such as a classification label). For example, the first data group contains 9 different voice data selected from the sampling features and 1 voice data about the battery life of a mobile phone with a classification label of "product consultation"; the second data group also contains 9 different sampling feature voice data and another voice data with a classification label (which may be a complaint suggestion or after-sales repair application), and so on, to generate multiple such data groups.

[0059] The preset speech classification model is trained based on the data samples in the multiple data sets to generate a trained speech classification model. The generated multiple data sets are used as training data and input into the preset speech classification model. The model uses a multi-task learning framework to simultaneously optimize the acoustic model and the language model. During the training process, the acoustic model learns the acoustic features of speech (such as the frequency components and rhythm of speech), and the language model learns the semantic and grammatical structure of speech (such as the relationship between words, the meaning of sentences, etc.). By continuously adjusting the parameters of the model, the model can accurately predict its classification label based on the input speech data. After multiple rounds of training, the trained speech classification model is obtained. For example, the model gradually learns to judge that the speech belongs to the battery life problem consultation in the product consultation based on the keywords such as "battery" and "short battery life" in the speech and the speaker's tone, intonation and other features.

[0060] The trained speech classification model is processed based on the test sample set to generate test results. If the data sample containing identification information in the test result is a risk factor that represents the risk factor affecting speech recognition classification, the trained speech classification model is used as an adaptive speech classification model, where the adaptive speech classification model uses a multi-task learning framework to simultaneously optimize the acoustic model and the language model. Prepare a test sample set that contains some new customer service voice data that have not participated in the training and their classification labels. Input these test data into the trained speech classification model, and the model classifies and predicts each voice data to generate test results. For example, there are 100 voice data in the test sample set, and the model predicts the category (product consultation, complaint suggestion, after-sales repair application, etc.) of each voice and outputs the prediction result.

[0061] Check the data samples that contain identification information (classification labels) in the test results. If there are many misclassified data samples in the test results, and the speech features represented by these samples are considered to be risk factors that affect speech recognition classification (for example, speech recognition errors caused by a specific accent, inaccurate recognition of certain low-frequency words, etc.), then the trained speech classification model is used as an adaptive speech classification model. This means that the model has a certain degree of adaptive ability when facing complex situations that may arise in actual applications, and can better handle these risk factors in subsequent speech classification tasks and improve the overall classification accuracy. For example, if it is found in the test results that many voices with southern accents are misclassified, and southern accents are common in actual customer service scenarios, then this trained model can be used as an adaptive speech classification model to further optimize the classification capabilities of such voices with special accents in actual applications.

[0062] S107, input the classification task influencing factor, the target acoustic feature vector and the device compensation feature vector into the adaptive speech classification model, perform feature fusion through the attention mechanism, and generate speech classification result information with confidence.

[0063] In one implementation, the classification task influencing factor, the target acoustic feature vector, and the device compensation feature vector are processed based on the adaptive speech classification model to generate a target feature template, an attention weight for each dimension, and an attention weight related to the frequency response curve correction parameter of the target frequency band. In the previously set electronic product customer service hotline scenario, suppose a customer calls the customer service number and says, "The battery life of your mobile phone is too bad. It just ran out of power after being fully charged for a while. I am very dissatisfied!" The system inputs the previously obtained classification task influencing factor, target acoustic feature vector, and device compensation feature vector into the adaptive speech classification model. Assuming that the classification task influencing factor indicates that the voice is highly related to the electronic product complaint scenario (such as the high domain adaptation index calculated in the previous text), the target acoustic feature vector contains information such as the customer's excited tone (high fundamental frequency mean, large fundamental frequency variance), the probability distribution of specific phonemes ("mobile phone", "battery", "battery life", etc.) and the contextual association information with the previous text, and the device compensation feature vector records the relevant parameters of the customer's use of the device (such as sampling rate compensation coefficient, frequency response curve correction parameters, etc.).

[0064] Based on these inputs, the adaptive speech classification model first generates a target feature template. This template is an abstract representation of the current speech features. For example, it may integrate a feature combination of electronic product-related vocabulary in the speech, as well as key information reflecting the customer's emotions and context. At the same time, the model calculates the attention weight of each dimension. For acoustic feature dimensions related to "battery life", such as the energy distribution dimension related to the pronunciation of "battery" and "life" in a specific frequency band, a higher attention weight is given, because these dimensions are crucial for judging the speech category; for parameters in the device compensation feature vector, if the response deviation of the device in certain frequency bands may affect the recognition of keywords in the speech, then the device compensation dimensions related to these frequency bands will also receive corresponding attention weights. In addition, attention weights related to the frequency response curve correction parameters of the target frequency bands are also generated. For example, if the key information in the speech is concentrated in certain specific frequency bands, and the device has gain deviations in these frequency bands, then the attention weights of the frequency response curve correction parameters of the corresponding frequency bands will be higher to highlight the attention and correction of these frequency bands.

[0065] The target feature template, the attention weight of each dimension, and the attention weight related to the correction parameters of the frequency response curve of the target frequency band are processed to generate several speech classification results and the confidence values ​​attached to each classification result. According to the generated target feature template and attention weight, the model further processes and generates several speech classification results and the confidence values ​​attached to each classification result. The model will match and judge the speech with pre-set categories (such as product consultation, complaint suggestions, after-sales repair applications, etc.). Because the customer's voice expressed dissatisfaction with the battery life of the mobile phone, the model believes that the voice is more likely to belong to the "complaint suggestion" category through analysis of the input features. At the same time, other categories will also be considered as possible results, but the probability is relatively low.

[0066] For the classification result of "complaints and suggestions", the model gives a confidence value based on a comprehensive consideration of the matching degree of features and the attention weight. Assume that after complex calculations and analysis, the model believes that the confidence level of the voice belonging to the "complaints and suggestions" category is 0.9. This means that the model has a high degree of confidence in this classification result, because the various features in the voice, whether it is vocabulary, tone, or device-related compensation information, strongly point to this being a complaint suggestion. For other possible categories, such as "product consultation", the confidence level given by the model may be only 0.1, because from the overall characteristics, the voice does not meet the typical characteristics of product consultation.

[0067] Process several speech classification results and the confidence values ​​attached to each classification result to generate speech classification result information with confidence. The system will sort out the classification result information, such as "speech classification result: complaint suggestion, confidence: 0.9". This information will be passed to the subsequent customer service business process, and the customer service staff can decide how to handle the speech based on the confidence. If the confidence is high, such as 0.9 in this example, the customer service staff can directly handle it according to the complaint suggestion process and give priority to solving the customer's problem; if the confidence is low, for example, the confidence of a speech classification result is about 0.5, the customer service staff may further verify or use manual assisted judgment to ensure the accuracy of the classification and avoid the decline in service quality caused by misjudgment.

[0068] In this application, the server obtains the voice stream information, application scenario information, user classification requirement information, acquisition equipment information, preset voice classification model and acoustic training set of the target voice in the preset scenario. Among them, the voice stream information covers audio data under different signal-to-noise ratios, speaker characteristics and speaking speed conditions. The application scenario information is subjected to domain feature extraction to generate classification task influencing factors including domain-specific vocabulary weights; the voice stream information is processed through time-frequency analysis to generate a standardized voice feature matrix; the matrix features are screened according to user classification requirements to obtain the target acoustic feature vector including rhythm, phoneme probability distribution and context-related features; the acquisition equipment information is analyzed to generate a device compensation feature vector including sampling rate compensation coefficients.

[0069] The preset speech classification model is transferred and learned using the acoustic training set. By acquiring data features, generating sampling ratios and sampling features, multiple data group training models are constructed. Based on the test results of the test sample set, if there are risk factors that affect speech recognition classification, the trained model is determined as an adaptive speech classification model using a multi-task learning framework. The classification task influencing factors, target acoustic feature vectors, and device compensation feature vectors are input into the adaptive speech classification model. The features are fused through the attention mechanism to generate target feature templates and attention weights, and then several speech classification results and their confidence values ​​are obtained. Finally, speech classification result information with confidence is generated to achieve accurate classification of the target speech.

[0070] In one embodiment, if Figure 2 As shown, the present application also provides a speech recognition classification device based on machine learning, comprising: An acquisition module 201 is used to acquire speech stream information of a target speech in a preset scenario, application scenario information of the target speech, classification requirement information of a target user, acquisition device information of the target speech, a preset speech classification model, and an acoustic training set, wherein the speech stream information of the target speech in the preset scenario includes audio data under different signal-to-noise ratios, speaker characteristics, and speech speed conditions; The processing module 202 is used to extract domain features from the application scenario information of the target speech and generate a classification task influencing factor, wherein the classification task influencing factor includes a domain-specific vocabulary weight, a real-time requirement parameter, and a fault tolerance threshold index; perform time-frequency analysis on the target speech stream information to generate a standardized speech feature matrix; perform feature dimension screening on the standardized speech feature matrix based on the classification requirement information of the target user to generate a target acoustic feature vector, wherein the target acoustic feature vector includes a rhythm feature, a phoneme probability distribution feature, and a context-related feature; perform signal characteristic analysis on the acquisition device information of the target speech and generate a device compensation feature vector, wherein the device compensation feature vector includes a sampling rate compensation coefficient, a frequency response curve correction parameter, and an environmental noise baseline value; perform transfer learning on the preset speech classification model based on the acoustic training set to generate an adaptive speech classification model; input the classification task influencing factor, the target acoustic feature vector, and the device compensation feature vector into the adaptive speech classification model, perform feature fusion through an attention mechanism, and generate speech classification result information with confidence.

[0071] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the machine learning-based speech recognition classification method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0072] Each embodiment in the present application is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for evaluating the speech recognition classification method based on machine learning, electronic devices, electronic devices, and readable storage medium embodiments, since they are basically similar to the above-mentioned embodiments of the speech recognition classification method based on machine learning, the description is relatively simple, and the relevant parts can be referred to the partial description of the embodiments of the speech recognition classification method based on machine learning.

Claims

1. A speech recognition classification method based on machine learning, characterized in that: include: Acquire speech stream information of the target speech in a preset scenario, application scenario information of the target speech, classification requirement information of the target user, acquisition device information of the target speech, a preset speech classification model and an acoustic training set, wherein the speech stream information of the target speech in the preset scenario includes audio data under different signal-to-noise ratios, speaker characteristics and speech speed conditions; Extracting domain features from the application scenario information of the target speech to generate a classification task influencing factor, wherein the classification task influencing factor includes a domain-specific vocabulary weight, a real-time requirement parameter, and a fault tolerance threshold indicator; Performing time-frequency analysis on the target speech stream information to generate a standardized speech feature matrix; Based on the classification requirement information of the target user, the standardized speech feature matrix is ​​screened by feature dimensions to generate a target acoustic feature vector, wherein the target acoustic feature vector includes a prosodic feature, a phoneme probability distribution feature, and a context-related feature; Performing signal characteristic analysis on the target speech acquisition device information to generate a device compensation feature vector, wherein the device compensation feature vector includes a sampling rate compensation coefficient, a frequency response curve correction parameter, and an environmental noise baseline value; Performing transfer learning on the preset speech classification model based on the acoustic training set to generate an adaptive speech classification model; The classification task influencing factor, the target acoustic feature vector and the device compensation feature vector are input into the adaptive speech classification model, and feature fusion is performed through the attention mechanism to generate speech classification result information with confidence.

2. The method according to claim 1, characterized in that Extracting domain features from the application scenario information of the target speech and generating classification task influencing factors include: Performing domain feature extraction on the application scenario information of the target speech to generate scenario information features described in natural language, wherein the scenario information features described in natural language are used to characterize scenario adaptation information of the target speech; Build domain knowledge ontology library; Mapping the scene information features described in the natural language based on the domain knowledge ontology library to generate structured features; Processing the structured features to generate a domain adaptation index, a real-time requirement parameter, and a fault tolerance threshold index; Processing the domain adaptation index, the real-time requirement parameter and the fault tolerance threshold index to generate a classification task impact factor; The method includes a calculation formula for obtaining a domain adaptation index, and the calculation formula is: ； ； in, Represents the domain adaptation index, Represents the number of occurrences of domain terms, Represents the length of the speech segment, represents the contextual association strength, represents the average time span of the context window, Represents a positive number, used to prevent the denominator of the logarithmic term from being zero. represents the i-th domain term appearing in the target speech, Represents the contextual information of the target speech; The method includes a calculation formula for obtaining a classification task impact factor, and the calculation formula is: ； in, Represents the domain adaptation index, represents the domain adaptation weight coefficient, Represents the real-time requirement parameters, Represents real-time adjustment parameters, Represents the fault tolerance threshold indicator, Represents a positive number.

3. The method according to claim 1, characterized in that Performing time-frequency analysis on the target voice stream information to generate a standardized voice feature matrix, including: Performing time-frequency analysis on the target voice stream information to generate a plurality of voice frames, wherein the voice frames are used to represent frame information generated by the voice information in the time dimension; Performing fast Fourier transform on several speech frames and converting them to Mel scale to generate Mel spectrum feature sequence, wherein the Mel spectrum feature sequence includes frequency components and energy distribution information of speech, which is used to characterize the acoustic characteristics of speech; The Mel spectrum feature sequence is dynamically time-warped to generate a standardized speech feature matrix, wherein each row of the standardized speech feature matrix represents a feature vector of a frame of speech, and each column represents a feature dimension, so that the speech features have a unified format in terms of time and feature dimension.

4. The method according to claim 3, characterized in that The standardized speech feature matrix is ​​screened for feature dimensions based on the classification requirement information of the target user to generate a target acoustic feature vector, including: Based on the classification demand information of the target user, the standardized speech feature matrix is ​​screened for feature dimensions to generate prosodic features, phoneme probability distribution features, and context-related features; Processing the prosodic features to generate a fundamental frequency mean and a fundamental frequency variance, wherein the fundamental frequency mean and the fundamental frequency variance are used to characterize the pitch and variation range of the intonation, as well as the speech speed value; Processing the phoneme probability distribution feature to generate a probability value of each phoneme appearing in the speech; Processing the context-related features to generate a context-related score, wherein the context-related score is used to represent the degree of semantic relevance between the current speech segment and the surrounding context; The fundamental frequency mean, fundamental frequency variance, probability value of each phoneme appearing in speech and context association score are processed to generate a target acoustic feature vector.

5. The method according to claim 1, characterized in that Performing signal characteristic analysis on the acquisition device information of the target speech to generate a device compensation feature vector includes: Performing sampling rate analysis on the acquisition device information of the target speech to generate a sampling rate compensation coefficient; Obtain frequency response curve information of the acquisition device and the noise signal of the acquisition device when there is no voice input; Processing the frequency response curve information to generate frequency response curve correction parameters, wherein the frequency response curve correction parameters are used to characterize the gain deviation of the acquisition device in different frequency bands; Processing the noise signal of the acquisition device when there is no voice input to generate an environmental noise baseline value, wherein the environmental noise baseline value includes an average power and a root mean square value of the noise; The sampling rate compensation coefficient, the frequency response curve correction parameter and the ambient noise baseline value are processed to generate a device compensation feature vector.

6. The method according to claim 1, characterized in that The method further comprises: performing transfer learning on the preset speech classification model based on the acoustic training set to generate an adaptive speech classification model, including: Get any number of data features in the acoustic training set; Generate a sampling ratio based on the number of each data feature in the acoustic training set; Processing the acoustic training set based on the sampling ratio to generate a preset number of sampling features; Processing based on any data feature and each sampling feature to generate multiple data groups, wherein each data group includes a target number of data samples and at least one data sample includes identification information; Training the preset speech classification model based on data samples in the multiple data groups to generate a trained speech classification model; Processing the trained speech classification model based on the test sample set to generate a test result; If the data sample containing identification information in the test result represents a risk factor affecting speech recognition classification, the trained speech classification model is used as an adaptive speech classification model, wherein the adaptive speech classification model adopts a multi-task learning framework to synchronously optimize the acoustic model and the language model.

7. The method according to claim 1, characterized in that Inputting the classification task influencing factor, the target acoustic feature vector and the device compensation feature vector into the adaptive speech classification model, performing feature fusion through an attention mechanism, and generating speech classification result information with confidence, including: Processing the classification task influencing factor, the target acoustic feature vector and the device compensation feature vector based on the adaptive speech classification model to generate a target feature template, an attention weight of each dimension, and an attention weight related to a frequency response curve correction parameter of a target frequency band; Processing the target feature template, the attention weight of each dimension, and the attention weight related to the correction parameter of the frequency response curve of the target frequency band, generating a number of speech classification results and a confidence value attached to each classification result; A number of speech classification results and the confidence values ​​attached to each classification result are processed to generate speech classification result information with confidence.

8. A speech recognition and classification device based on machine learning, characterized in that: The device comprises: An acquisition module is used to acquire speech stream information of a target speech in a preset scenario, application scenario information of the target speech, classification requirement information of a target user, acquisition device information of the target speech, a preset speech classification model, and an acoustic training set, wherein the speech stream information of the target speech in the preset scenario includes audio data under different signal-to-noise ratios, speaker characteristics, and speech speed conditions; The processing module is used to extract domain features from the application scenario information of the target speech and generate a classification task influencing factor, wherein the classification task influencing factor includes a domain-specific vocabulary weight, a real-time requirement parameter, and a fault tolerance threshold index; perform time-frequency analysis on the target speech stream information to generate a standardized speech feature matrix; perform feature dimension screening on the standardized speech feature matrix based on the classification requirement information of the target user to generate a target acoustic feature vector, wherein the target acoustic feature vector includes a rhythm feature, a phoneme probability distribution feature, and a context-related feature; perform signal characteristic analysis on the acquisition device information of the target speech and generate a device compensation feature vector, wherein the device compensation feature vector includes a sampling rate compensation coefficient, a frequency response curve correction parameter, and an environmental noise baseline value; perform transfer learning on the preset speech classification model based on the acoustic training set to generate an adaptive speech classification model; input the classification task influencing factor, the target acoustic feature vector, and the device compensation feature vector into the adaptive speech classification model, perform feature fusion through an attention mechanism, and generate speech classification result information with confidence.

9. An electronic device, characterized in that: include: a first processor; and a memory for storing executable instructions of the first processor; The first processor is configured to execute the machine learning-based speech recognition classification method described in any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the second processor, the speech recognition classification method based on machine learning described in any one of claims 1 to 7 is implemented.

Images