Voice intention recognition method and device, storage medium, and electronic device

Abstract

Embodiments of the present invention relate to a method and apparatus for speech intent recognition, a storage medium, and electronic equipment, and relate to the technical field of speech semantic recognition. The method comprises: obtaining spectral features corresponding to the speech to be recognized, and calculating the global features of the speech to be recognized based on the spectral features; calculating a text recognition result of the speech to be recognized based on the global features of the speech to be recognized, and calculating a context value for the text recognition result; and recognizing the intent of the speech to be recognized based on the context value of the text recognition result. Embodiments of the present invention improve the accuracy of intent recognition results.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of speech semantic recognition, and specifically, to a speech intention recognition method, a speech intention recognition device, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the rapid development of artificial intelligence technology in recent years, intelligent outbound call systems are increasingly being used to address the problem of repetitive, automated outbound calls, thereby improving the efficiency of manual customer service calls. Intelligent outbound call systems employ voice communication with customers, identifying the intent behind each sentence and initiating appropriate responses. Therefore, accurately and promptly capturing customer intent is crucial to the smooth operation of intelligent outbound call systems.

[0003] Existing intelligent outbound call systems primarily utilize two serially combined models for voice intent recognition: the Automatic Speech Recognition (ASR) model and the Natural Language Processing (NLP) model. Specifically, after receiving a customer's voice signal, the system uses ASR technology to convert it into text. The system then passes this text to the NLP model for processing, ultimately identifying the customer's intent.

[0004] However, the above method has the following drawbacks: since the recognition performance of the above models is independent of each other, it will cause propagation errors between the models, which in turn leads to a low accuracy of the recognition results.

[0005] Therefore, it is necessary to provide a new method and device for speech intention recognition.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0007] The purpose of the present invention is to provide a speech intention recognition method, a speech intention recognition device, a computer-readable storage medium and an electronic device, thereby at least to a certain extent overcoming the problem of low accuracy of recognition results caused by the limitations and defects of related technologies.

[0008] According to one aspect of the present disclosure, a method for speech intent recognition is provided, comprising:

[0009] Acquire a frequency spectrum feature corresponding to the speech to be recognized, and calculate a global feature of the speech to be recognized based on the frequency spectrum feature;

[0010] Calculating a text recognition result of the speech to be recognized based on the global features of the speech to be recognized, and calculating a context value of the text recognition result;

[0011] The intention of the speech to be recognized is recognized according to the context value of the text recognition result.

[0012] In an exemplary embodiment of the present disclosure, the step of calculating the global features of the speech to be recognized based on the spectral features to be recognized includes:

[0013] Inputting the spectral features to be recognized into a convolutional neural network included in a preset intention recognition model to obtain local features included in the speech to be recognized;

[0014] The local features included in the speech to be recognized are input into the self-attention module included in the intention recognition model to obtain the global features of the speech to be recognized.

[0015] In an exemplary embodiment of the present disclosure, calculating the text recognition result of the speech to be recognized based on the global features includes:

[0016] The first fully connected layer included in the intention recognition model is used to classify the global features of the speech to be recognized to obtain a text recognition result of the speech to be recognized.

[0017] In an exemplary embodiment of the present disclosure, calculating the context value of the text recognition result includes:

[0018] The context value of the text recognition result is calculated using the bidirectional long short-term memory network included in the intention recognition model.

[0019] In an exemplary embodiment of the present disclosure, identifying the intention of the speech to be recognized based on the context value of the text recognition result includes:

[0020] Performing dimensionality reduction processing on the context value of the text recognition result using the temporal maximum pooling layer included in the intent recognition model;

[0021] Using the second fully connected layer included in the intention recognition model to classify the context value after the dimensionality reduction process, to obtain an initial intention recognition result of the speech to be recognized;

[0022] The initial intention recognition result is normalized using the normalized exponential function layer included in the intention recognition model to obtain a target intention recognition result.

[0023] In an exemplary embodiment of the present disclosure, the voice intention recognition method further includes:

[0024] Acquire historical speech data, and sample the historical speech data using a preset short-time Fourier transform model to obtain a plurality of standard spectral features;

[0025] Constructing a training data set based on the standard spectral features and the intention category labeling results of the historical speech data;

[0026] The training data set is used to train a preset network model to obtain the intent recognition model.

[0027] In an exemplary embodiment of the present disclosure, the preset network model includes a speech recognition module and an intention prediction module;

[0028] The method of training a preset network model using the training data set to obtain the intent recognition model includes:

[0029] Inputting the standard spectral features in the training data set into the speech recognition module to obtain a standard text of the historical speech data corresponding to the standard spectral features;

[0030] Inputting the standard text into the intention prediction module to obtain intention prediction results of historical speech data corresponding to the standard spectral features;

[0031] A target loss function is constructed according to the intention prediction result and the intention category labeling result of the historical speech data corresponding to the standard spectral features, and the preset network model is trained based on the target loss function to obtain the intention recognition model.

[0032] In an exemplary embodiment of the present disclosure, constructing a target loss function based on the intention prediction result and the intention category labeling result of the historical speech data corresponding to the standard spectral feature includes:

[0033] Constructing a first loss function based on the intention category labeling results of the historical speech data corresponding to the standard spectral features;

[0034] Constructing a second loss function according to the number of categories included in the intent category labeling result, the intent prediction result, and the intent category labeling result;

[0035] constructing a third loss function according to the second norm of the parameters included in the intention prediction module;

[0036] The target loss function is constructed based on the first loss function, the second loss function and the third loss function.

[0037] According to one aspect of the present disclosure, there is provided a speech intention recognition apparatus, comprising:

[0038] A first calculation module is used to obtain a frequency spectrum feature corresponding to the speech to be recognized, and calculate a global feature of the speech to be recognized based on the frequency spectrum feature;

[0039] a second calculation module, configured to calculate a text recognition result of the speech to be recognized based on the global features of the speech to be recognized, and calculate a context value of the text recognition result;

[0040] The intention recognition module is used to recognize the intention of the speech to be recognized based on the context value of the text recognition result.

[0041] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for recognizing speech intention described above is implemented.

[0042] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0043] processor; and

[0044] a memory for storing executable instructions of the processor;

[0045] The processor is configured to execute any one of the above-mentioned speech intention recognition methods by executing the executable instructions.

[0046] A method for recognizing speech intentions provided by an embodiment of the present invention, on the one hand, obtains spectral features to be recognized corresponding to the speech to be recognized, and calculates the global features of the speech to be recognized based on the spectral features to be recognized; and calculates the text recognition results of the speech to be recognized based on the global features of the speech to be recognized, and calculates the context value of the text recognition results; and then recognizes the intention of the speech to be recognized based on the context value of the text recognition results. Since the speech recognition results are obtained directly based on the global features, and the intention of the speech to be recognized can be directly recognized based on the context value of the text recognition results, it can solve the problem in the prior art that the recognition performance of the models is independent of each other, which will lead to the propagation error of text information between models, and thus lead to low accuracy of the recognition results, thereby improving the accuracy of the recognition results; on the other hand, it solves the existing problem that due to the presence of various complex factors such as accents, dialects, background noise interference, etc. in the customer voice signal to be recognized, the text recognition results may be erroneous, resulting in differences between the text recognition results and the information expressed by the actual user, and even text recognition results with opposite intentions, which ultimately lead to errors in the recognized intention results.

[0047] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present invention, and together with the description, serve to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and it is clear that those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0049] Figure 1 The following schematically shows a flow chart of a method for speech intention recognition according to an exemplary embodiment of the present invention.

[0050] Figure 2 The following schematically shows an example structure diagram of an intent recognition model according to an example embodiment of the present invention.

[0051] Figure 3 A flowchart of another method for speech intention recognition according to an exemplary embodiment of the present invention is schematically shown.

[0052] Figure 4 A flowchart of a method for training a preset network model using the training data set to obtain the intent recognition model according to an example embodiment of the present invention is schematically shown.

[0053] Figure 5A flowchart of a method for constructing a target loss function based on intent prediction results and intent category labeling results of historical speech data corresponding to standard spectral features according to an example embodiment of the present invention is schematically shown.

[0054] Figure 6 A flowchart of another method for speech intention recognition according to an exemplary embodiment of the present invention is schematically shown.

[0055] Figure 7 The following schematically shows a block diagram of a speech intention recognition apparatus according to an exemplary embodiment of the present invention.

[0056] Figure 8 An electronic device for implementing the above-mentioned voice intention recognition method according to an exemplary embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0057] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solutions of the present invention may be practiced while omitting one or more of the specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present invention.

[0058] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0059] Existing intelligent outbound call systems primarily utilize two technologies in tandem to identify voice intent: Automatic Speech Recognition (ASR) and Natural Language Processing (NLP). First, after receiving the customer's voice signal, the system uses ASR to convert it into text. The system then passes this text to a natural language processing model for processing, ultimately identifying the customer's intent.

[0060] However, using this serial combination approach is likely to lead to system error propagation issues. For example, assume that in a speech intent recognition system, the recognition performance of the ASR model and the NLP model are independent of each other. If the ASR model has an accuracy of 90% and the NLP model has an accuracy of 90%, the accuracy of the two models in serial recognition of intent is only 81%.

[0061] Furthermore, in real-world scenarios, the customer's voice signal to be recognized may contain complex factors such as accents, dialects, and background noise. This inevitably leads to ASR model errors, resulting in discrepancies between the text information output by the ASR and the actual information expressed by the user, or even in text recognition results that contradict the intended meaning. Because the text information received by the NLP system is already severely biased, even if the system uses a relatively robust NLP model, it will still misjudge the customer's intent, adversely impacting the smooth operation of the intelligent outbound calling system.

[0062] This exemplary embodiment first provides a method for speech intent recognition, which can be run on a server, server cluster, or cloud server, etc. Of course, those skilled in the art can also run the method of the present invention on other platforms as needed, and this exemplary embodiment does not specifically limit this. Figure 1 As shown, the speech intention recognition method may include the following steps:

[0063] Step S110: Acquire the spectral features corresponding to the speech to be recognized, and calculate the global features of the speech to be recognized based on the spectral features;

[0064] Step S120. Calculating a text recognition result of the speech to be recognized based on the global features of the speech to be recognized, and calculating a context value of the text recognition result;

[0065] Step S130: Identify the intention of the speech to be recognized based on the context value of the text recognition result.

[0066] In the above-mentioned speech intention recognition method, on the one hand, by obtaining the spectral features to be recognized corresponding to the speech to be recognized, and calculating the global features of the speech to be recognized based on the spectral features to be recognized; and calculating the text recognition results of the speech to be recognized based on the global features of the speech to be recognized, and calculating the context value of the text recognition results; and then recognizing the intention of the speech to be recognized based on the context value of the text recognition result. Since the speech recognition result is obtained directly based on the global features, and the intention of the speech to be recognized can be directly recognized based on the context value of the text recognition result, it can solve the problem in the prior art that the recognition performance of the models is independent of each other, which will lead to the propagation error of text information between models, and thus lead to low accuracy of the recognition results, thereby improving the accuracy of the recognition results; on the other hand, it solves the existing problem that due to the presence of various complex factors such as accents, dialects, background noise interference, etc. in the customer voice signal to be recognized, the text recognition results may be erroneous, resulting in differences between the text recognition results and the information expressed by the actual user, and even text recognition results with opposite intentions, which ultimately lead to errors in the recognized intention results.

[0067] Hereinafter, various steps involved in the method for speech intention recognition according to an exemplary embodiment of the present invention will be explained and illustrated in detail with reference to the accompanying drawings.

[0068] First, the terms involved in the exemplary embodiments of the present invention are explained and illustrated.

[0069] Automatic Speech Recognition (ASR): technology that converts speech into text.

[0070] Natural Language Processing (NLP): Technology for processing text information, including text intent understanding technology.

[0071] Short-Time Fourier Transform (STFT): A Fourier transform method used to determine the frequency and phase of a local sinusoidal wave of a time-varying signal.

[0072] Convolutional Neural Network (CNN) is a neural network structure that includes convolution or related calculations.

[0073] Self-Attention Block (SAB): A network model structure built based on the self-attention mechanism.

[0074] Fully Connected (FC): A neural network structure that combines the features extracted previously by connecting each node to all nodes in the previous layer.

[0075] Bi-directional Long Short-Term Memory (Bi-LSTM): A network structure composed of a forward LSTM and a backward LSTM.

[0076] Time Max-Pooling (TMP): For time series vectors, the maximum values in different time dimensions are taken to form the final vector result.

[0077] Label Smoothing (LS): A model training method for label error tolerance.

[0078] Secondly, the purpose of the invention and the application scenarios of the exemplary embodiments of the present invention are explained and illustrated. Specifically, in order to address the shortcomings of the serial combination of ASR technology and NLP technology in existing intelligent outbound call systems, the present invention proposes an intelligent outbound call voice intention recognition method. Based on the pre-trained ASR model, this method constructs an end-to-end deep neural network. By fine-tuning the model on a small-scale training set, a model is finally obtained that directly converts voice signals into intention results, avoiding the use of text information as an intermediate result in the serial system, thereby solving the problem of system error propagation.

[0079] Furthermore, the intention recognition model involved in the exemplary embodiment of the present invention is explained and illustrated. Figure 2 As shown, the intention recognition model may include a speech recognition module F s 210 and intention prediction module F c 220, wherein the speech recognition module 210 may include a three-layer convolutional neural network (CNN) 211, a ten-layer self-attention block (SAB) 212, and a first fully connected layer (FC) 213. Furthermore, the parameters involved in the CNN, the SAB, and the FC are initialized using the parameters of the pre-trained ASR model.

[0080] Furthermore, based on the speech recognition module, the exemplary embodiment of the present invention also designs an intention prediction module F cSpecifically, the intention prediction module 220 includes a bidirectional long short-term memory network (Bi-directional LongShort-Term Memory, Bi-LSTM) 221, a time maximum pooling layer (Time Max-Pooling, TMP) 222, a second fully connected layer 223 and a normalized exponential function layer (softmax) 224; among them, the parameters involved in the bidirectional long short-term memory network, the time maximum pooling layer, the second fully connected layer and the normalized exponential function layer are determined by random initialization and are debugged one by one in the later training process.

[0081] It can be known from the above records that in the intent recognition model provided by the present application, only the parameters involved in the intent prediction module need to be debugged, and the parameters involved in the text recognition model do not need to be debugged. Through this method, the speed of model training can be improved. At the same time, it should be supplemented that the text recognition module and the intent prediction module included in the intent recognition model are connected front and back, that is, the input of the bidirectional long short-term memory network is the output of the first fully connected layer, which does not need to be forwarded through the system, and then the intent of the speech is recognized according to the recognized text, realizing end-to-end input and output, that is, the spectral features can be directly input and the intent prediction results can be output, avoiding the problem of system error propagation.

[0082] Further, Figure 3 Another method for speech intention recognition according to an exemplary embodiment of the present invention is schematically shown. Figure 3 As shown, the speech intention recognition method may include steps S310 to S330. In which:

[0083] In step S310, historical speech data is acquired, and the historical speech data is sampled using a preset short-time Fourier transform model to obtain a plurality of standard spectral features.

[0084] In this example embodiment, first, historical voice data is obtained, which may include multiple voice data items. Then, each voice data item is sampled using a traditional short-time Fourier transform (STFT) to obtain multiple standard spectral features. In the process of sampling each voice data item, the window size selected for the short-time Fourier transform can be 20ms, and the window sliding step size can be 10ms. Through the STFT, an 81-dimensional spectral information sequence for each segment of voice data can be obtained. Furthermore, in this example embodiment of the present invention, the sampling rate of the voice data is 16kHz, the duration of each voice data item is between 0.5s and 15s, and the total duration is 50 hours. It should be noted that the above-mentioned window size and window sliding step size can be adjusted according to actual needs, and this example does not impose any special restrictions on this. The above-mentioned values are selected in the example embodiment of the present invention only to take into account the needs of model training and the actual length of the voice data.

[0085] In step S320, a training data set is constructed based on the standard spectral features and the intention category labeling results of the historical speech data.

[0086] Specifically, let the sample of any speech intent recognition training data be: S i ={x i ,y i}, where i∈[i,N], N is the number of training samples in the dataset. i is the standard spectrum feature of each voice data, y i The result of labeling the intent category of the sample, where y i ∈[1,K], K represents the number of all intent types in the dataset. Then, a training dataset can be constructed based on multiple samples.

[0087] In step S330, the preset network model is trained using the training data set to obtain the intent recognition model.

[0088] In this example embodiment, reference Figure 4 As shown, using the training data set to train the preset network model to obtain the intention recognition model may include steps S410 to S430.

[0089] In step S410, the standard spectrum features in the training data set are input into the speech recognition module to obtain the standard text of the historical speech data corresponding to the standard spectrum features.

[0090] In this example embodiment, first, the convolutional neural network included in the speech recognition module is used to extract local features of the historical speech data; second, the self-attention module included in the speech recognition module is used to calculate the global features of the historical speech data based on the local features; finally, the first fully connected layer included in the speech recognition module is used to classify the global features to obtain the standard text of the historical speech data.

[0091] In step S420, the standard text is input into the intention prediction module to obtain the intention prediction result of the historical speech data corresponding to the standard spectrum feature.

[0092] In this example embodiment, first, the context value of the standard text is calculated using the bidirectional long short-term memory network included in the intention prediction module; secondly, the context value is reduced in dimension using the time maximum pooling layer included in the intention prediction module; then, the context value after dimension reduction is classified using the second fully connected layer included in the intention prediction module to obtain the initial intention recognition result of the historical speech data; finally, the context value after dimension reduction is normalized using the normalized exponential function included in the intention prediction module to obtain the final intention prediction result.

[0093] In step S430, a target loss function is constructed according to the intention prediction result and the intention category labeling result of the historical speech data corresponding to the standard spectral feature, and the preset network model is trained based on the target loss function to obtain the intention recognition model.

[0094] In this example embodiment, first, it is necessary to design an optimization objective function. During the model training process, by adjusting the model parameters, the objective function is reduced to convergence, so that the model can classify the intent of the audio. In this example embodiment of the present invention, cross entropy can be designed as the target loss function of the optimization target to perform intent classification. Specifically, refer to Figure 5 As shown, constructing a target loss function based on the intention prediction result and the intention category labeling result of the historical speech data corresponding to the standard spectrum feature may include steps S510 to S540.

[0095] In step S510, a first loss function is constructed based on the intention category labeling results of the historical speech data corresponding to the standard spectral features;

[0096] In step S520, a second loss function is constructed according to the number of categories included in the intention category labeling result, the intention prediction result, and the intention category labeling result;

[0097] In step S530, a third loss function is constructed according to the second norm of the parameters included in the intention prediction module;

[0098] In step S540, the target loss function is constructed according to the first loss function, the second loss function and the third loss function.

[0099] Below, steps S510 to S540 will be explained and illustrated. Specifically, the first loss function L1, the second loss function L2, the third loss function L3, and the target loss function L can be expressed as follows:

[0100]

[0101]

[0102] L3=λ||w c ||; Formula (3)

[0103] L=L1+L2+L3; Formula (4)

[0104] Among them, N b is the number of samples in each training batch, where N b ∈N, and there are N b =32; K is the number of categories included in the intent category labeling results, and there are:

[0105]

[0106] o i =[o i,1 ,o i,2 ,...,o i,K ] is the K-dimensional output vector of the preset network model, o i,k The probability that the intention prediction result of the first historical speech data belongs to the kth category, k∈[1,K], and:

[0107] The intention category labeling result of each i-th historical speech data; ||w c || is the sum of the two norms of the parameters included in the intention prediction module, λ is a constant parameter, and λ=10 -5 .

[0108] It should be noted that since speech intent annotation involves semantic understanding, the annotated data may contain certain errors. Considering the possibility of errors in speech intent annotation, the present invention uses label smoothing to construct a regularization term, that is, the second loss function shown in formula (2). By configuring the second loss function, the accuracy of the model can be further improved, thereby improving the accuracy of the intent recognition results.

[0109] Furthermore, after obtaining the above target loss function, the preset network model can be trained based on the target loss function to obtain the intention recognition model. The intention recognition model F can be shown as the following formula (5):

[0110]

[0111] Then the output of the intent recognition model can be expressed as follows:

[0112] o i =F(x i ,w); Formula (6)

[0113] Among them, w is the speech recognition module F s The parameters w included in s And the intention prediction module F c The parameters w included in c A collection of .

[0114] Below, steps S110 to S130 will be explained and illustrated in conjunction with the above-mentioned intention recognition model.

[0115] In step S110, a frequency spectrum feature corresponding to the speech to be recognized is obtained, and a global feature of the speech to be recognized is calculated based on the frequency spectrum feature;

[0116] In this example embodiment, first, it is necessary to use a traditional short-time Fourier transform (STFT) to sample the speech to be recognized, and then obtain multiple spectral features to be recognized, wherein, in the process of sampling the speech to be recognized, the window size selected by the short-time Fourier transform can be 20ms, and the window sliding step size can be 10ms. An 81-dimensional spectral information sequence of each segment of speech data can be obtained by STFT. Furthermore, in this example embodiment of the present invention, the sampling rate of the speech data is 16kHz, the duration of each speech data is between 0.5s and 15s, and the total duration is 50 hours.

[0117] Secondly, the spectral features to be recognized corresponding to the speech to be recognized are obtained, and the global features of the speech to be recognized are calculated based on the spectral features to be recognized. Specifically, first, the spectral features to be recognized are input into the convolutional neural network included in the preset intention recognition model to obtain the local features included in the speech to be recognized; the local features included in the speech to be recognized are input into the self-attention module included in the intention recognition model to obtain the global features of the speech to be recognized. The convolutional neural network may include three layers, and the self-attention module may include ten layers.

[0118] In step S120, a text recognition result of the speech to be recognized is calculated based on the global features of the speech to be recognized, and a context value of the text recognition result is calculated;

[0119] In this example embodiment, first, the global features of the speech to be recognized are classified using the first fully connected layer included in the intent recognition model to obtain a text recognition result of the speech to be recognized; secondly, the context value of the text recognition result is calculated using the bidirectional long short-term memory network included in the intent recognition model.

[0120] In step S130 , the intention of the speech to be recognized is recognized according to the context value of the text recognition result.

[0121] In this example embodiment, first, the context value of the text recognition result is reduced in dimension using the temporal maximum pooling layer included in the intent recognition model; secondly, the context value after dimensionality reduction is classified using the second fully connected layer included in the intent recognition model to obtain the initial intent recognition result of the speech to be recognized; further, the initial intent recognition result is normalized using the normalized exponential function layer included in the intent recognition model to obtain the target intent recognition result.

[0122] It should be noted here that the specific implementation methods recorded in steps S110 to S130 have been described in detail in the training process of the intent recognition model recorded above, and will not be repeated here.

[0123] The following, combined Figure 6 The speech intention recognition method according to the exemplary embodiment of the present invention is further explained and illustrated. Figure 6 As shown, the speech intention recognition method may include the following steps:

[0124] Step S610: Prepare corresponding training data according to the speech intent recognition requirements of the business scenario;

[0125] Step S620: designing an end-to-end deep neural network model structure based on the pre-trained ASR model;

[0126] Step S630, constructing a loss function for training a deep neural network model for speech intent recognition;

[0127] Step S640: fine-tune the model parameters using the training data to obtain an intent recognition model;

[0128] Step S650: Input the feature spectrum to be recognized obtained according to the speech to be recognized into the intention recognition model to obtain a target intention recognition result.

[0129] The speech intent recognition method provided in the example embodiment of the present invention builds an end-to-end deep neural network based on a pre-trained ASR model. By fine-tuning the model on a small-scale training set, a model is finally obtained that directly converts speech signals into intent results, avoiding the use of text information as an intermediate result in a serial system, thereby solving the problem of system error propagation.

[0130] The exemplary embodiment of the present invention also provides a speech intention recognition device. Figure 7 As shown, the speech intention recognition device may include a first calculation module 710, a second calculation module 720 and an intention recognition module 730. Among them:

[0131] The first calculation module 710 may be configured to obtain a frequency spectrum feature corresponding to the speech to be recognized, and calculate a global feature of the speech to be recognized based on the frequency spectrum feature;

[0132] The second calculation module 720 may be configured to calculate a text recognition result of the speech to be recognized based on the global features of the speech to be recognized, and calculate a context value of the text recognition result;

[0133] The intention recognition module 730 can be used to recognize the intention of the speech to be recognized based on the context value of the text recognition result.

[0134] In an exemplary embodiment of the present disclosure, the step of calculating the global features of the speech to be recognized based on the spectral features to be recognized includes:

[0135] Inputting the spectral features to be recognized into a convolutional neural network included in a preset intention recognition model to obtain local features included in the speech to be recognized;

[0136] The local features included in the speech to be recognized are input into the self-attention module included in the intention recognition model to obtain the global features of the speech to be recognized.

[0137] In an exemplary embodiment of the present disclosure, calculating the text recognition result of the speech to be recognized based on the global features includes:

[0138] The first fully connected layer included in the intention recognition model is used to classify the global features of the speech to be recognized to obtain a text recognition result of the speech to be recognized.

[0139] In an exemplary embodiment of the present disclosure, calculating the context value of the text recognition result includes:

[0140] The context value of the text recognition result is calculated using the bidirectional long short-term memory network included in the intention recognition model.

[0141] In an exemplary embodiment of the present disclosure, identifying the intention of the speech to be recognized based on the context value of the text recognition result includes:

[0142] Performing dimensionality reduction processing on the context value of the text recognition result using the temporal maximum pooling layer included in the intent recognition model;

[0143] Using the second fully connected layer included in the intention recognition model to classify the context value after the dimensionality reduction process, to obtain an initial intention recognition result of the speech to be recognized;

[0144] The initial intention recognition result is normalized using the normalized exponential function layer included in the intention recognition model to obtain a target intention recognition result.

[0145] In an exemplary embodiment of the present disclosure, the speech intention recognition device further includes:

[0146] The sampling and processing module can be used to obtain historical speech data and sample the historical speech data using a preset short-time Fourier transform model to obtain multiple standard spectral features;

[0147] A training data set construction module can be used to construct a training data set based on the standard spectral features and the intention category labeling results of the historical speech data;

[0148] The intention recognition model training module can be used to train a preset network model using the training data set to obtain the intention recognition model.

[0149] In an exemplary embodiment of the present disclosure, the preset network model includes a speech recognition module and an intention prediction module.

[0150] The method of training a preset network model using the training data set to obtain the intent recognition model includes:

[0151] Inputting the standard spectral features in the training data set into the speech recognition module to obtain a standard text of the historical speech data corresponding to the standard spectral features;

[0152] Inputting the standard text into the intention prediction module to obtain intention prediction results of historical speech data corresponding to the standard spectral features;

[0153] A target loss function is constructed according to the intention prediction result and the intention category labeling result of the historical speech data corresponding to the standard spectral features, and the preset network model is trained based on the target loss function to obtain the intention recognition model.

[0154] In an exemplary embodiment of the present disclosure, constructing a target loss function based on the intention prediction result and the intention category labeling result of the historical speech data corresponding to the standard spectral feature includes:

[0155] Constructing a first loss function based on the intention category labeling results of the historical speech data corresponding to the standard spectral features;

[0156] Constructing a second loss function according to the number of categories included in the intent category labeling result, the intent prediction result, and the intent category labeling result;

[0157] constructing a third loss function according to the second norm of the parameters included in the intention prediction module;

[0158] The target loss function is constructed based on the first loss function, the second loss function and the third loss function.

[0159] The specific details of each module in the above-mentioned speech intention recognition device have been described in detail in the corresponding speech intention recognition method, so they will not be repeated here.

[0160] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0161] Furthermore, although the steps of the method of the present invention are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0162] In an exemplary embodiment of the present invention, an electronic device capable of implementing the above method is also provided.

[0163] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Therefore, various aspects of the present invention may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0164] Refer to the following Figure 8 An electronic device 800 according to this embodiment of the present invention will be described. Figure 8 The electronic device 800 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0165] like Figure 8 As shown, electronic device 800 is implemented as a general-purpose computing device. Components of electronic device 800 may include, but are not limited to, the aforementioned at least one processing unit 810, the aforementioned at least one storage unit 820, a bus 830 connecting various system components (including storage unit 820 and processing unit 810), and a display unit 840.

[0166] The storage unit stores program codes that can be executed by the processing unit 810, so that the processing unit 810 performs the steps according to various exemplary embodiments of the present invention described in the "Exemplary Method" section above. For example, the processing unit 810 may perform the following steps: Figure 1 Step S110 shown in: obtaining the spectral features to be recognized corresponding to the speech to be recognized, and calculating the global features of the speech to be recognized based on the spectral features to be recognized; step S120: calculating the text recognition result of the speech to be recognized based on the global features of the speech to be recognized, and calculating the context value of the text recognition result; step S130: recognizing the intention of the speech to be recognized based on the context value of the text recognition result.

[0167] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0168] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0169] Bus 830 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0170] The electronic device 800 can also communicate with one or more external devices 900 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 800, and / or any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication can occur via an input / output (I / O) interface 850. Furthermore, the electronic device 800 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 800, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0171] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present invention.

[0172] In exemplary embodiments of the present invention, a computer-readable storage medium is also provided, storing a program product capable of implementing the methods described above. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product comprising program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to perform the steps according to various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0173] According to an embodiment of the present invention, a program product for implementing the above-mentioned method can be a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, and the program can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0174] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0175] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0176] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0177] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, and the like, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0178] Furthermore, the above-described figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above-described figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0179] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow from the general principles of the invention and include common knowledge or customary techniques in the art not invented herein. The specification and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the claims.

Claims

1. A method for speech intention recognition, characterized in that: include: A preset short-time Fourier transform model is used to sample the speech to be recognized to obtain the spectral features to be recognized, and based on the speech recognition model in the preset intention recognition model, the global features of the speech to be recognized are calculated according to the spectral features to be recognized; wherein, the preset intention recognition model is obtained in the following manner: a preset network model is trained using a training data set to obtain the intention recognition model, including: inputting the standard spectral features in the training data set into the speech recognition model to obtain the standard text of the historical speech data corresponding to the standard spectral features; inputting the standard text into the intention prediction model to obtain the intention prediction results of the historical speech data corresponding to the standard spectral features; constructing a target loss function according to the intention prediction results and the intention category labeling results of the historical speech data corresponding to the standard spectral features, and training the preset network model based on the target loss function to obtain the intention recognition model; Calculating a text recognition result of the speech to be recognized based on the speech recognition model according to the global features of the speech to be recognized, and calculating a context value of the text recognition result based on the intent prediction model in the intent recognition model; Based on the intention prediction model, the intention of the speech to be recognized is identified according to the context value of the text recognition result, including: using the time maximum pooling layer included in the intention prediction model to reduce the dimension of the context value of the text recognition result; using the second fully connected layer included in the intention prediction model to classify the context value after dimension reduction to obtain the initial intention recognition result of the speech to be recognized; using the normalized exponential function layer included in the intention prediction model to normalize the initial intention recognition result to obtain the target intention recognition result.

2. The method for speech intention recognition according to claim 1, wherein: The calculating the global features of the speech to be recognized according to the spectral features to be recognized includes: Inputting the spectral features to be recognized into a convolutional neural network included in a preset intention recognition model to obtain local features included in the speech to be recognized; The local features included in the speech to be recognized are input into the self-attention module included in the intention recognition model to obtain the global features of the speech to be recognized.

3. The method for speech intention recognition according to claim 2, wherein: The calculating the text recognition result of the speech to be recognized according to the global features includes: The first fully connected layer included in the intention recognition model is used to classify the global features of the speech to be recognized to obtain a text recognition result of the speech to be recognized.

4. The method for speech intention recognition according to claim 2, wherein: The calculating the context value of the text recognition result includes: The context value of the text recognition result is calculated using the bidirectional long short-term memory network included in the intention recognition model.

5. The method for speech intention recognition according to claim 1, wherein: The training data set is obtained in the following way: Acquire historical speech data, and sample the historical speech data using a preset short-time Fourier transform model to obtain a plurality of standard spectral features; A training data set is constructed based on the standard spectral features and the intention category labeling results of the historical speech data.

6. The method for speech intention recognition according to claim 1, wherein: Constructing a target loss function based on the intention prediction result and the intention category labeling result of the historical speech data corresponding to the standard spectrum feature includes: Constructing a first loss function based on the intention category labeling results of the historical speech data corresponding to the standard spectral features; Constructing a second loss function according to the number of categories included in the intent category labeling result, the intent prediction result, and the intent category labeling result; constructing a third loss function according to the second norm of the parameters included in the intention prediction module; The target loss function is constructed based on the first loss function, the second loss function and the third loss function.

7. A speech intention recognition device, characterized in that: include: A first calculation module is used to sample the speech to be recognized using a preset short-time Fourier transform model to obtain spectral features to be recognized, and calculate the global features of the speech to be recognized according to the spectral features to be recognized based on the speech recognition model in the preset intention recognition model; wherein the preset intention recognition model is obtained in the following manner: a preset network model is trained using a training data set to obtain the intention recognition model, including: inputting the standard spectral features in the training data set into the speech recognition model to obtain a standard text of the historical speech data corresponding to the standard spectral features; inputting the standard text into the intention prediction model to obtain the intention prediction results of the historical speech data corresponding to the standard spectral features; constructing a target loss function according to the intention prediction results and the intention category labeling results of the historical speech data corresponding to the standard spectral features, and training the preset network model based on the target loss function to obtain the intention recognition model; a second calculation module, configured to calculate a text recognition result of the speech to be recognized based on the speech recognition model according to the global features of the speech to be recognized, and to calculate a context value of the text recognition result based on the intent prediction model in the intent recognition model; An intention recognition module is used to recognize the intention of the speech to be recognized based on the context value of the text recognition result based on the intention prediction model, including: using the time maximum pooling layer included in the intention prediction model to reduce the dimension of the context value of the text recognition result; using the second fully connected layer included in the intention prediction model to classify the context value after dimension reduction to obtain the initial intention recognition result of the speech to be recognized; using the normalized exponential function layer included in the intention prediction model to normalize the initial intention recognition result to obtain the target intention recognition result.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for speech intention recognition according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the speech intention recognition method according to any one of claims 1 to 6 by executing the executable instructions.

Images