Voice recognition method, device, smart speaker, home appliance, electronic device and medium

Abstract

The embodiments of the present application provide a speech recognition method, apparatus, smart speaker, smart home appliance, electronic device, and computer-readable storage medium, relating to the field of artificial intelligence. The speech recognition method includes: inputting speech frame data of a preset length into a speech recognition model, wherein the preset length is determined based on the receptive field size of the starting convolutional layer of the first network portion of the speech recognition model corresponding to the input data size of the second network portion of the speech recognition model; and recognizing the speech frame data through the speech recognition model to obtain text information corresponding to the speech frame data. The present application can improve the efficiency of speech signal processing and reduce latency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular to a speech recognition method, a speech recognition device, a smart speaker, a smart home appliance, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the research and advancement of artificial intelligence technology, deep learning-based algorithms have made tremendous progress in areas such as visual imagery, speech recognition, and natural language processing. Among them, speech recognition technology, which enables computers to hear, see, speak, and feel, is the future direction of human-computer interaction.

[0003] Currently, combining artificial intelligence with edge devices can bring new interactive experiences. However, when performing voice recognition through edge devices, due to their low computing power and small memory, the processing speed is slow and the delay in obtaining voice recognition results is long. Summary of the Invention

[0004] The purpose of this application is to propose a speech recognition method, a speech recognition device, a smart speaker, a smart home appliance, an electronic device and a computer-readable storage medium, which are used to solve the technical problems of slow processing speed and long delay time when recognizing speech signals in the prior art.

[0005] According to a first aspect of an embodiment of the present application, a speech recognition method is provided, comprising:

[0006] Inputting speech frame data of a preset length into a speech recognition model, wherein the preset length is determined based on a receptive field size of a starting convolutional layer of a first network part in the speech recognition model corresponding to an input data size of a second network part in the speech recognition model;

[0007] The speech frame data is recognized by the speech recognition model to obtain text information corresponding to the speech frame data.

[0008] According to a second aspect of an embodiment of the present application, a speech recognition method is provided, including:

[0009] Receive a voice command input by a user and convert it into voice frame data, and stream the voice frame data into a voice recognition model according to a preset length, wherein the preset length is determined based on the receptive field size of the starting convolutional layer of the first network part of the voice recognition model corresponding to the input data size of the second network part of the voice recognition model;

[0010] Recognize the speech frame data using the speech recognition model to obtain text information corresponding to the speech frame data;

[0011] According to the text information, reply information corresponding to the voice instruction is determined, and the reply information is converted into voice data and outputted by voice.

[0012] According to a third aspect of an embodiment of the present application, a speech recognition device is provided, including:

[0013] A speech frame data input module, configured to input speech frame data of a preset length into a speech recognition model, wherein the preset length is determined based on the receptive field size of the starting convolutional layer of the first network part of the speech recognition model corresponding to the input data size of the second network part of the speech recognition model;

[0014] The speech frame processing module is used to recognize the speech frame data through the speech recognition model to obtain text information corresponding to the speech frame data.

[0015] According to a fourth aspect of the embodiments of the present application, a smart speaker is provided, including:

[0016] A microphone is used to obtain user input voice commands and convert them into voice frame data;

[0017] a processor configured to input the speech frame data into the speech recognition model according to a preset length, recognize the speech frame data through the speech recognition model, and obtain text information corresponding to the speech frame data; wherein the preset length is determined based on the receptive field size of the starting convolutional layer of the first network part of the speech recognition model corresponding to the input data size of the second network part of the speech recognition model; and determine, based on the text information, reply information corresponding to the speech command;

[0018] The speaker is used to convert the reply information into voice data and output the voice data.

[0019] According to a fifth aspect of the embodiments of the present application, there is provided a smart home appliance, including:

[0020] A microphone is used to obtain user input voice commands and convert them into voice frame data;

[0021] A processor, configured to input speech frame data of a preset length into the speech recognition model, recognize the speech frame data through the speech recognition model, and obtain text information corresponding to the speech frame data; wherein the preset length is determined based on the receptive field size of the starting convolutional layer of the first network part of the speech recognition model corresponding to the input data size of the second network part of the speech recognition model;

[0022] The processor is further configured to control the smart home appliance to execute the operation indicated by the voice instruction according to the text information.

[0023] According to the sixth aspect of the embodiments of the present application, an electronic device is provided, comprising: one or more processors; a computer-readable storage medium configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the speech recognition method as described in the first aspect or the second aspect of the above-mentioned embodiments.

[0024] According to the seventh aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the speech recognition method as described in the first aspect or the second aspect of the above embodiments is implemented.

[0025] According to the speech recognition method provided in the embodiment of the present application, the length of the speech frame data to be processed can be determined based on the input data size of the second network part in the speech recognition model corresponding to the receptive field size of the starting convolution layer of the first network part. On the one hand, generally speaking, the input data size of the second network part is usually the output data size of the first network part. Based on this, the receptive field size of the starting convolution layer of the first network part is determined based on the input data size, and the length of the input speech frame data is determined accordingly, so that the processing output of the speech frame data of this length by the first network part can meet the input requirements of the second network part. On the other hand, since the receptive field size is usually much smaller than the length of the original complete speech frame data, the length of the currently processed speech frame data is much smaller than the length of the original complete speech frame data, which greatly reduces the amount of data processed by the speech recognition model each time, improves the data processing speed, and reduces the data processing delay. Therefore, in the process of voice interaction with the user, the user's voice data can be recognized more quickly, and the user's voice interaction experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0027] Figure 1 A schematic diagram of a speech recognition method in related art;

[0028] Figure 2 A flow chart of the speech recognition method in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of a convolution operation;

[0030] Figure 4a This is a schematic diagram of an exemplary scenario in an embodiment of the present application;

[0031] Figure 4bA schematic diagram for implementing the above-mentioned speech recognition method;

[0032] Figure 5 This is another flow chart of the speech recognition method in the embodiment of the present application;

[0033] Figure 6 A schematic diagram of a speech recognition method in an embodiment of the present application;

[0034] Figure 7 This is a flow chart of recognizing speech frame data using a speech recognition model in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of a convolutional neural network processing adjacent speech frame data in an embodiment of the present application;

[0036] Figure 9 This is another flow chart of the speech recognition method in the embodiment of the present application;

[0037] Figure 10a A schematic diagram of the structure of a speech recognition device in an embodiment of the present application;

[0038] Figure 10b This is a structural diagram of the smart speaker in the embodiment of this application;

[0039] Figure 10c This is a schematic diagram of the structure of the smart home appliance in the embodiment of this application;

[0040] Figure 11 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application;

[0041] Figure 12 This is the hardware structure of the electronic device in the embodiment of this application. DETAILED DESCRIPTION

[0042] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely configured to explain the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only the portions relevant to the relevant invention are shown in the accompanying drawings.

[0043] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] Artificial intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve desired results. As a key branch of AI, speech processing technology has widespread applications in various fields.

[0045] Key speech processing technologies include automatic speech recognition, speech synthesis, and voiceprint recognition. Automatic speech recognition converts speech data into text. This technology is increasingly being used in instant messaging applications, search engines, in-vehicle systems, and smart electronic devices.

[0046] Furthermore, IoT technology enables real-time awareness of the current operating status of each "object." However, this process generates a large amount of data, placing significant processing pressure on the cloud. Deploying edge devices (anything outside the cloud) allows for local versions of cloud-based services, alleviating this data processing pressure.

[0047] At present, the combination of artificial intelligence and edge devices can bring new interactive experiences, and edge devices are entering a new era of end intelligence.

[0048] For example, in speech processing technology, the process of recognizing speech signals can be performed either in the cloud or on edge devices. When deep learning algorithms, which require high computational and storage resources, are deployed in the cloud, the speech recognition process is shorter and latency is lower due to the cloud's higher computing power and faster data processing. However, when deep learning algorithms, which require high computational and storage resources, are deployed on edge devices, speech recognition takes longer due to their lower computing power and memory. Consequently, in practical applications, such as for smart electronic devices, there is a longer delay between receiving the corresponding text information after a user speaks and the response time, impacting the user experience.

[0049] Specifically, in the related art, when recognizing voice signals, an end-to-end technical solution can be adopted. First, the voice recognition model can be trained based on multiple complete sample sentences and the text information corresponding to the sample sentences. The complete sample sentences can be set according to the actual scenario. For example, for smart electronic devices, the duration of the voice signal input by the user is generally not more than 10 seconds, and the duration of the sample sentences can be set to 10 seconds. For example, when the duration of the sample sentence is greater than 10 seconds, the sample sentence can be truncated. When the duration of the sample sentence is less than 10 seconds, the sample sentence can be padded with zeros, that is, multiple zero frames are added after the sample sentence to make the duration of the sample sentence 10 seconds.

[0050] Later, during the model application process, after the user finishes speaking a sentence, the complete voice signal input by the user can be obtained, and the complete voice frame data corresponding to the voice signal is input into the speech recognition model. After passing through the convolutional neural network, long short-term memory network and decoder, the corresponding text information is obtained. Figure 1 , Figure 1 This is a schematic diagram of the speech recognition method in the related art. Of course, the network structure of the convolutional neural network can also be compared with Figure 1 The network structure shown is different, for example, it can include three convolution layers and one pooling layer. Or, the convolution kernel or step size used in the convolution operation is different, etc. This application does not make specific restrictions on this, and only uses Figure 1 Take this as an example to illustrate.

[0051] It can be seen that the data input to the convolutional neural network is complete speech frame data, with each square representing one frame of speech frame data. Furthermore, the complete speech frame data (usually 320 frames of speech frame data) input to the convolutional neural network undergoes four convolutions and one pooling process, outputting multiple speech feature data. Subsequently, these multiple speech feature data are processed one by one by the long short-term memory network. However, the above process takes a long time to process complete speech frame data, resulting in a large delay.

[0052] In order to solve the above problems, the embodiments of the present application provide a speech recognition method, a speech recognition device, a smart speaker, a smart home appliance, an electronic device and a computer-readable storage medium, which can increase the speed of speech recognition, reduce latency, and enhance user experience.

[0053] The solution provided in the embodiments of this application involves artificial intelligence speech recognition technology, which is specifically illustrated by the following embodiments:

[0054] Reference Figure 2 , shows a flow chart of the speech recognition method according to an embodiment of the present application, which may include the following steps:

[0055] Step S210: Input speech frame data of a preset length into the speech recognition model, wherein the preset length is determined according to the receptive field size of the starting convolutional layer of the first network part in the speech recognition model corresponding to the input data size of the second network part in the speech recognition model.

[0056] It is understood that a speech signal is composed of speech frame data arranged in chronological order, and the length of each speech frame data is usually fixed, such as 20 milliseconds or 30 milliseconds. Therefore, if the total duration of the speech signal is 6 seconds and the length of each speech frame data is 30 milliseconds, the number of speech frame data contained in the speech signal is 200.

[0057] In an embodiment of the present application, speech recognition can be performed in a streaming manner. That is, speech recognition is performed during the process of voice interaction between the user and the intelligent electronic device, or it can be understood as recognition while speaking, rather than performing speech recognition after the user has finished speaking. Since the speech signal generated during the user's speaking process is constantly increasing, it is possible to continuously obtain speech frame data of a preset length for speech recognition until the user has finished speaking.

[0058] It should be noted that the preset length can be determined according to the network structure of the speech recognition model. Different network structures will result in different preset lengths for speech recognition. Specifically, the network structure of the speech recognition model includes at least a first network part and a second network part. Figure 1 In the example shown, the speech recognition model also includes a decoder. Speech frame data is processed by the first network before being input to the second network. Based on the input data size of the second network, which is also the output data size of the first network, the receptive field size of the starting convolutional layer of the first network can be determined.

[0059] In convolutional neural networks, the receptive field is the size of the area where the feature points on the feature map output by each convolutional layer of the convolutional neural network are mapped on the input data area, that is, the area on the input data area corresponding to a point on the feature map.

[0060] For example, if Figure 3 As shown in the figure, a network structure consists of three convolutional layers: the first layer inputs 7×7 data, and after three convolution operations, the fourth layer outputs a 1×1 data, where the convolution kernel size is 3×3, the convolution kernel stride is 1, and the padding is 0. It can be determined that the receptive fields of the 1×1 data corresponding to the third, second, and first layers are 3, 5, and 7, respectively. It can be seen that the receptive fields of the final output data corresponding to different convolutional layers are different.

[0061] In the embodiment of the present application, the output of the first network portion can be the input of the second network portion. In order to generate data of the size required by the second network portion, data of a preset length can be directly obtained. For example, the preset length can be equal to the size of the receptive field corresponding to the starting convolutional layer. Of course, the preset length is less than the total length of the complete speech frame data.

[0062] That is, when specifically determining the preset length, the receptive field size of the starting convolutional layer can be deduced based on the input size of the second part, and the preset length can be determined accordingly.

[0063] Step S220: Recognize the speech frame data through a speech recognition model to obtain text information corresponding to the speech frame data.

[0064] For example, Figure 1 The model structure shown is an example. Speech frames of a preset length can be recognized sequentially through the first and second network components of the speech recognition model to produce an intermediate speech recognition result. This intermediate speech recognition result can be the output of the acoustic model (AM) used in speech recognition, such as the probability of each phoneme. The corresponding text information is then derived from the intermediate speech recognition result, such as by decoding the intermediate speech recognition result. This allows for real-time recognition of shorter speech frames, improving speech recognition speed and reducing recognition latency.

[0065] An exemplary scenario of the above process is as follows Figure 4a As shown in the figure, the user's voice is continuously input into the first network component of the speech recognition model in the form of speech frame data, such as the convolutional neural network component. For example, the length of each input is 12 frames of speech frame data. After entering the speech recognition model, these 12 frames of speech frame data are sequentially passed through four convolutional layers and one pooling layer, and finally the speech feature data corresponding to the 12 frames of speech frame data is output. This speech feature data is then input into the second network component of the speech recognition model, such as the long short-term memory network component, to obtain intermediate speech recognition results, such as the probabilities of individual phonemes. The long short-term memory network is particularly effective at processing data containing sequence information. This data is then streamed into the decoder. After the decoder decodes all the intermediate speech recognition results, the text information corresponding to the user's voice data is obtained.

[0066] As can be seen from this process, since only 12 frames of speech frame data are input each time, the corresponding intermediate speech recognition results are obtained, and the corresponding text information is decoded and output, the delay perceived by the user is the time spent calculating the last 12 frames. Compared with the end-to-end network, the delay is reduced to 12 / 320 of the original.

[0067] The speech recognition method of the embodiment of the present application can determine the length of the speech frame data to be processed based on the receptive field size of the starting convolution layer of the first network part corresponding to the input data size of the second network part in the speech recognition model. On the one hand, the input data size of the second network part is usually the output data size of the first network part. Based on this, the receptive field size of the starting convolution layer of the first network part is determined based on the input data size, and the length of the input speech frame data is determined accordingly, so that the processing output of the speech frame data of this length by the first network part can meet the input requirements of the second network part. On the other hand, since the receptive field size is usually much smaller than the length of the original complete speech frame data, the length of the currently processed speech frame data is much smaller than the length of the original complete speech frame data, which greatly reduces the amount of data processed each time by the speech recognition model, improves the data processing speed, and reduces the data processing delay. Therefore, in the process of voice interaction with the user, the user's voice data can be recognized more quickly, and the user's voice interaction experience is improved.

[0068] Figure 4b To implement a schematic diagram of the above-mentioned speech recognition method, the following will refer to Figure 4b The schematic diagram shown uses a specific scenario as an example to illustrate the main operation process in the above steps S210 to S220.

[0069] In this example, the length of the real-time voice signal can be any length. Assuming that the duration of a voice frame data is 30 milliseconds, the number of voice frames contained in the voice signal can be 100, 120, or 200, etc. Of course, the voice signal corresponds to a maximum length, which can be set according to actual needs. For example, for a certain smart electronic product, the duration of a user's speech generally does not exceed 10 seconds. Here, the maximum length of the voice signal can be set to 320 frames, or 9.6 seconds. In other words, the maximum length of speech that can be processed is 9.6 seconds, and the portion of the user's speech that exceeds 9.6 seconds can be unprocessed.

[0070] The speech recognition model may include: a convolutional neural network (first network component), a long short-term memory network (long short-term memory network) (second network component), and a decoder. The convolutional neural network extracts speech features from speech frame data, and the long short-term memory network further processes these features to produce intermediate speech recognition results. Finally, the decoder decodes the data and produces the corresponding text information. Assuming that the input data of the long short-term memory network corresponds to 12 frames of speech frame data at the starting convolution layer of the convolutional neural network, then during a user's speech, speech frames of length 12 can be first obtained and processed by the speech recognition model to produce the corresponding text information. Subsequently, speech frames of length 12 can be obtained and processed by the speech recognition model to produce the corresponding text information. This process continues in this manner until all the speech information has been obtained.

[0071] For example, you can first acquire speech frame data from frames 1 to 12, perform speech recognition through the speech recognition model, and output the corresponding text information. Then, acquire speech frame data from frames 13 to 24, perform speech recognition through the speech recognition model again, and output the corresponding text information. And so on, until at least all the text information is output.

[0072] Due to the streaming nature of voice input and to improve voice recognition accuracy, any two adjacent voice frames of length 12 can contain the same voice frame data. For example, the first voice frame data may be frames 1 to 12, the second voice frame data may be frames 3 to 14, the third voice frame data may be frames 5 to 16, and so on.

[0073] Furthermore, as previously mentioned, if the user speaks for more than 9.6 seconds, the voice signal can be truncated. If the user speaks for 6 seconds, the corresponding text information can be directly output after processing the 6-second voice signal. The prior art uses an end-to-end processing approach, requiring input data to be 320 frames of voice data, which requires zero padding. Clearly, the present application's voice recognition method is more flexible.

[0074] Through the embodiments of the present application, the end-to-end convolutional neural network can be split into a streaming convolutional neural network based on the receptive field of the starting convolutional layer of the convolutional neural network corresponding to the input data of the long short-term memory network. The calculation of the end-to-end convolutional neural network is simulated by the streaming convolutional neural network, so that the voice frame data can be processed in a streaming manner. For example, while the user is speaking, only 12 frames of voice frame data are processed at a time. So, if the duration of the voice signal input by the user is 9.6 seconds, after the user finishes speaking, the delay in generating text information is only the delay of processing 12 frames of voice frame data, rather than the delay of processing 320 frames of voice frame data. Therefore, the delay can be reduced and the user has basically no perception, thereby improving the user experience.

[0075] Reference Figure 5 , shows another flow chart of the speech recognition method according to an embodiment of the present application, which may include the following steps:

[0076] Step S510, determining the receptive field size of the starting convolutional layer of the first network part according to the input data size of the second network part in the speech recognition model and the information of the convolutional layer of the first network part in the speech recognition model, and using the determined receptive field size as the preset length.

[0077] In an embodiment of the present application, the first network part in the speech recognition model can be a convolutional neural network part, and the second network part can be a long short-term memory network part. Alternatively, the second network part can also be a recurrent neural network, a bidirectional long short-term memory network, etc. Since the data output by the first network part can be directly processed by the second network part, that is, the first network part and the second network part are coupled, the receptive field size of the starting convolution layer of the first network part can be determined based on the input data size of the second network part and the information of the convolution layer of the first network part. Among them, the information of the convolution layer includes: the number of layers of the convolution layer and the size information of the convolution kernel corresponding to the convolution layer.

[0078] Of course, if the first network also includes a pooling layer, the receptive field size of the starting convolutional layer of the first network can be determined based on the input data size of the second network, the information of the convolutional layer of the first network, and the information of the pooling layer of the first network. The pooling layer information includes: the number of pooling layers and the size of the pooling kernel corresponding to the pooling layer.

[0079] In the embodiment of the present application, the speech frame data input into the speech recognition model can be two-dimensional data. For example, it can be two-dimensional data of W×h, where W represents the length of the speech frame data and h represents the width of each frame of speech frame data. The complete speech frame data can be data with W=320 and h=160, that is, it contains 320 frames of speech frame data, and each frame of speech frame data contains 160 digital characters.

[0080] Here with Figure 1 As an example, we will use one dimension (length) of the data as an example. It can be seen that the data input to the LSTM network is speech feature data of length 2. By reverse deduction, we can deduce that for 12 frames of speech frame data, after being processed by the convolutional neural network, speech feature data of length 2 can be obtained.

[0081] Specifically, since the speech feature data of length 2 is obtained after pooling processing in the pooling layer, the size of the pooling kernel is 3×3, and the stride is 1, it can be obtained that the data input to the convolution layer is speech feature data of length 4. Since the size of the convolution kernel of the four convolution processes is 3×3 and the stride is 1, the data input to the fourth convolution layer is speech feature data of length 6. Similarly, it can be obtained that the data input to the first convolution layer is 12 frames of speech frame data. In other words, the receptive field of the starting convolution layer of the first network part corresponding to the speech feature data of length 2 is 12.

[0082] Of course, if the network structure of the convolutional neural network is different, or the convolution kernel size and step size used in the convolution processing are different, or the pooling kernel size and step size used in the pooling processing are different, the receptive field of the starting convolution layer of the first network part corresponding to the speech feature data of length 2 will also be different.

[0083] Based on this, the convolutional neural network can be directly split according to the receptive field, and the receptive field size is used as the preset length, so that the convolutional neural network can directly process the speech frame data of the preset length. Figure 6 As shown, Figure 6 This is a schematic diagram of the speech recognition method in an embodiment of the present application. The present application can directly process speech frame data of a preset length, and further process the output speech feature data through a long short-term memory network, and decode it through a decoder to obtain corresponding text information.

[0084] Step S520: input the speech frame data of the preset length into the speech recognition model, recognize the speech frame data through the speech recognition model, and obtain text information corresponding to the speech frame data.

[0085] In the embodiment of the present application, after the preset length is determined, voice frame data of the preset length is continuously acquired for voice recognition while the user is speaking. Each time the voice frame data is acquired, the corresponding text information can be output.

[0086] In one implementation of the present application, before inputting the speech frame data of the preset length into the speech recognition model, the speech frame data input into the first network part may also be subjected to frame alignment processing. The details are as follows:

[0087] The first speech feature data of length 2 output by the convolutional neural network to the long short-term memory network can be calculated based on the first 7 frames of speech frame data. First, a zero frame is added before the 7 frames of speech frame data to obtain 8 frames of speech frame data. After the 8 frames of speech frame data are convolved with a convolution kernel size of 3×3 and a stride of 1, speech feature data of length 6 is obtained. Zeros are added before the 6-frame speech feature data to obtain speech feature data of length 7. After the convolution kernel size of 3×3 and a stride of 1 is further convolved, speech feature data of length 5 is obtained. This process is repeated in this way to obtain the first speech feature data of length 2.

[0088] Similarly, a second set of speech feature data with a length of 2 can be calculated based on the first 9 frames of speech frame data, and a third set of speech feature data with a length of 2 can be calculated based on the first 11 frames of speech frame data. In this process, because the input data is less than 12 frames of speech frame data, a special zero-padding operation can be performed. If the input data is greater than 12 frames of speech frame data, no special zero-padding operation is required.

[0089] In one implementation of the present application, the process of recognizing speech frame data by using a speech recognition model can be found in Figure 7 , which may include the following steps:

[0090] Step S710: The first network part of the speech recognition model receives and extracts features from the speech frame data to obtain speech feature data, which is then input into the second network part.

[0091] In an embodiment of the present application, as described above, two adjacent speech frame data of a preset length may contain the same speech frame data. If the speech frame data of the preset length is directly calculated each time, it will cause a waste of resources. Therefore, in one implementation of the present application, based on the predetermined repeated data positions in the speech frame data of two adjacent inputs to the first network part, in the process of extracting features of the currently input speech frame data through multiple convolutional layers in the first network part, the feature values ​​of the speech frame data at the repeated data positions output by each convolutional layer can be cached for use when extracting features of the speech frame data to be input next time.

[0092] For example, the first preset length of speech frame data is the 1st to 12th frame, and the second preset length of speech frame data is the 3rd to 14th frame, wherein the 3rd to 12th frame are repeated data. Figure 8As shown in the figure, speech frame data 1 and speech frame data 2 are input into the convolutional neural network in sequence. It can be seen that speech frame data 1 and speech frame data 2 contain the same data in each layer of the convolutional neural network (except the last layer). If the same data in each layer is cached, then the next speech frame data can directly use the previously cached data, which can save computational effort.

[0093] On this basis, for each convolutional layer of the first network part, the feature values ​​corresponding to the data at the repeated data position of the previous voice frame data in the current voice frame data are obtained from the cache, and feature extraction is performed on the data at other positions in the current voice frame data to obtain the corresponding feature values; based on the feature values ​​obtained from the cache and the feature values ​​obtained by feature extraction, the feature values ​​corresponding to the current voice frame data are obtained as the voice feature data output by the current convolutional layer.

[0094] by Figure 8 Taking the speech frame data 1 as an example, after feature extraction of frames 1 to 12, the 3rd to 10th speech features after the first convolution operation, the 3rd to 8th speech features after the second convolution operation, the 3rd to 6th speech features after the third convolution operation, and the 3rd to 4th speech features after the fourth convolution operation can be cached.

[0095] For speech frame data 2, when extracting features from frames 3 to 14, the first convolution operation can only process frames 11 to 14. The convolution results for the remaining data can be directly obtained from the cache. The second, third, and fourth convolution operations are similar. Figure 8 The curly brackets in the text represent the data that needs to be processed each time. As can be seen, this application can optimize the processing of 12 frames to 4 frames at a time, saving 67% of the computational effort. Correspondingly, the speech recognition latency is also reduced from 320 frames to 4 frames, a reduction of 1 / 80 of the original latency.

[0096] It should be noted that since the first three speech feature data inputs to the LSTM network are based on the first seven, nine, and eleven frames of data, respectively, and a special zero-padding operation is performed during the calculation process, the cached data during this process is not completely calculated based on the actual speech frame data and is therefore inaccurate. During the calculation process, the cached data is gradually calculated based on the actual speech frame data, and the previously calculated inaccurate data is also input into the LSTM network. Since the sixth data input to the LSTM network is calculated based on the correct cached data, the LSTM network can be reset, preventing it from accumulating erroneous data information, thereby improving the accuracy of speech recognition.

[0097] Step S720: Perform speech recognition processing on the speech feature data through the second network part to obtain a speech recognition intermediate result.

[0098] In the embodiments of the present application, the second network component can be a long short-term memory (LSTM) network. LSTM is a special recursive neural network that modifies a recurrent neural network and creates three gates: a "forget gate," an "update gate," and an "output gate." The sequential nature of LSTM makes it ideal for processing data containing temporal and sequencing information. LSTM has also become a crucial module for deep learning in natural language processing.

[0099] The speech recognition intermediate result output by the second network part can be a vector, which represents the proportion of each phoneme, and the sum of the proportions of each phoneme is 1.

[0100] Step S730: decode the speech recognition intermediate result to obtain text information corresponding to the speech frame data.

[0101] Specifically, based on the intermediate results of speech recognition, we can search for word sequences that meet the requirements in the search space composed of four knowledge sources, including acoustic model, acoustic context, pronunciation dictionary and language model, through algorithms such as Viterbi search, and thus obtain the corresponding text information.

[0102] The speech recognition method provided by the embodiment of the present application can correspond to the receptive field size of the starting convolution layer of the first network part according to the input data size of the second network part in the speech recognition model, split the end-to-end convolutional neural network according to the receptive field size, and simulate the end-to-end convolutional neural network through a streaming convolutional neural network. The receptive field size is used as a preset length, so that speech frame data of a preset length can be processed. Since the receptive field size is usually much smaller than the length of the original complete speech frame data, the speech frame data can be gradually recognized during the user's speaking process to obtain the corresponding text information, rather than waiting for the user to finish speaking a sentence before recognizing the corresponding speech information. It can be seen that the present application can perform speech recognition in a more real-time and efficient manner and reduce delays. In addition, in the process of calculation through the convolutional neural network, the present application can reduce the amount of calculation when actually calculating each speech frame data of a preset length by caching the feature values ​​corresponding to the repeated data. On the basis of reducing the amount of calculation, the delay can be further reduced.

[0103] The speech recognition method provided in this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: terminals, mobile terminals, PCs, servers, in-vehicle devices, entertainment devices, advertising devices, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, glasses, watches, wearable devices, virtual display devices, or display enhancement devices. In practical applications, it can be applied to devices with limited storage space and computing power, such as edge devices.

[0104] Reference Figure 9 , shows another flow chart of the speech recognition method in an embodiment of the present application, which may include the following steps:

[0105] Step S902: Receive the voice command input by the user and convert it into voice frame data, and stream the voice frame data into the voice recognition model according to a preset length, wherein the preset length is determined according to the receptive field size of the starting convolutional layer of the first network part in the voice recognition model corresponding to the input data size of the second network part in the voice recognition model.

[0106] In an embodiment of the present application, the intelligent electronic device can also perform intelligent question and answering using the above-mentioned voice recognition method. Specifically, the user can send a voice command to the intelligent electronic device. The voice command can be a question that the user wants to query, for example, if the user wants to query tomorrow's weather, he can send a voice command such as "What's the weather like tomorrow?" The intelligent electronic device can convert the voice command into voice frame data. The length of each frame of voice frame data can be fixed (for example, 20 milliseconds or 30 millimeters, etc.), thereby obtaining multiple frames of voice frame data.

[0107] Step S904: Recognize the speech frame data through a speech recognition model to obtain text information corresponding to the speech frame data.

[0108] After getting the voice frame data, you can follow Figure 2 The method of the embodiment performs speech recognition to obtain text information corresponding to the speech frame data. For example, for a speech instruction "How is the weather tomorrow?", the recognized text information may be "How is the weather tomorrow?"

[0109] Step S906: Determine reply information corresponding to the voice command based on the text information, convert the reply information into voice data, and output the voice data.

[0110] In an embodiment of the present application, after the text information corresponding to the voice command is recognized, the reply information corresponding to the text information can be obtained from a pre-stored database, or the reply information corresponding to the text information can be determined based on a pre-trained neural network model, or the reply information corresponding to the text information can be obtained by searching relevant knowledge, etc. This application does not limit this.

[0111] After receiving the reply information, the reply information may be converted into voice data, and the voice data may be output to reply to the voice command input by the user.

[0112] In the speech recognition method of the embodiment of the present application, after receiving a voice command, the intelligent electronic device can convert the voice command into voice frame data, and determine the length of the voice frame data to be processed each time based on the input data size of the second network part in the speech recognition model corresponding to the receptive field size of the starting convolution layer of the first network part. On the one hand, the input data size of the second network part is usually the output data size of the first network part. Based on this, the receptive field size of the starting convolution layer of the first network part is determined based on the input data size, and the length of the input voice frame data is determined accordingly, so that the processing output of the voice frame data of this length by the first network part can meet the input requirements of the second network part. On the other hand, since the receptive field size is usually much smaller than the length of the original complete voice frame data, the length of the voice frame data currently being processed is much smaller than the length of the original complete voice frame data, which greatly reduces the amount of data processed by the speech recognition model each time, improves the data processing speed, and reduces the data processing delay. Therefore, during the voice interaction process with the user, the user's voice data can be recognized faster and the voice message can be replied to, thereby improving the user's voice interaction experience.

[0113] Reference Figure 10a , shows a structural diagram of a speech recognition device in an embodiment of the present application, including:

[0114] A speech frame data input module 1001 is configured to input speech frame data of a preset length into a speech recognition model, wherein the preset length is determined based on the receptive field size of the starting convolutional layer of the first network part of the speech recognition model corresponding to the input data size of the second network part of the speech recognition model;

[0115] The speech frame processing module 1003 is configured to recognize the speech frame data through a speech recognition model to obtain text information corresponding to the speech frame data.

[0116] Optionally, the voice frame processing module 1003 includes:

[0117] A feature extraction unit, configured to receive and extract features from speech frame data through the first network portion of the speech recognition model, obtain speech feature data, and input the data into the second network portion;

[0118] a speech recognition unit, configured to perform speech recognition processing on the speech feature data via the second network portion to obtain an intermediate speech recognition result;

[0119] The decoding unit is used to decode the intermediate result of speech recognition to obtain text information corresponding to the speech frame data.

[0120] Optionally, the first network part is a convolutional neural network part, and the second network part is a long short-term memory network part.

[0121] Optionally, the speech recognition device of the embodiment of the present application further includes:

[0122] The length determination module is used to determine the receptive field size of the starting convolution layer of the first network part according to the input data size of the second network part and the information of the convolution layer of the first network part, and use the determined receptive field size as the preset length.

[0123] Optionally, the information of the convolution layer includes: information about the number of convolution layers and size information of a convolution kernel corresponding to the convolution layer.

[0124] Optionally, the feature extraction unit is specifically used to cache the feature values ​​of the speech frame data at the repeated data position output by each convolution layer during the process of feature extraction of the currently input speech frame data through multiple convolution layers in the first network part based on the predetermined repeated data positions in the speech frame data input twice adjacently, so as to be used when performing feature extraction on the speech frame data input next time.

[0125] Optionally, the feature extraction unit is specifically used to obtain, for each convolutional layer of the first network part, feature values ​​corresponding to data in the current speech frame data that is at a repeated data position with the previous speech frame data from the cache, and perform feature extraction on data at other positions in the current speech frame data to obtain corresponding feature values; based on the feature values ​​obtained from the cache and the feature values ​​obtained by feature extraction, obtain the feature values ​​corresponding to the current speech frame data as the speech feature data output by the current convolutional layer.

[0126] Optionally, the speech recognition device of the embodiment of the present application further includes:

[0127] The alignment module is used to perform frame alignment processing on the voice frame data input into the first network part.

[0128] The speech recognition device provided in this embodiment is used to implement the corresponding speech recognition methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0129] The present application also provides a smart speaker. Figure 10b , the smart speaker includes:

[0130] Microphone 1002, used to obtain voice commands input by the user and convert them into voice frame data;

[0131] Processor 1004 is configured to input speech frame data of a preset length into a speech recognition model, recognize the speech frame data through the speech recognition model, and obtain text information corresponding to the speech frame data. Furthermore, based on the text information, a response message corresponding to the speech command is determined. The preset length is determined based on the receptive field size of the starting convolutional layer of the first network portion of the speech recognition model relative to the input data size of the second network portion of the speech recognition model.

[0132] The speaker 1006 is used to convert the reply information into voice data and output the voice data.

[0133] In the embodiment of the present application, the smart speaker is a tool that can interact with the user through voice. The user can use the smart speaker to perform various common functions, such as ordering songs, understanding weather forecasts, setting alarms, and checking information. It can also control smart home devices, such as opening curtains, setting the temperature of water heaters, etc.

[0134] In practice, smart speakers generally don't constantly monitor user conversations. Therefore, a trigger mechanism can be configured for the smart speaker. Once the trigger mechanism is met, the smart speaker begins analyzing user needs. Otherwise, the smart speaker remains dormant. The wake-up function acts as a trigger. When the user attempts to activate the smart speaker, they can simply speak the wake-up keyword. Upon detecting the user's voice signal, the dormant smart speaker quickly switches to active mode. The wake-up keyword can be set based on actual needs or by the user.

[0135] In existing technology, smart speakers can collect user voice data through microphones, analyze the user's needs based on the voice data, and output corresponding response information. However, when analyzing the user's needs based on the voice data, voice recognition is only performed after the user enters a complete sentence, resulting in a long delay.

[0136] In the smart speaker of the embodiment of the present application, when performing speech recognition, since the input data size of the second network part is usually the output data size of the first network part, the receptive field size of the starting convolution layer of the first network part is determined based on the input data size, and the length of the input speech frame data is determined accordingly, so that the processing output of the speech frame data of this length by the first network part can meet the input requirements of the second network part. On the other hand, since the receptive field size is usually much smaller than the length of the original complete speech frame data, the length of the currently processed speech frame data is much smaller than the length of the original complete speech frame data, which greatly reduces the amount of data processed by the speech recognition model each time, improves the data processing speed, and reduces the data processing delay. Therefore, in the process of voice interaction with the user, the user's voice data can be recognized more quickly, improving the user's voice interaction experience.

[0137] Optionally, the processor 1004 is specifically used to receive and perform feature extraction on the speech frame data through the first network part of the speech recognition model, obtain speech feature data, and input it into the second network part; perform speech recognition processing on the speech feature data through the second network part to obtain a speech recognition intermediate result; decode the speech recognition intermediate result to obtain text information corresponding to the speech frame data.

[0138] Optionally, the first network part is a convolutional neural network part, and the second network part is a long short-term memory network part.

[0139] Optionally, the processor 1004 is further used to determine the receptive field size of the starting convolutional layer of the first network part based on the input data size of the second network part and the information of the convolutional layer of the first network part, and use the determined receptive field size as the preset length.

[0140] Optionally, the information of the convolution layer includes: information about the number of convolution layers and size information of a convolution kernel corresponding to the convolution layer.

[0141] Optionally, the processor 1004 is specifically used to cache the feature values ​​of the speech frame data at the repeated data position output by each convolution layer during the process of feature extraction of the currently input speech frame data through multiple convolution layers in the first network part based on the predetermined repeated data positions in the speech frame data of two adjacent inputs to the first network part, so as to be used when performing feature extraction on the speech frame data input next time.

[0142] Optionally, the processor 1004 is specifically used to obtain, for each convolution layer of the first network part, feature values ​​corresponding to data in the current speech frame data that is at a repeated data position with the previous speech frame data from the cache, and perform feature extraction on data at other positions in the current speech frame data to obtain corresponding feature values; based on the feature values ​​obtained from the cache and the feature values ​​obtained by feature extraction, obtain the feature values ​​corresponding to the current speech frame data as the speech feature data output by the current convolution layer.

[0143] Optionally, the processor 1004 is further configured to perform frame alignment processing on the voice frame data input into the first network part.

[0144] See also Figure 10c , Figure 10c This is a schematic diagram of the structure of the smart home appliance in the embodiment of the present application, including:

[0145] Microphone 1010, used to obtain voice commands input by the user and convert them into voice frame data;

[0146] Processor 1020, configured to input speech frame data of a preset length into a speech recognition model, recognize the speech frame data through the speech recognition model, and obtain text information corresponding to the speech frame data; wherein the preset length is determined based on the receptive field size of the starting convolutional layer of the first network portion of the speech recognition model corresponding to the input data size of the second network portion of the speech recognition model;

[0147] The processor 1020 is further configured to control the smart home appliance to execute the operation indicated by the voice command based on the text information.

[0148] In the embodiment of the present application, users can perform voice control on smart home appliances, thereby improving the user's operating experience. Specifically, similar to the aforementioned smart speakers, smart home appliances can also recognize the voice commands input by the user and obtain corresponding text information. Afterwards, the smart home appliances are controlled to perform corresponding operations according to the text information. Among them, the operations performed by the smart home appliances include: operations performed by the smart home appliances themselves, operations of controlling other smart home appliances that establish communication connections with the smart home appliances, etc. The smart home appliances of the application embodiment may include: toasters, ovens, dishwashers, microwave ovens, refrigerators, air conditioners, washing machines, televisions, and door and window controllers, etc.

[0149] For example, if a user wants to open the refrigerator, they can enter the voice command "open" to the refrigerator. The refrigerator can then perform voice recognition on the voice command, obtain the corresponding text message "open", and perform the corresponding operation, that is, open the refrigerator. Alternatively, if the refrigerator is connected to a microwave oven, it can also open the microwave oven.

[0150] The smart home appliances of the embodiments of the present application can more quickly recognize the text information corresponding to the voice command and perform corresponding operations based on the text information, thereby improving the user's operating experience.

[0151] Optionally, the processor 1020 is specifically used to receive and perform feature extraction on the speech frame data through the first network part of the speech recognition model, obtain speech feature data, and input it into the second network part; perform speech recognition processing on the speech feature data through the second network part to obtain a speech recognition intermediate result; decode the speech recognition intermediate result to obtain text information corresponding to the speech frame data.

[0152] Optionally, the first network part is a convolutional neural network part, and the second network part is a long short-term memory network part.

[0153] Optionally, the processor 1020 is further used to determine the receptive field size of the starting convolutional layer of the first network part based on the input data size of the second network part and the information of the convolutional layer of the first network part, and use the determined receptive field size as the preset length.

[0154] Optionally, the information of the convolution layer includes: information about the number of convolution layers and size information of a convolution kernel corresponding to the convolution layer.

[0155] Optionally, the processor 1020 is specifically used to cache the feature values ​​of the speech frame data at the repeated data position output by each convolution layer during the process of feature extraction of the currently input speech frame data through multiple convolution layers in the first network part based on the predetermined repeated data positions in the speech frame data input twice adjacently, so as to be used when performing feature extraction on the speech frame data input next time.

[0156] Optionally, the processor 1020 is specifically used to obtain, for each convolution layer of the first network part, feature values ​​corresponding to data in the current speech frame data that is at a repeated data position with respect to the previous speech frame data, from the cache, and perform feature extraction on data at other positions in the current speech frame data to obtain corresponding feature values; based on the feature values ​​obtained from the cache and the feature values ​​obtained by feature extraction, obtain the feature values ​​corresponding to the current speech frame data as speech feature data output by the current convolution layer.

[0157] Optionally, the processor 1020 is further configured to perform frame alignment processing on the voice frame data input into the first network part.

[0158] In the embodiment of the present application, an electronic device is also provided. Figure 11 , Figure 11 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application, which may include:

[0159] One or more processors 1101;

[0160] Computer readable storage medium 1102 may be configured to store one or more programs,

[0161] When one or more programs are executed by one or more processors 1101 , the one or more processors 1101 implement the speech recognition method as described in the above embodiment.

[0162] Figure 12 The hardware structure of the electronic device in the embodiment of the present application; Figure 12 As shown, the hardware structure of the electronic device may include: a processor 1201, a communication interface 1202, a memory 1203 and a communication bus 1204;

[0163] The processor 1201, the communication interface 1202, and the memory 1203 communicate with each other via the communication bus 1204;

[0164] Optionally, the communication interface 1202 may be an interface of a communication module, such as an interface of a GSM module;

[0165] The processor 1201 may be specifically configured to execute the speech recognition method as described in the above embodiment.

[0166] The processor 1201 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0167] The memory 1203 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0168] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access storage medium, a read-only storage medium, an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program configured for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0169] Computer program code configured to perform the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0170] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions configured to implement the specified logical functions. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When implementing the specific embodiments, these steps may be fewer, more, or the execution order may be adjusted. That is, in some alternative implementations, the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0171] The modules involved in the embodiments described in this application can be implemented by software or hardware. The modules described can also be set in a processor. For example, it can be described as follows: a processor includes a voice frame data input module and a voice frame processing module.

[0172] As another aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the speech recognition method described in the first or second embodiment above. The computer-readable storage medium may be included in the apparatus described in the above embodiment, or may exist independently, and this application does not limit this.

[0173] As used in various embodiments of the present application, the terms "first," "second," "the first," or "the second" may modify various components regardless of order and / or importance, but these terms do not limit the corresponding components. The above terms are merely configured to distinguish an element from other elements. For example, a first user device and a second user device represent different user devices, even though both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present application.

[0174] When one element (for example, a first element) is referred to as being “(operably or communicably) coupled” or “(operably or communicably) coupled to” or “connected to” another element (for example, a second element), it should be understood that the one element is directly connected to the other element or that the one element is indirectly connected to the other element via yet another element (for example, a third element). Conversely, it should be understood that when an element (for example, a first element) is referred to as being “directly connected” or “directly coupled” to another element (the second element), there is no element (for example, a third element) interposed therebetween.

[0175] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A speech recognition method, comprising: Inputting speech frame data of a preset length into a speech recognition model, wherein the speech recognition model includes at least a first network part and a second network part, the preset length is determined based on a receptive field size of a starting convolutional layer of the first network part of the speech recognition model corresponding to an input data size of the second network part of the speech recognition model, the first network part is a convolutional neural network part, and the second network part is a long short-term memory network part; The speech frame data is recognized by the speech recognition model to obtain text information corresponding to the speech frame data.

2. The method according to claim 1, wherein The step of recognizing the speech frame data by using the speech recognition model to obtain text information corresponding to the speech frame data includes: Receive and perform feature extraction on the speech frame data through the first network part of the speech recognition model to obtain speech feature data, and input the data into the second network part; performing speech recognition processing on the speech feature data via the second network portion to obtain a speech recognition intermediate result; The speech recognition intermediate result is decoded to obtain text information corresponding to the speech frame data.

3. The method according to claim 1 or 2, wherein: The first network part is a convolutional neural network part, and the second network part is a long short-term memory network part.

4. The method according to claim 3, wherein: Before inputting the speech frame data of the preset length into the speech recognition model, the method further includes: According to the input data size of the second network part and the information of the convolution layer of the first network part, the receptive field size of the starting convolution layer of the first network part is determined, and the determined receptive field size is used as the preset length.

5. The method according to claim 4, wherein The information of the convolution layer includes: the number of layers of the convolution layer and the size information of the convolution kernel corresponding to the convolution layer.

6. The method according to claim 2, wherein: The receiving and performing feature extraction on the speech frame data through the first network part of the speech recognition model to obtain speech feature data includes: According to the predetermined repeated data positions in the speech frame data input into the first network part twice adjacently, in the process of the first network part performing feature extraction on the currently input speech frame data through multiple convolutional layers, the feature values ​​of the speech frame data at the repeated data positions output by each convolutional layer are cached for use when performing feature extraction on the speech frame data input next time.

7. The method according to claim 6, wherein: The receiving and performing feature extraction on the speech frame data through the first network part of the speech recognition model to obtain speech feature data includes: For each convolutional layer of the first network part, obtaining, from the cache, feature values ​​corresponding to data at the repeated data position of the previous voice frame data in the current voice frame data, and performing feature extraction on data at other positions in the current voice frame data to obtain corresponding feature values; According to the feature values ​​obtained from the cache and the feature values ​​obtained by feature extraction, the feature values ​​corresponding to the current speech frame data are obtained as the speech feature data output by the current convolutional layer.

8. The method according to claim 1, wherein Before inputting the speech frame data of the preset length into the speech recognition model, the method further includes: Perform frame alignment processing on the voice frame data input into the first network part.

9. A speech recognition method, comprising: Receive a voice command input by a user and convert it into voice frame data, and stream the voice frame data into a voice recognition model according to a preset length, wherein the voice recognition model includes at least a first network part and a second network part, and the preset length is determined according to the receptive field size of the starting convolution layer of the first network part in the voice recognition model corresponding to the input data size of the second network part in the voice recognition model, the first network part is a convolutional neural network part, and the second network part is a long short-term memory network part; Recognize the speech frame data using the speech recognition model to obtain text information corresponding to the speech frame data; According to the text information, reply information corresponding to the voice instruction is determined, and the reply information is converted into voice data and outputted by voice.

10. A speech processing device, comprising: a speech frame data input module, configured to input speech frame data of a preset length into a speech recognition model, wherein the speech recognition model comprises at least a first network portion and a second network portion, wherein the preset length is determined based on a receptive field size of a starting convolutional layer of the first network portion of the speech recognition model corresponding to an input data size of the second network portion of the speech recognition model, wherein the first network portion is a convolutional neural network portion, and the second network portion is a long short-term memory network portion; The speech frame processing module is used to recognize the speech frame data through the speech recognition model to obtain text information corresponding to the speech frame data.

11. A smart speaker, comprising: A microphone is used to obtain user input voice commands and convert them into voice frame data; A processor, configured to input the speech frame data into the speech recognition model according to a preset length, recognize the speech frame data through the speech recognition model, and obtain text information corresponding to the speech frame data; wherein the speech recognition model includes at least a first network part and a second network part, and the preset length is determined based on the receptive field size of the starting convolution layer of the first network part of the speech recognition model corresponding to the input data size of the second network part of the speech recognition model, the first network part is a convolutional neural network part, and the second network part is a long short-term memory network part; and, based on the text information, determine reply information corresponding to the speech command; The speaker is used to convert the reply information into voice data and output the voice data.

12. A smart home appliance comprising: A microphone is used to obtain user input voice commands and convert them into voice frame data; A processor, configured to input speech frame data of a preset length into the speech recognition model, recognize the speech frame data through the speech recognition model, and obtain text information corresponding to the speech frame data; wherein the speech recognition model includes at least a first network part and a second network part, the preset length is determined based on the receptive field size of the starting convolutional layer of the first network part of the speech recognition model corresponding to the input data size of the second network part of the speech recognition model, the first network part is a convolutional neural network part, and the second network part is a long short-term memory network part; The processor is further configured to control the smart home appliance to execute the operation indicated by the voice instruction according to the text information.

13. An electronic device comprising: one or more processors; A computer-readable storage medium configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the speech recognition method according to any one of claims 1 to 8; or, the one or more processors implement the speech recognition method according to claim 9.

14. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the program implements the speech recognition method according to any one of claims 1 to 8; or, when the program is executed by a processor, the program implements the speech recognition method according to claim 9.

Images