An acceleration device and computing device for deep learning

Abstract

The present application discloses an acceleration device for deep learning and a computing device for deep learning. The acceleration device for deep learning includes: a data buffer, a computing module connected to the data buffer; the data buffer is used to cache the intermediate computing results of the network layer of the vocoder neural network; the computing module is used to obtain the intermediate computing results of the first network layer of the vocoder neural network from the data buffer according to the received control instructions, and send the intermediate computing results of the first network layer to the data buffer. The acceleration device for deep learning provided by the present application improves the processing efficiency of the vocoder neural network.

Description

Technical Field

[0001] The present application relates to the field of circuit design, and specifically to an acceleration device and a computing device for deep learning. Background Art

[0002] In the field of artificial intelligence applications, voice applications account for a considerable proportion. Text to speech (TTS) technology, as a voice application, is often used in human-computer interaction. Generally, the TTS algorithm can be further divided into three parts: the front end for preprocessing text into linguistic features such as morphemes, the back end for transforming linguistic features into acoustic features such as spectra and phonemes, and the vocoder that uses acoustic features to generate the final speech. Among these three parts, since the output frequency of the vocoder is equal to the sampling frequency of the audio signal, that is, 16 to 22kHz, the vocoder has a large amount of calculation and memory access, which has always been the bottleneck restricting the calculation speed in the TTS calculation process.

[0003] The current TTS algorithms that generate the best speech quality are all based on neural network models. Compared with the existing technical solutions based on the traditional Hidden Markov Model (HMM), the speech synthesis effect based on the neural network method is closer to the actual human speech effect. WaveNet is the vocoder with the best speech generation quality at present and is widely used in a variety of TTS algorithms based on neural networks.

[0004] Wavenet is completely based on iteration, that is, the operation of generating the next frame of speech needs to wait for the output result of the previous frame of speech before it can start. The frequency range of human speech is about 20kHz, which means that generating 1 second of speech requires the entire neural network to be iterated tens of thousands of times. Common hardware acceleration structures are based on single instruction multiple data (SIMD), so the acceleration effect on this iterative network is very limited. It takes dozens to hundreds of seconds to generate 1 second of speech, which is completely unable to meet the actual needs of real-time speech generation products.

[0005] How to complete all operators of vocoder neural networks such as Wavenet with a simple hardware structure, avoid tens of thousands of off-chip interactions of vocoder neural networks per second, reduce computing delays, and thus improve the processing efficiency of vocoder neural networks has become an urgent problem to be solved. Summary of the invention

[0006] The present application provides an acceleration device for deep learning to improve the processing efficiency of a vocoder neural network.

[0007] The acceleration device includes: a data buffer, and a computing module connected to the data buffer;

[0008] The data buffer is used to cache the intermediate calculation results of the network layer of the vocoder neural network;

[0009] The operation module is used to obtain the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction, and send the intermediate calculation result of the first network layer to the data buffer.

[0010] Optionally, the acceleration device applied to the vocoder neural network further includes a controller;

[0011] The controller is used to send the control instruction to the computing unit.

[0012] Optionally, obtaining the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction, and sending the intermediate calculation result of the first network layer to the data buffer, comprises: obtaining the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction;

[0013] According to the intermediate calculation result of the first network layer, obtaining the intermediate calculation result of the second network layer cascaded with the first network layer;

[0014] The intermediate calculation result of the second network layer is sent to the data buffer.

[0015] Optionally, the operation module includes: a matrix operation unit, an addition unit connected to the matrix operation unit, an activation unit connected to the addition unit, a multiplication unit connected to the activation unit, and a deviation unit connected to the multiplication unit;

[0016] The matrix operation unit is used to perform a multiplication-accumulation operation on the intermediate calculation result of the first network layer to obtain multiplication-accumulation data;

[0017] The adding unit is used to perform an addition operation on the multiplication and accumulation data to obtain addition sum data;

[0018] The activation unit is used to perform an activation operation on the addition data to obtain first activation data:

[0019] The multiplication operation unit is used to perform a multiplication operation on the first activation data to obtain product data;

[0020] The deviation unit is used to perform a deviation operation on the product data to obtain an intermediate calculation result of the second network layer.

[0021] Optionally, the operation module further includes: a modified linear activation unit connected to the addition unit;

[0022] The modified linear activation unit is used to perform a modified linear activation operation on the addition data to obtain third activation data;

[0023] The third activation data is used as intermediate data of a tail network layer of the vocoder neural network.

[0024] Optionally, the operation module further includes: a normalization and random selection unit connected to the matrix operation unit;

[0025] The normalization and random selection unit is used to perform normalization processing on the third activation data to obtain classification data; and obtain the output audio data of the vocoder neural network according to the classification data with the highest probability in the classification data.

[0026] Optionally, it is characterized in that the normalization and random selection unit is also used for:

[0027] Performing a random selection operation in the classified data to obtain random data;

[0028] The random data is used as input data of the vocoder neural network.

[0029] Optionally, performing a logistic regression activation operation on the addition data to obtain first activation data includes:

[0030] obtaining the added sum data;

[0031] According to the addition and data, a search is performed in a preset activation function value lookup table to obtain first activation data.

[0032] Optionally, the acceleration device applied to the vocoder neural network further includes: a jump unit;

[0033] The jumper unit is used for pipelined execution of data between the matrix operation unit and the first network layer.

[0034] Optionally, the acceleration device for deep learning is implemented using a field programmable gate array or an application-specific integrated circuit.

[0035] The present application provides a computing device for deep learning, comprising: a matrix operation unit, an addition unit connected to the matrix operation unit, an activation unit connected to the addition unit, a multiplication unit connected to the activation unit, and a deviation unit connected to the multiplication unit;

[0036] The matrix operation unit is used to perform a multiplication-accumulation operation on the intermediate calculation result of the first network layer to obtain multiplication-accumulation data;

[0037] The adding unit is used to perform an addition operation on the multiplication and accumulation data to obtain addition sum data;

[0038] The logistic regression activation unit is used to perform an activation operation on the addition data to obtain first activation data;

[0039] The multiplication operation unit is used to perform a multiplication operation on the first activation data to obtain product data;

[0040] The deviation unit is used to perform a deviation operation on the product data to obtain an intermediate calculation result of the second network layer.

[0041] Optionally, the operation module further includes: a modified linear activation unit connected to the addition unit;

[0042] The modified linear activation unit is used to perform a modified linear activation operation on the addition data to obtain third activation data;

[0043] The third activation data is used as intermediate data of a tail network layer of the vocoder neural network.

[0044] Optionally, the operation module further includes: a normalization and random selection unit connected to the matrix operation unit;

[0045] The normalization and random selection unit is used to perform normalization processing on the third activation data to obtain classification data; and obtain the speech data output by the vocoder neural network based on the classification data with the highest probability in the classification data.

[0046] Optionally, the normalization and random selection unit is further used for:

[0047] Performing a random selection operation in the classified data to obtain random data;

[0048] The random data is used as input data of the vocoder neural network.

[0049] Optionally, performing a logistic regression activation operation on the addition data to obtain first activation data includes:

[0050] obtaining the added sum data;

[0051] According to the addition and data, a search is performed in a preset activation function value lookup table to obtain first activation data.

[0052] Optionally, the computing device for deep learning is implemented using a field programmable gate array or an application-specific integrated circuit.

[0053] The present application provides an electronic device, which includes the above-mentioned acceleration device for deep learning or the above-mentioned computing device for deep learning.

[0054] The present application provides a speaker, which includes the above-mentioned acceleration device for deep learning or the above-mentioned computing device for deep learning.

[0055] The present application provides a text information processing method, comprising:

[0056] Get the text information to be processed;

[0057] Performing sequence conversion on the text information to obtain spectrum data;

[0058] According to the spectrum data, the voice data corresponding to the text information is obtained using the computing device for deep learning.

[0059] Compared with the prior art, this application has the following advantages:

[0060] The acceleration device for deep learning provided in the present application includes a data buffer and a computing module connected to the data buffer; the data buffer is used to cache the intermediate calculation results of the network layer of the vocoder neural network; the computing module is used to obtain the intermediate calculation results of the first network layer of the vocoder neural network from the data buffer according to the received control instructions, and send the intermediate calculation results of the first network layer of the vocoder neural network to the data buffer.

[0061] By using the acceleration device for deep learning provided in the present application, according to the received control instructions, through the data buffer, the vocoder neural network does not need off-chip interaction during the iterative calculation process of tens of thousands of times per second, thereby reducing the calculation delay and improving the data processing efficiency of the vocoder neural network.

[0062] The computing device for deep learning provided in the present application includes: a matrix operation unit, an addition unit connected to the matrix operation unit, an activation unit connected to the addition unit, a multiplication unit connected to the activation unit, and a deviation unit connected to the multiplication unit; the matrix operation unit is used to perform a multiplication-accumulation operation on the intermediate calculation result of the first network layer to obtain multiplication-accumulation data; the addition unit is used to perform an addition operation on the multiplication-accumulation data to obtain addition data; the activation unit is used to perform an activation operation on the addition data to obtain first activation data; the multiplication operation unit is used to perform a multiplication operation on the first activation data to obtain product data; the deviation unit is used to perform a deviation operation on the product data to obtain the intermediate calculation result of the second network layer.

[0063] By using the computing device for deep learning provided in the present application, various operations including matrix operations in the vocoder neural network are implemented in a simplified manner, reducing the data cache cost and instruction switching cost between operator switching, thereby improving the computing efficiency of the vocoder neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of an embodiment of an application scenario of an acceleration device for deep learning provided by the present application;

[0065] Figure 2 is a schematic diagram of the structure of an acceleration device for deep learning provided in the first embodiment of the present application;

[0066] Figure 3a is a schematic diagram of an application system involved in the first embodiment of the present application;

[0067] Figure 3b is a flowchart of a vocoder neural network according to the first embodiment of the present application;

[0068] Figure 3c is a flowchart of the residual network layer of the vocoder neural network involved in the first embodiment of the present application;

[0069] Figure 4 This is a typical structural diagram of a convolutional neural network according to the first embodiment of the present application.

[0070] Figure 5 is a schematic diagram of the structure of the computing module involved in the first embodiment of the present application;

[0071] Figure 6 This is a flowchart of a text information processing method provided in the fourth embodiment of the present application. DETAILED DESCRIPTION

[0072] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0073] In order to enable those skilled in the art to better understand the present application, a specific application scenario embodiment of the present application is first described in detail. Figure 1 As shown, it is a schematic diagram of an embodiment of an application scenario of an acceleration device for deep learning provided by the present application. Figure 1 The server in can be deployed in the cloud. The server adopts the acceleration device for deep learning provided by the present application. In the specific implementation process, the client can send a request message for processing text data to the server; after receiving the request message, the server first converts the text data into spectrum data; then the spectrum data is input into the acceleration device for deep learning to obtain the voice data corresponding to the text data. Among them, the acceleration device for deep learning includes: a data buffer, and a calculation module connected to the data buffer; the data buffer is used to cache the intermediate calculation results of the network layer of the vocoder neural network; the calculation module is used to obtain the intermediate calculation results of the first network layer of the vocoder neural network from the data buffer according to the received control instruction, and send the intermediate calculation results of the first network layer to the data buffer. Finally, the server returns the processed voice data to the client.

[0074] The first embodiment of the present application provides an acceleration device for deep learning. Figure 2 , which is a schematic diagram of the structure of an acceleration device for deep learning provided by the first embodiment of the present application. Figure 2 The first embodiment of the present application provides an acceleration device for deep learning and is described in detail.

[0075] In this embodiment, the vocoder refers to a calculation method, module or device for converting spectrum data into speech data. The vocoder neural network model in this embodiment can be WaveNet. In order to enable those skilled in the art to more quickly understand the WaveNet vocoder neural network, the following is combined with Figure 3a-3c The working process of the vocoder neural network in the prior art is explained.

[0076] Figure 3a A schematic diagram of an application system for converting text data into audio data using a vocoder neural network is provided. Figure 3aIt can be seen that first, the application system receives text data as input data; then, the text data is converted into a sequence to obtain the corresponding spectrogram data; further, the spectrogram data is used as the input of WaveNet, and after being processed by WaveNet, audio data is finally obtained.

[0077] Figure 3b A flowchart of the vocoder neural network is provided. Figure 3b It can be seen that the vocoder neural network constitutes an iterative loop structure. In each iteration, the vocoder neural network outputs a frame of speech signal and returns it to the input port of the vocoder neural network as the calculation input for generating the next frame of speech. This iterative loop structure is exactly why the neural network cannot be parallelized on a large scale. The workflow of the vocoder neural network is as follows: First, the current input and the historical input are integrated into a unified tensor and transmitted to multiple cascaded residual network layers (residualblock). The workflow of a residual network layer is as follows: Figure 3c As shown in the figure. The output of one residual network layer will be used as the input of the next residual layer. In addition, the intermediate results will be directly output to the outside of the residual network layer for skip connection. After several layers of transformation, the results of all skip connections will generate speech output, which will be used as the input of the next iteration of the vocoder neural network after several layers of transformation.

[0078] Although the existing technology has provided many neural network acceleration solutions based on field programmable gate arrays (FPGA) and application specific integrated circuits (ASIC), including hardware structure design and performance optimization methods, these solutions have verified the huge application prospects of hardware computing. However, the hardware design of vocoder neural networks is still in a blank stage, and there are no relevant comparative solutions. The following analyzes two types of hardware accelerators for the overall artificial intelligence (AI) field:

[0079] The first category is accelerators for specific algorithms, such as accelerators for convolutional neural networks and accelerators for long short-term memory networks (LSTM). This type of accelerator takes into account the characteristics of specific algorithms and is very efficient when executing algorithms that match them. However, the vocoder neural network itself does not have the above algorithm characteristics and cannot be accelerated using this type of accelerator.

[0080] The second category is a general accelerator for all neural network operations, represented by the tensor processing unit (TPU). Its implementation method is to define a new instruction structure that can cover all calculations of most neural networks. When executing the calculation, the neural network algorithm is compiled into hardware instructions, and the hardware calculation of the neural network is completed by continuously issuing instructions. Since these instruction sets do have the completeness in the field of neural network calculation, this type of solution can be used to perform the calculation of the vocoder neural network. However, its acceleration effect is not good, which is mainly due to two reasons: First, this type of method is similar to the graphics processing unit (GPU), which is based on deploying a large number of parallel computing units on the chip to achieve acceleration. For the iterative structure of small-scale operators such as the vocoder neural network, due to the low parallel capacity of the neural network, large-scale parallel acceleration cannot be achieved. Second, in the speech generation application, the vocoder network needs to be repeatedly called tens of thousands of times to generate 1 second of speech, so the software and hardware interaction method based on instruction distribution will waste a lot of delay in instruction communication, and the final performance cannot meet the real-time requirements.

[0081] The present embodiment provides an acceleration device for deep learning, comprising: a data buffer 201, and a computing module 202 connected to the data buffer;

[0082] The data buffer is used to cache the intermediate calculation results of the network layer of the vocoder neural network;

[0083] The operation module is used to obtain the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction, and send the intermediate calculation result of the first network layer to the data buffer.

[0084] The step of obtaining the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction and sending the intermediate calculation result of the first network layer to the data buffer comprises: obtaining the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction;

[0085] According to the intermediate calculation result of the first network layer, obtaining the intermediate calculation result of the second network layer cascaded with the first network layer;

[0086] The intermediate calculation result of the second network layer is sent to the data buffer.

[0087] Figure 2 In the embodiment, the I / O interface 206 for off-chip instruction and data communication and the on-chip storage 205 for storing neural network parameters are typical modules of the hardware accelerator.

[0088] Figure 2 A self-iterative structure based on a data buffer is provided. The vocoder neural network consists of multiple cascaded neural network layers, and the input data size, weight size, and computational complexity of each neural network layer are much smaller than computationally intensive algorithms such as convolutional neural networks. Based on this, this embodiment uses a data buffer to directly cache the calculation results of the previous neural network layer in a high-speed storage medium such as a register, and directly use them for the calculation of the next neural network layer. Figure 2 As shown in Figure 2, this self-iterative structure is much better than the traditional inter-layer data interaction method that requires access to on-chip RAM or even low-speed storage such as off-chip DDR (such as Figure 4 This is a typical architecture diagram for convolutional neural networks. Note Figure 4 The same is the architecture diagram. Figure 4 The processing module (Processing Element) in the embodiment is different from the internal specific design of the processing module in this embodiment), which can effectively reduce the delay cost caused by inter-layer data access and is suitable for applications such as vocoder neural networks that are extremely sensitive to delay.

[0089] In this embodiment, the network layer of the vocoder neural network can be Figure 3b The intermediate calculation result of the network layer can be Figure 3b The data calculation results of the residual network layer in .

[0090] The acceleration device applied to the vocoder neural network also includes a controller;

[0091] The controller is used to send the control instruction to the computing unit.

[0092] The acceleration device for the vocoder neural network provided in this embodiment is used to Figure 2 The extremely simple hardware structure realizes the acceleration function of the vocoder neural network. The controller is used to control the function switching and data reading operation of the operation module.

[0093] The following is an explanation of the computing module of the acceleration device applied to the vocoder neural network. Figure 5 , which is a schematic diagram of the structure of the operation module. The operation module may include: a matrix operation unit 501, an addition unit 502 connected to the matrix operation unit, a sigmoid activation unit (sigmoid) 503 and a hyperbolic tangent activation unit 504 connected to the addition unit, a multiplication unit 505 connected to the sigmoid activation unit and the hyperbolic tangent activation unit (tanh) respectively, and a deviation unit 506 connected to the multiplication unit; the sigmoid activation unit 503 and the hyperbolic tangent activation unit 504 may constitute an activation unit.

[0094] The matrix operation unit is used to perform a multiplication-accumulation operation on the intermediate calculation result of the first network layer to obtain multiplication-accumulation data;

[0095] The adding unit is used to perform an addition operation on the multiplication and accumulation data to obtain addition sum data;

[0096] The sigmoid function activation unit is used to perform an activation operation on the addition data to obtain first activation data;

[0097] The hyperbolic tangent activation unit is used to perform a hyperbolic tangent activation operation on the addition data to obtain second activation data;

[0098] The multiplication operation unit is used to perform a multiplication operation on the first activation data and the second activation data to obtain product data;

[0099] The deviation unit is used to perform a deviation operation on the product data to obtain an intermediate calculation result of the second network layer.

[0100] In this embodiment, the matrix operation unit is based on the matrix operation operator that accounts for the main amount of operation in the vocoder neural network, extracts the common calculation mode between two matrix operation operators in the vocoder neural network, and obtains an operation unit that can cover all calculation modes in the vocoder neural network. The matrix operation unit is implemented as a multiply-accumulate (MAC) array; Figure 5 The tanh, sigmoid, element-to-element multiplication Mul and bias addition modules are used to support Figure 3c The main operations in the residual network layer in ; normalized index and random selection (SoftMax+RandomChoice) units are used to support Figure 3b The tail operation at the output of one iteration of the vocoder neural network (in Figure 3b In the example, the output tail of the vocoder neural network iteration is composed of a rectified linear unit, a convolution operation unit, a normalized index unit, and a random selection unit); ReLU is used to support Figure 3b The Residual Block external transformation operation in the upper left corner; the addition unit will be reused by multiple paths. The function switching of the operation module is realized by the on-chip controller. When executing the same function, the data of each module does not need to be cached in any form, which reduces the data cache cost and the instruction switching cost between operators compared to the traditional method.

[0101] The operation module further includes: a modified linear activation unit 507 connected to the addition unit;

[0102] The rectified linear unit is used to perform a rectified linear activation operation on the addition data to obtain third activation data;

[0103] The third activation data is used as intermediate data of a tail network layer of the vocoder neural network.

[0104] exist Figure 3b In the embodiment, the tail network layer of the vocoder neural network is composed of a rectified linear activation unit, a convolution operation unit, a normalized exponential unit, and a random selection unit.

[0105] The operation module further includes: a normalization and random selection unit connected to the matrix operation unit;

[0106] The normalization and random selection unit is used to perform normalization processing on the third activation data to obtain classification data; and obtain the output audio data of the vocoder neural network according to the classification data with the highest probability in the classification data.

[0107] See also Figure 3b In this embodiment, a softmax classifier is used to perform normalization processing on the third activation data to obtain classification data. The output audio data of the vocoder neural network is obtained according to the classification data with the highest probability in the classification data.

[0108] The normalization and random selection unit is also used for:

[0109] Performing a random selection operation in the classified data to obtain random data;

[0110] The random data is used as input data of the vocoder neural network.

[0111] See also Figure 3b In this embodiment, a random selection operation is performed in the classified data to obtain random data; and the random data is used as input data of the vocoder neural network.

[0112] The performing a logistic regression activation operation on the addition data to obtain first activation data includes:

[0113] obtaining the added sum data;

[0114] According to the addition data, a search is performed in a preset logistic regression lookup table to obtain the first activation data.

[0115] In this embodiment, in order to generate 1 second of audio data, the sigmoid and tanh activation functions need to be run about 320,000 times. In order to improve the calculation efficiency of sigmoid and tanh, this embodiment first prepares a preset logistic regression lookup table, and then searches in the preset sigmoid function value lookup table according to the addition and data to obtain the first activation data. This implementation method reduces the running time of the activation function.

[0116] The performing a hyperbolic tangent activation operation on the addition data to obtain second activation data includes:

[0117] obtaining the added sum data;

[0118] According to the addition data, a search is performed in a preset hyperbolic tangent lookup table to obtain the second activation data.

[0119] In this embodiment, in order to generate 1 second of audio data, the tanh activation function needs to be run about 320,000 times. In order to improve the calculation efficiency of tanh, this embodiment first prepares a preset logistic regression lookup table, and then searches in a preset hyperbolic tangent lookup table according to the addition and data to obtain the second activation data. This implementation method reduces the running time of the activation function.

[0120] The acceleration device for deep learning further includes: a jump unit;

[0121] The jumper unit is used for pipelined execution of data between the matrix operation unit and the first network layer.

[0122] The acceleration device for deep learning is implemented using a field programmable gate array or a dedicated integrated circuit.

[0123] The acceleration device for deep learning provided in this embodiment is implemented using a field programmable gate array or an application-specific integrated circuit.

[0124] The acceleration device for deep learning provided in this embodiment can be deployed in the cloud. The direct product is the deep learning speech generation computing service, and the indirect product is the product based on the speech generation service, such as text-to-speech, speech translation, etc.

[0125] The acceleration device for deep learning provided in this embodiment can also be deployed on an embedded device, and the products are various devices that include speech generation functions, such as various intelligent speech interaction devices.

[0126] The second embodiment of the present application provides a computing device for deep learning, which is the computing module in the first embodiment of the present application.

[0127] Please refer to Figure 5 , which is a schematic diagram of the structure of the computing device. The computing device includes: a matrix operation unit 501, an addition unit 502 connected to the matrix operation unit, a sigmoid function activation unit 503 and a hyperbolic tangent activation unit 504 connected to the addition unit, a multiplication unit 505 connected to the sigmoid function activation unit and the hyperbolic tangent activation unit respectively, and a deviation unit 506 connected to the multiplication unit; the sigmoid function activation unit and the hyperbolic tangent activation unit can constitute an activation unit.

[0128] The matrix operation unit is used to perform a multiplication-accumulation operation on the intermediate calculation result of the first network layer to obtain multiplication-accumulation data;

[0129] The adding unit is used to perform an addition operation on the multiplication and accumulation data to obtain addition sum data;

[0130] The sigmoid function activation unit is used to perform an activation operation on the addition data to obtain first activation data;

[0131] The hyperbolic tangent activation unit is used to perform a hyperbolic tangent activation operation on the addition data to obtain second activation data;

[0132] The multiplication operation unit is used to perform a multiplication operation on the first activation data and the second activation data to obtain product data;

[0133] The deviation unit is used to perform a deviation operation on the product data to obtain an intermediate calculation result of the second network layer.

[0134] In this embodiment, the matrix operation unit is based on the matrix operation operator that accounts for the main amount of operation in the vocoder neural network, extracts the common calculation mode between two matrix operation operators in the vocoder neural network, and obtains an operation unit that can cover all calculation modes in the vocoder neural network. The matrix operation unit is implemented as a multiply-accumulate (MAC) array; Figure 5 The tanh, sigmoid, element-to-element multiplication Mul and bias addition modules are used to support Figure 3c The main operations in the residual network layer in ; normalized index and random selection (SoftMax+RandomChoice) units are used to support Figure 3b The tail operation at the output of one iteration of the vocoder neural network (in Figure 3b In the example, the output tail of the vocoder neural network iteration is composed of a modified linear activation unit, a convolution operation unit, a normalized index unit, and a random selection unit); ReLU is used to support Figure 3bThe Residual Block external transformation operation in the upper left corner; the addition unit will be reused by multiple paths. The function switching of the operation module is realized by the on-chip controller. When executing the same function, the data of each module does not need to be cached in any form, which reduces the data cache cost and the instruction switching cost between operators compared to the traditional method.

[0135] The operation module further includes: a modified linear activation unit 507 connected to the addition unit;

[0136] The rectified linear unit is used to perform a rectified linear activation operation on the addition data to obtain third activation data;

[0137] The third activation data is used as intermediate data of a tail network layer of the vocoder neural network.

[0138] exist Figure 3b In the embodiment, the tail network layer of the vocoder neural network is composed of a rectified linear activation unit, a convolution operation unit, a normalized exponential unit, and a random selection unit.

[0139] The operation module further includes: a normalization and random selection unit connected to the matrix operation unit;

[0140] The normalization and random selection unit is used to perform normalization processing on the third activation data to obtain classification data; and obtain the output audio data of the vocoder neural network according to the classification data with the highest probability in the classification data.

[0141] See also Figure 3b In this embodiment, a softmax classifier is used to perform normalization processing on the third activation data to obtain classification data. The output audio data of the vocoder neural network is obtained according to the classification data with the highest probability in the classification data.

[0142] The normalization and random selection unit is also used for:

[0143] Performing a random selection operation in the classified data to obtain random data;

[0144] The random data is used as input data of the vocoder neural network.

[0145] See also Figure 3b In this embodiment, a random selection operation is performed in the classified data to obtain random data; and the random data is used as input data of the vocoder neural network.

[0146] The performing a sigmoid function activation operation on the addition and data to obtain first activation data comprises:

[0147] obtaining the added sum data;

[0148] According to the addition and data, a search is performed in a preset sigmoid function value lookup table to obtain the first activation data.

[0149] In this embodiment, in order to generate 1 second of audio data, the sigmoid and tanh activation functions need to be run about 320,000 times. In order to improve the calculation efficiency of sigmoid and tanh, this embodiment first prepares a preset logistic regression lookup table, and then searches in the preset sigmoid function value lookup table according to the addition and data to obtain the first activation data. This implementation method reduces the running time of the activation function.

[0150] The step of performing a hyperbolic tangent activation operation on the addition data to obtain second activation data includes:

[0151] obtaining the added sum data;

[0152] According to the addition data, a search is performed in a preset hyperbolic tangent lookup table to obtain the second activation data.

[0153] In this embodiment, in order to generate 1 second of audio data, the tanh activation function needs to be run about 320,000 times. In order to improve the calculation efficiency of tanh, this embodiment first prepares a preset logistic regression lookup table, and then searches in a preset hyperbolic tangent lookup table according to the addition and data to obtain the second activation data. This implementation method reduces the running time of the activation function.

[0154] The acceleration device for deep learning further includes: a jump unit;

[0155] The jumper unit is used for pipelined execution of data between the matrix operation unit and the first network layer.

[0156] The acceleration device for deep learning is implemented using a field programmable gate array or a dedicated integrated circuit.

[0157] The acceleration device for deep learning provided in this embodiment is implemented using a field programmable gate array or an application-specific integrated circuit.

[0158] The third embodiment of the present application provides an electronic device, which includes the acceleration device for deep learning provided by the first embodiment of the present application or the computing device for deep learning provided by the second embodiment of the present application.

[0159] The fourth embodiment of the present application provides a text information processing method, see Figure 6, which is a flowchart of a text information processing method provided in the fourth embodiment of the present application. The text information processing method includes:

[0160] Step S601: Obtain text information to be processed.

[0161] This step is used to obtain the text information to be processed.

[0162] For example, the text information to be processed is a piece of text to be converted into voice data.

[0163] Step S602: performing sequence conversion on the text information to obtain spectrum data.

[0164] This step is used to perform sequence conversion on the text information to obtain spectrum data.

[0165] Since this step is a prior art, it will not be described in detail here.

[0166] Step S603: According to the spectrum data, using the acceleration device for deep learning provided by the first embodiment of the present application or the computing device for deep learning provided by the second embodiment of the present application, obtain the voice data corresponding to the text information.

[0167] This step is used to obtain the voice data corresponding to the text information based on the spectrum data, using the acceleration device for deep learning provided by the first embodiment of the present application or the computing device for deep learning provided by the second embodiment of the present application.

[0168] The fifth embodiment of the present application provides a speaker, which includes the acceleration device for deep learning provided by the first embodiment of the present application or the computing device for deep learning provided by the second embodiment of the present application.

[0169] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0170] In a typical configuration, a computing device includes one or more operating units (CPU), input / output interfaces, network interfaces, and memory.

[0171] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0172] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0173] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. An acceleration device for deep learning, characterized in that: include: A data buffer, and a computing module connected to the data buffer; The data buffer is used to cache the intermediate calculation results of the network layer of the vocoder neural network; The operation module is used to obtain the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction, and send the intermediate calculation result of the first network layer to the data buffer; The operation module includes: a matrix operation unit, an addition unit connected to the matrix operation unit, an activation unit connected to the addition unit, a multiplication operation unit connected to the activation unit, and a deviation unit connected to the multiplication operation unit; The matrix operation unit is used to perform a multiplication-accumulation operation on the intermediate calculation result of the first network layer to obtain multiplication-accumulation data; The adding unit is used to perform an addition operation on the multiplication and accumulation data to obtain addition sum data; The activation unit is used to perform an activation operation on the addition data, search in a preset activation function value lookup table, and obtain first activation data; The multiplication operation unit is used to perform a multiplication operation on the activation data to obtain product data; The deviation unit is used to perform a deviation operation on the product data to obtain an intermediate calculation result of a second network layer cascaded with the first network layer; The function switching of the operation module is realized by the on-chip controller, and when executing the same function, the data of each module does not need to be cached.

2. The acceleration device for deep learning according to claim 1, characterized in that: Also includes a controller; The controller is used to send the control instruction to the computing unit.

3. The acceleration device for deep learning according to claim 1, characterized in that: The step of obtaining the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction and sending the intermediate calculation result of the first network layer to the data buffer comprises: obtaining the intermediate calculation result of the first network layer of the vocoder neural network from the data buffer according to the received control instruction; According to the intermediate calculation result of the first network layer, obtaining the intermediate calculation result of the second network layer cascaded with the first network layer; The intermediate calculation result of the second network layer is sent to the data buffer.

4. The acceleration device for deep learning according to claim 1, characterized in that: The operation module further includes: a modified linear activation unit connected to the addition unit; The modified linear activation unit is used to perform a modified linear activation operation on the addition data to obtain third activation data; The third activation data is used as intermediate data of a tail network layer of the vocoder neural network.

5. The acceleration device for deep learning according to claim 4, characterized in that: The operation module further includes: a normalization and random selection unit connected to the matrix operation unit; The normalization and random selection unit is used to perform normalization processing on the third activation data to obtain classification data; and obtain the output audio data of the vocoder neural network according to the classification data with the highest probability in the classification data.

6. The acceleration device for deep learning according to claim 5, characterized in that: The normalization and random selection unit is also used for: Performing a random selection operation in the classified data to obtain random data; The random data is used as input data of the vocoder neural network.

7. The acceleration device for deep learning according to claim 1, characterized in that: The performing an activation operation on the addition data to obtain first activation data includes: obtaining the added sum data; According to the addition and data, a search is performed in a preset activation function value lookup table to obtain first activation data.

8. The acceleration device for deep learning according to claim 1, characterized in that: Also includes: Jumper unit; The jumper unit is used for pipelined execution of data between the matrix operation unit and the first network layer.

9. The acceleration device for deep learning according to any one of claims 1 to 8, characterized in that: The acceleration device for deep learning is implemented using a field programmable gate array or a dedicated integrated circuit.

10. A computing device for deep learning, characterized in that: include: A matrix operation unit, an addition unit connected to the matrix operation unit, an activation unit connected to the addition unit, a multiplication operation unit connected to the activation unit, and a deviation unit connected to the multiplication operation unit; The matrix operation unit is used to perform a multiplication-accumulation operation on the intermediate calculation result of the first network layer of the vocoder neural network to obtain multiplication-accumulation data; The adding unit is used to perform an addition operation on the multiplication and accumulation data to obtain addition sum data; The activation unit is used to perform an activation operation on the addition data, search in a preset activation function value lookup table, and obtain first activation data; The multiplication operation unit is used to perform a multiplication operation on the activation data to obtain product data; The deviation unit is used to perform a deviation operation on the product data to obtain an intermediate calculation result of a second network layer cascaded with the first network layer; The function switching of the computing device for deep learning is achieved through an on-chip controller, and when executing the same function, the data of each unit does not need to be cached.

11. The computing device for deep learning according to claim 10, characterized in that: It also includes: a modified linear activation unit connected to the adding unit; The modified linear activation unit is used to perform a modified linear activation operation on the addition data to obtain third activation data; The third activation data is used as intermediate data of a tail network layer of the vocoder neural network.

12. The computing device for deep learning according to claim 11, characterized in that: Also includes: A normalization and random selection unit connected to the matrix operation unit; The normalization and random selection unit is used to perform normalization processing on the third activation data to obtain classification data; and obtain the speech data output by the vocoder neural network based on the classification data with the highest probability in the classification data.

13. The computing device for deep learning according to claim 12, characterized in that: The normalization and random selection unit is also used for: Performing a random selection operation in the classified data to obtain random data; The random data is used as input data of the vocoder neural network.

14. The computing device for deep learning according to claim 10, characterized in that: The performing an activation operation on the addition data to obtain first activation data includes: obtaining the added sum data; According to the addition and data, a search is performed in a preset activation function value lookup table to obtain first activation data.

15. The computing device for deep learning according to any one of claims 10 to 14, characterized in that: The computing device for deep learning is implemented using a field programmable gate array or a dedicated integrated circuit.

16. An electronic device, characterized in that: The electronic device includes any one of the acceleration devices for deep learning described in claims 1-9 or any one of the computing devices for deep learning described in claims 10-15.

17. A sound box, characterized in that: The speaker includes any one of the acceleration devices for deep learning described in claims 1-9 or any one of the computing devices for deep learning described in claims 10-15.

18. A text information processing method, characterized in that: include: Get the text information to be processed; Performing sequence conversion on the text information to obtain spectrum data; According to the spectrum data, the speech data corresponding to the text information is obtained using the acceleration device for deep learning described in any one of claims 1 to 9 or the computing device for deep learning described in any one of claims 10 to 15.

Images