Method and computing device for performing calculations of multiple neural networks

Abstract

A method and computing device for performing calculations on multiple neural networks. The method includes the following steps performed by a computer: when the structures of corresponding layers of multiple neural networks are identical, the corresponding layers of the multiple neural networks are spliced ​​into a spliced ​​layer, and the original input tensors of the corresponding layers are spliced ​​into a tensor with one dimension higher than the original input tensor; the calculation on the corresponding layer is completed by inputting the one-dimensional tensor into the spliced ​​layer to obtain the output tensor of the spliced ​​layer; when the structures of multiple corresponding layers of the multiple neural networks connected in sequence are identical, the first corresponding layer of the multiple corresponding layers of the multiple neural networks is spliced ​​into a first spliced ​​layer; the original input tensors of the first corresponding layer are spliced ​​into a tensor with one dimension higher than the original input tensor; and the calculation on the first corresponding layer is completed by inputting the one-dimensional tensor into the first spliced ​​layer to obtain the output tensor of the first spliced ​​layer.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and more particularly, to a method and a computing device for performing calculations of multiple neural networks. Background Art

[0002] With the emergence of massive data, artificial intelligence technology has developed rapidly, and machine learning is an inevitable product of the development of artificial intelligence to a certain stage. It is committed to mining valuable potential information from large amounts of data through computational means (for example, by using neural networks).

[0003] However, as the amount of data increases, the computer's processing power for neural networks (for example, computing speed) is also being put to the test. In addition, in fields such as image processing, named entity recognition, and speech recognition, neural networks with layers of identical structure (for example, isomorphic neural networks with exactly the same layer structure) are often required to process a set of corresponding inputs (for example, image input, text input, and speech input, etc.). For the calculation of neural networks with this structure, the traditional approach is to perform calculations on each network separately. When the number of networks is large, the amount of calculation is very large and time-consuming. It can be seen that the traditional calculation method for isomorphic neural networks has low computational efficiency and does not meet the needs of rapid updates and iterations in actual business. Summary of the Invention

[0004] An object of the present invention is to provide a method and a computing device for performing calculations of multiple neural networks to at least partially solve the above-mentioned problems.

[0005] One aspect of the present invention provides a method for executing calculations of multiple neural networks by at least one computer, wherein the method includes the following steps executed by the at least one computer: when the structures of corresponding layers of the multiple neural networks are the same, splicing the corresponding layers of the multiple neural networks into a splicing layer, and splicing the original input tensors of the corresponding layers of the multiple neural networks into a tensor with one dimension higher than the original input tensor; completing the calculations on the corresponding layers in the multiple neural networks by inputting the one-dimensional tensor into the splicing layer to obtain the output tensor of the splicing layer; when the structures of multiple corresponding layers of the multiple neural networks connected sequentially are respectively the same, splicing the first corresponding layer of the multiple corresponding layers of the multiple neural networks connected sequentially into a first splicing layer; splicing the original input tensors of the first corresponding layer of the multiple neural networks into a tensor with one dimension higher than the original input tensor; and completing the calculations on the first corresponding layer in the multiple neural networks by inputting the one-dimensional tensor into the first splicing layer to obtain the output tensor of the first splicing layer.

[0006] Optionally, when the structures of the multiple corresponding layers connected sequentially in the multiple neural networks are respectively the same, the method further includes: for each subsequent corresponding layer after the first corresponding layer in the multiple neural networks, splicing the subsequent corresponding layers of the multiple neural networks into a second splicing layer; and completing the calculation of the subsequent corresponding layer in the multiple neural networks by inputting the output result of the previous corresponding layer of the subsequent corresponding layer into the second splicing layer to obtain the output result of the second splicing layer.

[0007] Optionally, the method further includes: for the last corresponding layer of the multiple corresponding layers connected sequentially in the multiple neural networks, transforming the output tensor of the spliced ​​layer formed by splicing the last corresponding layers of the multiple neural networks into a tensor with one dimension lower than the output tensor itself.

[0008] Optionally, the method further comprises: when the structures of corresponding layers of the plurality of neural networks are the same, transforming the output tensor of the concatenated layer into a tensor with one dimension lower than the output tensor itself

[0009] Optionally, the corresponding layer is any one of the following: a fully connected layer; a batch normalization layer; an activation function layer; an element-wise product layer; or a class decomposition layer.

[0010] Optionally, when the corresponding layer is a fully connected layer, the spliced ​​layer after splicing is a locally connected layer.

[0011] Optionally, the calculations on corresponding layers with the same structure in the multiple neural networks are completed by calling an underlying parallel matrix operation library, wherein the parallel matrix operation library includes at least one of MKL, OpenBlas, CUDA, and cublas.

[0012] Optionally, when the corresponding layer is a fully connected layer and the spliced ​​layer after splicing is a locally connected layer, the matrix multiplication in the locally connected layer is accelerated using the matrix operation library batch matrix multiplication function, wherein the batch matrix multiplication function includes at least one of the following functions: the cblas_dgemm_batch function of the Mkl library; the cublasDgemmStridedBatched function of the cublas library.

[0013] Optionally, the multiple neural networks are homogeneous neural networks in deep neural networks.

[0014] One aspect of the present invention provides a computing device for performing calculations on multiple neural networks, wherein the computing device includes: a network splicing unit configured to: when the structures of a corresponding layer of the multiple neural networks are the same, splice the corresponding layers of the multiple neural networks into a spliced ​​layer; when the structures of the multiple corresponding layers of the multiple neural networks connected in sequence are respectively the same, splice the first corresponding layer of the multiple corresponding layers of the multiple neural networks connected in sequence into a first spliced ​​layer; an input splicing unit configured to: when the structures of a corresponding layer of the multiple neural networks are the same, splice the original input tensors of the corresponding layers of the multiple neural networks into a tensor with one dimension higher than the original input tensor; when the structures of the multiple neural networks are the same, splice the original input tensors of the corresponding layers of the multiple neural networks into a tensor with one dimension higher than the original input tensor; When the structures of multiple corresponding layers connected in sequence are respectively the same, the original input tensor of the first corresponding layer of the multiple neural networks is spliced ​​into a tensor with one dimension higher than the original input tensor; the computing unit is configured to: when the structures of a corresponding layer of the multiple neural networks are the same, the calculation of the corresponding layer in the multiple neural networks is completed by inputting the one-dimensional tensor into the splicing layer to obtain the output tensor of the splicing layer; when the structures of multiple corresponding layers of the multiple neural networks connected in sequence are respectively the same, the calculation of the first corresponding layer in the multiple neural networks is completed by inputting the one-dimensional tensor into the first splicing layer to obtain the output tensor of the first splicing layer.

[0015] Optionally, when the structures of the multiple corresponding layers connected sequentially in the multiple neural networks are respectively the same, the network splicing unit is configured to splice the subsequent corresponding layers of the multiple neural networks into a second splicing layer for each subsequent corresponding layer after the first corresponding layer in the multiple neural networks; the calculation unit is configured to complete the calculation of the subsequent corresponding layer in the multiple neural networks by inputting the output result of the previous corresponding layer of the subsequent corresponding layer into the second splicing layer to obtain the output result of the second splicing layer.

[0016] Optionally, the computing device further includes: an output splicing unit, configured to transform, for the last corresponding layer of the multiple corresponding layers connected sequentially in the multiple neural networks, an output tensor of a spliced ​​layer spliced ​​by the last corresponding layers of the multiple neural networks into a tensor having one dimension lower than the output tensor itself.

[0017] Optionally, the computing device further includes: an output splicing unit configured to, when the structures of corresponding layers of the multiple neural networks are the same, transform the output tensor of the splicing layer into a tensor with one dimension lower than the output tensor itself.

[0018] Optionally, the corresponding layer is any one of the following: a fully connected layer; a batch normalization layer; an activation function layer; an element-wise product layer; or a class decomposition layer.

[0019] Optionally, when the corresponding layer is a fully connected layer, the spliced ​​layer after splicing is a locally connected layer.

[0020] Optionally, the computing unit is configured to complete the calculations on corresponding layers with the same structure in the multiple neural networks by calling an underlying parallel matrix operation library, wherein the parallel matrix operation library includes at least one of MKL, OpenBlas, CUDA, and cublas.

[0021] Optionally, when the corresponding layer is a fully connected layer and the spliced ​​layer after splicing is a locally connected layer, the matrix multiplication in the locally connected layer is accelerated using the matrix operation library batch matrix multiplication function, wherein the batch matrix multiplication function includes at least one of the following functions: the cblas_dgemm_batch function of the Mkl library; the cublasDgemmStridedBatched function of the cublas library.

[0022] Optionally, the multiple neural networks are homogeneous neural networks in deep neural networks.

[0023] One aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by one or more computing devices, the one or more computing devices are enabled to implement any one of the methods described above.

[0024] One aspect of the present invention provides a system comprising one or more computing devices and one or more storage devices, wherein the one or more storage devices have recorded thereon a computer program, which, when executed by the one or more computing devices, enables the one or more computing devices to implement any one of the methods described above.

[0025] The present invention combines multiple neural network layers with identical structures, concatenates the original input tensors of the corresponding layers into tensors one dimension higher than the original input, and performs operations using the higher-dimensional tensors and the concatenated layers. This technical solution converts operations corresponding to each corresponding layer with a relatively high number of operations into batch operations corresponding to the concatenated layers with a relatively low number of operations. Because each operation requires a call to the operation library, and each activation of the operation library incurs a certain fixed overhead, the more operations are performed, the greater this fixed overhead. Therefore, the present invention significantly reduces this fixed overhead by reducing the number of operations while ensuring operational equivalence, thereby improving operational efficiency.

[0026] Additional aspects and / or advantages of the present general inventive concept will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present general inventive concept. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other objects and features of the present invention will become more apparent from the following description with reference to the accompanying drawings which illustrate an example, in which:

[0028] Figure 1A A method for performing calculations of multiple neural networks by at least one computer according to one embodiment of the present invention is shown;

[0029] Figure 1B A method for executing calculations of a plurality of neural networks by at least one computer according to another embodiment of the present invention is shown;

[0030] Figure 2 A computing device for performing calculations of a plurality of neural networks according to the present invention is shown;

[0031] Figure 3A An example of performing calculations of multiple neural networks in the prior art is shown;

[0032] Figure 3B The present invention shows Figure 3A Examples of multiple neural networks processing and calculations;

[0033] Figure 4A It is a schematic diagram of three independent and identically structured fully connected layers;

[0034] Figure 4B Yes Figure 4A Schematic diagram of the local connection layer obtained by concatenating the three fully connected layers in . DETAILED DESCRIPTION

[0035] The following description, with reference to the accompanying drawings, is provided to facilitate a more complete understanding of the exemplary embodiments of the present invention as defined by the claims and their equivalents. The description includes various specific details to facilitate understanding, but these details are to be regarded as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Furthermore, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0036] In the present invention, the multiple neural networks may be multiple neural networks included in a deep neural network (DNN) (e.g., a deep sparse network (DSN)). It should be understood that the above description is merely an example, and the present invention is not limited thereto. The multiple neural networks may also include multiple neural networks in other types of neural networks. In addition, the multiple neural networks herein may be used for multiple neural networks for various purposes. For example, the multiple neural networks may be used to process image input, voice input, text input, and the like. It should be understood that the inputs of the multiple neural networks described above are merely examples, and the present invention is not limited thereto. The multiple neural networks may also be used to process other inputs.

[0037] Figure 1A A method for executing calculations of multiple neural networks by at least one computer according to one embodiment of the present invention is shown.

[0038] Reference Figure 1A , the method for executing calculations of a plurality of neural networks by at least one computer according to the present invention includes steps S110, S120, and S130 executed by the at least one computer.

[0039] In step S101, when the structures of corresponding layers of the plurality of neural networks are the same, the corresponding layers of the plurality of neural networks are spliced ​​into a spliced ​​layer. Here, the spliced ​​layer refers to the layer after splicing.

[0040] Here, the identical structure of corresponding layers in multiple neural networks may mean that the number of neurons in the corresponding layers and the connectivity between the neurons are identical. In other words, the multiple neural networks all have corresponding layers with identical number of neurons and connectivity. As an example, the multiple neural networks are isomorphic neural networks in a deep neural network.

[0041] The corresponding layer is any one of the following: a fully connected layer; a batch normalization layer; an activation function layer; an element-wise product layer; or a factorization machine layer (FM layer). It should be understood that the above corresponding layers are merely examples and the present invention is not limited thereto. The corresponding layer may also be another layer whose input is operated on. For example, when the corresponding layer is a fully connected layer, the spliced ​​layer after splicing is a locally connected layer.

[0042] Here, the connection relationship between each neuron in the spliced ​​layer is the same as that of each corresponding neuron in the corresponding layer before splicing in the plurality of neural networks. In other words, the connection relationship between each neuron in the corresponding layer before and after splicing does not change.

[0043] In step S102, the original input tensors of the corresponding layers of the plurality of neural networks are concatenated into a tensor having one dimension higher than the original input tensor. Here, the original input tensor may refer to the input tensor of the corresponding layer before concatenation.

[0044] In other words, when the dimension of the original input tensor is k (k is an integer greater than or equal to 0), the dimension of the resulting tensor is k+1. For example, when the original input tensors are [0,1] and [0,2], each with a dimension of 1, the resulting tensor is a tensor with a dimension of 2. It should be understood that the above examples are illustrative and the present invention is not limited thereto. The original input tensor can be various types of tensors with the same dimension.

[0045] It should be understood that although Figure 1A 102 is shown after step S101, but the present invention is not limited thereto. For example, step S101 may be performed after step S102 or step S102 and step S101 may be performed simultaneously.

[0046] In step S103, the calculation of the corresponding layer in the multiple neural networks is completed by inputting the higher-dimensional tensor into the concatenated layer to obtain the output tensor of the concatenated layer.

[0047] At this time, when a tensor with one dimension higher than the original input tensor is input to the spliced ​​layer formed by splicing the corresponding layers of multiple neural networks, the matrix operations of the corresponding layers of the multiple neural networks before splicing are converted into batch matrix operations.

[0048] In this case, the calculation of the corresponding layers with the same structure in the multiple neural networks can be completed by calling the underlying parallel matrix operation library, wherein the parallel matrix operation library includes at least one of MKL, OpenBlas, CUDA, and cublas. As an example, the parallel matrix operation library can be the MKL (Math Kernel Library) library, the cublas library in the CUDA library, etc. However, the present invention does not limit the parallel matrix operation library. In one embodiment, when the corresponding layer is a fully connected layer and the spliced ​​layer after splicing is a local connection layer, the matrix multiplication in the local connection layer is accelerated using the matrix operation library batch matrix multiplication function, wherein the batch matrix multiplication function includes at least one of the following functions: the cblas_dgemm_batch function of the Mkl library; the cublasDgemmStridedBatched function of the cublas library.

[0049] In practice, there is a fixed overhead associated with starting up a computational library (e.g., GPU kernel startup time). Therefore, by converting multiple matrix operations into batch operations and performing them using a parallel matrix operation library, we can reduce the startup overhead of the computational library while making better use of computing resources.

[0050] Alternatively, the output tensor of the concatenated layer can be transformed into a tensor with one dimension lower than the output vector itself. In this case, multiple tensors identical to the output tensors of the corresponding layers before concatenation in multiple neural networks can be obtained. In other words, the concatenated layer can now achieve the same output results as the corresponding layers before concatenation.

[0051] That is, the method of executing calculations of multiple neural networks by at least one computer according to the present invention can greatly improve the calculation efficiency without changing the outputs of corresponding layers of the multiple neural networks.

[0052] Figure 1B A method for executing calculations of a plurality of neural networks by at least one computer according to another embodiment of the present invention is shown.

[0053] Reference Figure 1B , the method for executing calculations of a plurality of neural networks by at least one computer according to the present invention includes steps S111, S112 and S113 executed by the at least one computer.

[0054] In step S111, when the structures of the multiple corresponding layers connected sequentially in the multiple neural networks are respectively the same, the first corresponding layer of the multiple corresponding layers connected sequentially in the multiple neural networks is spliced ​​into a first splicing layer.

[0055] In other words, refer to Figure 1B The method for executing calculations of a plurality of neural networks by at least one computer according to the present invention is described with reference to the case where the structures of the plurality of corresponding layers connected sequentially in the plurality of neural networks are the same. Figure 1A The method of executing calculations of a plurality of neural networks by at least one computer according to the present invention is described for the case where only one corresponding layer of the plurality of neural networks has the same structure. Figure 1A The description of the corresponding layers is also applicable to the corresponding layers here. For the sake of brevity, the same description will not be repeated here.

[0056] As an example, when the plurality of neural networks include a first neural network and a second neural network, the first neural network includes a first network layer L1 and a second network layer L2 directly connected to the first network layer L1 (i.e., the output of the first network layer L1 is input to the second network layer L2), the second neural network includes a third network layer L3 and a fourth network layer L4 directly connected to the third network layer L3 (i.e., the output of the third network layer L3 is input to the fourth network layer L4), and the first network layer L1 and the third network layer L3 have the same structure, and the second network layer L2 and the fourth network layer L4 have the same structure, the first network layer L1 and the third network layer L3 can be regarded as one corresponding layer, and the second network layer L2 and the fourth network layer can be regarded as another corresponding layer. It should be understood that the above example is merely illustrative and the present invention is not limited thereto. Other numbers of neural networks and other numbers of corresponding layers are also feasible.

[0057] Furthermore, the first corresponding layer of the plurality of neural networks refers to the first corresponding layer in the direction from the input layer to the output layer. For example, in the example where the plurality of neural networks includes a first neural network and a second neural network, the first network layer L1 and the third network layer L3 can be considered the first corresponding layer. According to the solution of step S111, the first network layer L1 and the third network layer L3 are spliced ​​into a spliced ​​layer.

[0058] In step S112, the original input tensors of the first corresponding layers of the multiple neural networks are concatenated into a tensor with one dimension higher than the original input tensor.

[0059] Here, the spelling method can be compared with the reference Figure 1A The method described in step S120 is similar, and similar descriptions are not repeated here.

[0060] In the example where the plurality of neural networks includes a first neural network and a second neural network, since the first network layer L1 and the third network layer L3 are considered the first corresponding layer, the original input tensor of the first network layer L1 and the original input tensor of the second network layer L2 are concatenated into a tensor with one dimension higher than the original input tensor of the first network layer L1. It should be understood that the first network layer L1 and the third network layer L3 are corresponding layers with the same structure, and therefore, the input dimensions of the original input tensors of the first network layer L1 and the third network layer L3 are the same. It should be understood that the above example is merely for the purpose of facilitating the understanding of step S112 and is not intended to limit step S112.

[0061] In step S113, the calculation of the first corresponding layer in the plurality of neural networks is completed by inputting the higher-dimensional tensor into the first concatenated layer to obtain the output tensor of the first concatenated layer. Here, the method of obtaining the output tensor of the first concatenated layer can be the same as that of reference numeral 114. Figure 1AThe method described in step S103 is similar, and similar descriptions are not repeated here.

[0062] When the structures of the multiple corresponding layers of the multiple neural networks connected sequentially are respectively the same, the calculations on the first corresponding layers in the multiple neural networks can be completed by calling the underlying parallel matrix operation library, and the calculation efficiency can be greatly improved without changing the outputs of the first corresponding layers of the multiple neural networks.

[0063] In addition, optionally, when the structures of the multiple corresponding layers of the multiple neural networks connected in sequence are respectively the same, the method for performing calculations on multiple neural networks by at least one computer according to the present invention further includes: for each subsequent corresponding layer after the first corresponding layer in the multiple neural networks, the subsequent corresponding layers of the multiple neural networks are spliced ​​into a second splicing layer; the calculation of the subsequent corresponding layer in the multiple neural networks is completed by inputting the output result of the previous corresponding layer of the subsequent corresponding layer into the second splicing layer to obtain the output result of the second splicing layer. Corresponding to the previous example, the first network layer L2 and the third network layer L4 are spliced ​​into a splicing layer, namely the second splicing layer. Here, the splicing method can be the same as that in reference to Figure 1A The method of obtaining the output result of the second splicing layer is similar to the method of the splicing method in step S101 described above. Figure 1A The manner of describing step S103 is similar, and therefore, similar descriptions will not be repeated.

[0064] In other words, when the structures of multiple corresponding layers connected sequentially in the multiple neural networks are respectively the same, the calculations on the subsequent corresponding layers in the multiple neural networks can be completed by calling the underlying parallel matrix operation library, and the calculation efficiency can be greatly improved without changing the outputs of the subsequent corresponding layers of the multiple neural networks.

[0065] Furthermore, optionally, the method for executing computations on multiple neural networks using at least one computer according to the present invention may further include: for the last corresponding layer among the plurality of sequentially connected corresponding layers in the plurality of neural networks, transforming the output tensor of a concatenated layer formed by concatenating the last corresponding layers of the plurality of neural networks into a tensor having one dimension lower than the output tensor itself. In this case, multiple tensors identical to the output tensors of the last corresponding layer in the plurality of neural networks before concatenation can be obtained.

[0066] That is, the method of executing calculations of multiple neural networks by at least one computer according to the present invention can greatly improve the calculation efficiency without changing the final outputs of multiple corresponding layers in a sequential manner of the multiple neural networks.

[0067] Figure 2A computing device 200 is shown performing computations for multiple neural networks in accordance with the present invention.

[0068] Reference Figure 2 The computing device 200 includes a network splicing unit 210, an input splicing unit 220, and a computing unit 230. Here, the computing device 200 may be configured to perform a reference Figure 1A or Figure 1B Therefore, repeated descriptions may be omitted for the sake of brevity. In other words, it should be understood that referring to Figure 1A or Figure 1B The described aspects may also be applied to corresponding aspects of the computing device 200 .

[0069] In the present invention, the network splicing unit 210 is configured to: when the structures of a corresponding layer of the multiple neural networks are the same, splice the corresponding layers of the multiple neural networks into a splicing layer; when the structures of multiple corresponding layers connected sequentially of the multiple neural networks are respectively the same, splice the first corresponding layer of the multiple corresponding layers connected sequentially of the multiple neural networks into a first splicing layer.

[0070] That is, the network splicing unit 210 is configured to perform a reference Figure 1A Step S101 described and reference Figure 1B Therefore, referring to step S111. Figure 1A Step S101 described and reference Figure 1B The description of step S111 is also applicable to the network splicing unit 210 , and similar descriptions are not repeated here.

[0071] In addition, optionally, when the structures of multiple corresponding layers connected sequentially in the multiple neural networks are respectively the same, the network splicing unit 210 can also be configured to splice each subsequent corresponding layer after the first corresponding layer in the multiple neural networks into a second splicing layer.

[0072] In the present invention, the input splicing unit 220 is configured to: when the structures of a corresponding layer of the multiple neural networks are the same, splice the original input tensors of the corresponding layer of the multiple neural networks into a tensor with one dimension higher than the original input tensor; when the structures of multiple corresponding layers of the multiple neural networks connected sequentially are respectively the same, splice the original input tensor of the first corresponding layer of the multiple neural networks into a tensor with one dimension higher than the original input tensor.

[0073] That is, the input assembly unit 220 is configured to perform a reference Figure 1A Step S102 described and referenced Figure 1BTherefore, referring to step S112 Figure 1A Step S102 described and referenced Figure 1B The description of step S112 is also applicable to inputting the splicing unit 220, and similar descriptions are not repeated here.

[0074] In the present invention, the computing unit 230 is configured to: when the structures of a corresponding layer of the multiple neural networks are the same, the computing unit 230 completes the computing of the corresponding layer in the multiple neural networks by inputting the higher-dimensional tensor into the splicing layer to obtain the output tensor of the splicing layer; when the structures of multiple corresponding layers of the multiple neural networks connected sequentially are respectively the same, the computing unit 230 completes the computing of the first corresponding layer in the multiple neural networks by inputting the higher-dimensional tensor into the first splicing layer to obtain the output tensor of the first splicing layer.

[0075] That is, the computing unit 230 is configured to perform a reference Figure 1A Step S103 described and reference Figure 1B Therefore, referring to step S113 Figure 1A Step S103 described and reference Figure 1B The description of step S113 is also applicable to the calculation unit 230. Similar descriptions will not be repeated here.

[0076] In one embodiment, the computing unit 230 may be configured to complete the calculations on the corresponding layers of the same structure in the multiple neural networks by calling the underlying parallel matrix operation library, wherein the parallel matrix operation library includes at least one of MKL, OpenBlas, CUDA, and cublas. As an example, the parallel matrix operation library may be the MKL (Math Kernel Library) library, the cublas library in the CUDA library, etc. However, the present invention does not limit the parallel matrix operation library. In one embodiment, when the corresponding layer is a fully connected layer and the spliced ​​layer after splicing is a locally connected layer, the matrix multiplication in the locally connected layer is accelerated using the matrix operation library batch matrix multiplication function, wherein the batch matrix multiplication function includes at least one of the following functions: the cblas_dgemm_batch function of the MKL library; the cublasDgemmStridedBatched function of the cublas library.

[0077] Generally speaking, there is a certain fixed overhead in starting up the operation library (such as GPU kernel startup time). Therefore, the computing device 200 converts multiple matrix operations into batch matrix operations and uses the parallel matrix operation library to perform the operations, thereby reducing the startup overhead of the operation library and rationally utilizing computing resources.

[0078] In addition, optionally, when the structures of multiple corresponding layers connected sequentially in the multiple neural networks are respectively the same, the computing unit 230 can also be configured to complete the calculation of the subsequent corresponding layer in the multiple neural networks by inputting the output result of the previous corresponding layer of the subsequent corresponding layer into the second splicing layer generated by the network splicing unit 210 to obtain the output result of the second splicing layer.

[0079] In addition, optionally, the computing device 200 may further include an output splicing unit (not shown). In one embodiment, the output splicing unit may be configured to, for the last corresponding layer of the plurality of corresponding layers sequentially connected in the plurality of neural networks, transform the output tensor of the spliced ​​layer formed by splicing the last corresponding layers of the plurality of neural networks into a tensor with one dimension lower than the output tensor itself. In another embodiment, the output splicing unit may be configured to, when the structures of a corresponding layer of the plurality of neural networks are the same, transform the output tensor of the spliced ​​layer into a tensor with one dimension lower than the output tensor itself. That is, the computing device 200 according to the present invention can greatly improve computing efficiency without changing the final outputs of the corresponding layers of the plurality of neural networks.

[0080] Figure 3A An example of performing calculations of multiple neural networks according to the prior art is shown.

[0081] Reference Figure 3A , although the corresponding layers of the plurality of neural networks are shown as fully connected layers, the present invention is not limited thereto. The corresponding layers may be any of the following: fully connected layers; batch normalization layers; activation function layers; element wise product layers; factorization machine layers (FM layers). In addition, although Figure 3A Although multiple corresponding layers are shown, the present invention is not limited thereto, and a corresponding layer with the same structure may also be used. In other words, Figure 3A The neural network in FIG is only an example, and the present invention is not limited thereto.

[0082] In neural networks, the same network structure is usually used to process a set of inputs. Figure 3AThe box represents data, the oval represents the network layer, b represents the batch size (the number of input data items in a single iteration), and i0, f1, f2, and f3 represent the dimensions of the output data of each layer. The entire network is represented as n tensors of shape (b, i0), each of which passes through a fully connected layer to become a tensor of shape (b, f1). Each of these tensors then passes through several layers to become n tensors of shape (b, f3). Finally, these n tensors are concatenated together to form a tensor of shape (b, n×f3). In other words, a tensor of shape (b, n×f3) is formed by concatenating n tensors of shape (b, f3). This type of network is characterized by a set of networks with identical network structures from input to intermediate layers to output. This type of network is often used for model fusion.

[0083] To make it easier to understand the term "putting it together", here, assuming b is 2, f3 is 3, n is 2, a tensor (b, f3) is Another tensor (b, f3) is At this time, by putting the above two tensors together, the result (b, n×f3) is It should be understood that the above assumptions are merely examples, and b, f3, and n in the present invention may be any other natural numbers greater than 0.

[0084] For ease of explanation, it is assumed that the number of networks n is 3. It should be understood that n being 3 is only an example, and n can also be other numbers (for example, 2 or an integer greater than 3). Figure 3A , the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 are corresponding layers with the same structure; the fully connected layer 12 of network 1, the fully connected layer 22 of network 2, and the fully connected layer 32 of network 3 are corresponding layers with the same structure; the fully connected layer 13 of network 1, the fully connected layer 23 of network 2, and the fully connected layer 33 of network 3 are corresponding layers with the same structure.

[0085] In addition, the corresponding layer corresponding to the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 is the corresponding layer closest to the input layer among the multiple corresponding layers of network 1, network 2, and network 3 connected sequentially, and therefore, it can be referred to as the first corresponding layer among the multiple corresponding layers of network 1, network 2, and network 3. The corresponding layer corresponding to the fully connected layer 12 of network 1, the fully connected layer 22 of network 2, and the fully connected layer 32 of network 3, and the corresponding layer corresponding to the fully connected layer 13 of network 3, the fully connected layer 23 of network 2, and the fully connected layer 33 of network 3 are after the first corresponding layer, and therefore, they can be referred to as subsequent corresponding layers among the multiple corresponding layers of network 1, network 2, and network 3. In addition, since the corresponding layer corresponding to the fully connected layer 13 of network 3, the fully connected layer 23 of network 2, and the fully connected layer 33 of network 3 is the corresponding layer farthest from the input layer among the multiple corresponding layers of network 1, network 2, and network 3 connected sequentially, it can be called the last corresponding layer among the multiple corresponding layers of network 1, network 2, and network 3.

[0086] Here, although Figure 3A Three corresponding layers of three neural networks are shown, but the present invention is not limited thereto. For example, the number of neural networks is not limited to three, but may be two or more than three; the number of corresponding layers is also not limited to three, but may be one, two, or more than three. In other words, Figure 3A The neural network in the embodiment is only an example, and the present invention is not limited thereto. The following describes an example in which only one corresponding layer and an example in which only some layers are corresponding layers, with reference to different examples, but the present invention is not limited thereto.

[0087] In one example, when the number of corresponding layers is 1, the corresponding layer is both the first corresponding layer and the last corresponding layer; at this time, there is no subsequent corresponding layer. Figure 3AAssume that the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 are corresponding layers with the same structure, while the fully connected layer 12 of network 1, the fully connected layer 22 of network 2, and the fully connected layer 32 of network 3 are not corresponding layers with the same structure, and the fully connected layer 13 of network 1, the fully connected layer 23 of network 2, and the fully connected layer 33 of network 3 are not corresponding layers with the same structure. In this case, only the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 are concatenated to obtain the concatenated layer. In this case, the original input tensors of the fully connected layer 11 of network 1, the original input tensors of the fully connected layer 21 of network 2, and the original input tensors of the fully connected layer 31 of network 3 can be concatenated into a tensor with one dimension higher than each of the original input tensors, wherein the original input tensors of the fully connected layer 11 of network 1, the original input tensors of the fully connected layer 21 of network 2, and the original input tensors of the fully connected layer 31 of network 3 have the same dimension due to the identical structure of each fully connected layer. Afterwards, the concatenated tensor can be input into the concatenation layer to complete the calculations on the corresponding layers in the multiple neural networks by obtaining the output tensor of the concatenation layer. To facilitate the fully connected layer 12 of network 1, the fully connected layer 22 of network 2, and the fully connected layer 32 of network 3 to receive input, the output tensor of the concatenation layer formed by concatenating the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 can be transformed into a tensor with one dimension lower than the output tensor itself. At this time, the number of tensors with one dimension lower than the output tensor itself corresponds to the number of neural networks. In this example, the number of tensors with one dimension lower than the output tensor itself is 3.

[0088] In yet another example, referring to Figure 3AAssume that the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 are corresponding layers with the same structure, and the fully connected layer 12 of network 1, the fully connected layer 22 of network 2, and the fully connected layer 32 of network 3 are also corresponding layers with the same structure, but the fully connected layer 13 of network 1, the fully connected layer 23 of network 2, and the fully connected layer 33 of network 3 are not corresponding layers with the same structure. In this case, the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 can be concatenated to obtain a concatenated layer. In this case, the original input tensors of the fully connected layer 11 of network 1, the original input tensors of the fully connected layer 21 of network 2, and the original input tensors of the fully connected layer 31 of network 3 can be concatenated into a tensor with one dimension higher than the original input tensors. Thereafter, the concatenated tensor can be input into the concatenated layer to obtain the output tensor of the concatenated layer to complete the calculations for the corresponding layers in the multiple neural networks. Furthermore, the fully connected layer 12 of network 1, the fully connected layer 22 of network 2, and the fully connected layer 32 of network 3 can be concatenated to form a second concatenated layer, and the output of the previous concatenated layer can be input into the second concatenated layer to form the output of the second concatenated layer, thereby completing the calculation of the corresponding layer in the multiple neural networks. To facilitate the fully connected layer 13 of network 1, the fully connected layer 23 of network 2, and the fully connected layer 33 of network 3 to receive input, the output tensor of the second concatenated layer, which is formed by concatenating the fully connected layer 12 of network 1, the fully connected layer 22 of network 2, and the fully connected layer 32 of network 3, can be transformed into a tensor with one dimension lower than the output tensor itself. In this case, the number of tensors with one dimension lower than the output tensor itself corresponds to the number of neural networks. In this example, the number of tensors with one dimension lower than the output tensor itself is 3.

[0089] In addition, although Figure 3A The first corresponding layer corresponding to the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 is shown to be directly connected to the input layer, but the present invention is not limited to this. Other network layers may also exist between the first corresponding layer and the input layer.

[0090] In the following examples, we set Figure 3A The structures of the neural networks in are exactly the same, that is, the fully connected layer 11 of network 1, the fully connected layer 21 of network 2, and the fully connected layer 31 of network 3 are corresponding layers with the same structure, the fully connected layer 12 of network 1, the fully connected layer 22 of network 2, and the fully connected layer 32 of network 3 are corresponding layers with the same structure, and the fully connected layer 13 of network 1, the fully connected layer 23 of network 2, and the fully connected layer 33 of network 3 are corresponding layers with the same structure.

[0091] In the prior art, Figure 3AFor neural networks with the same structure, each neural network needs to be connected separately, and matrix operations (such as matrix multiplication and other operations) need to be performed on each neural network during calculation. Figure 3A For example, a total of 9×n matrix multiplication operations are required (3×n forward + 6×n backward). Generally speaking, matrix multiplication operations in neural networks require the use of underlying parallel matrix operation libraries (MKL, OpenBlas, CUDA, cublas, etc.). The operation library has a certain fixed overhead when starting the operation (such as GPU kernel startup time). The more operations are performed, the greater this overhead is. When the number of networks n is large, this computational efficiency is low.

[0092] Figure 3B The present invention shows Figure 3A Here, similarly, for the sake of convenience, it is assumed that the number of networks n is 3. In this case, Figure 3A The fully connected layer 11, the fully connected layer 21 and the fully connected layer 31 in the CNN are sequentially spliced ​​to obtain Figure 3B The local connection layer 1 in Figure 3A The fully connected layer 12, the fully connected layer 22 and the fully connected layer 32 in are sequentially spliced ​​to obtain Figure 3B The local connection layer 2 in Figure 3A The fully connected layer 13, the fully connected layer 23 and the fully connected layer 33 in are sequentially spliced ​​to obtain Figure 3B The local connection layer 3 in .

[0093] Reference Figure 3B ,and Figure 3A Similarly, the box represents data, the oval represents the network layer, b represents the batch size, that is, the number of input data in one iteration, i0, f1, f2, and f3 represent the dimensions of each layer respectively. The input of the network is n two-dimensional tensors of shape (b, i0) (here, the two-dimensional tensor is a tensor in a relative sense (for example, a tensor relative to the three-dimensional tensor below), not necessarily an absolute two-dimensional tensor). It is first pieced together into a three-dimensional tensor of shape (n, b, i0), and then after passing through the local connection layer 1, it becomes a tensor of shape (n, b, f1). After passing through the local fully connected layer 2, it is deformed into a tensor of shape (n, b, f2). After passing through the local connection layer 3, it becomes a tensor of shape (n, b, f3). Finally, this three-dimensional tensor undergoes shape transformation and deforms (or restores) into a two-dimensional tensor of shape (b, n×f3).

[0094] Reference Figure 3A and Figure 3B It can be seen that the solution according to the present invention Figure 3A The results of multiple neural network processing and calculation are the same as Figure 3AThe results of multiple neural networks calculated using traditional methods are exactly the same. However, the computational efficiency is greatly improved. Figure 3A In the traditional calculation of the neural network, 9×n matrix multiplication operations are required. Figure 3B To calculate the layers of a neural network, only nine batch matrix multiplications are required. In practice, batch matrix multiplications can be accelerated using functions provided by the matrix library (such as the cblas_dgemm_batch function in the Mkl library and the cublasDgemmStridedBatched function in the cublas library), significantly reducing the time required for matrix operations. Other operations, such as addition and activation functions, also require a factor of n, improving overall efficiency.

[0095] In order to clarify that splicing the corresponding layers will not change the original operation results, that is, it is still an equivalent operation, the following Figure 4A and Figure 4B Here, Figure 4A It is a schematic diagram of three independent and identically structured fully connected layers. Figure 4B Yes Figure 4A Schematic diagram of the local connection layer obtained by concatenating the three fully connected layers in .

[0096] Reference Figure 4A , fully connected layer 1, fully connected layer 2, and fully connected layer 3 are three fully connected layers with the same structure and are independent of each other. That is, the number of neurons in fully connected layer 1, fully connected layer 2, and fully connected layer 3 is exactly the same ( Figure 4A In the example, each independent fully connected layer has 4 neurons on each side, and each neuron is connected to the four neurons on the opposite side).

[0097] Will Figure 4A The fully connected layer 1, fully connected layer 2 and fully connected layer 3 are sequentially connected to obtain the following Figure 4B The diagram shows a locally connected layer with 12 neurons on each side. It can be seen that the connections between individual neurons remain unchanged after splicing. However, as a whole, a neuron is not connected to all 12 neurons on the opposite side, so it is a locally connected layer.

[0098] See also Figure 4A and Figure 4B , before and after splicing, the number of neurons and the connection relationship between neurons have not changed. Therefore, the output of each layer before splicing is completely equivalent to the output of the corresponding layer after splicing.

[0099] Here, the fully connected layer 1 can be referenced Figure 3A The fully connected layer 11 and fully connected layer 2 of the described network 1 can be referred to Figure 3A The fully connected layer 21 and fully connected layer 3 of the network 2 described can be referenced Figure 3A The fully connected layer 31 of the network 3 is described, however, the present invention is not limited thereto, and the fully connected layer 1, the fully connected layer 2 and the fully connected layer 3 can also be referenced Figure 3A Other corresponding layers described or additional corresponding layers.

[0100] The method and computing device for performing calculations of multiple neural networks according to exemplary embodiments of the present invention have been described above with reference to Figures 1 to 4. However, it should be understood that the devices, systems, units, etc. used in Figures 1 to 4 can be configured as software, hardware, firmware, or any combination of the above items to perform specific functions. For example, these systems, devices, or units, etc. may correspond to dedicated integrated circuits, pure software codes, or units that combine software and hardware. In addition, one or more functions implemented by these systems, devices, or units, etc. may also be uniformly executed by components in physical entity devices (e.g., processors, clients, or servers, etc.).

[0101] Furthermore, the above methods may be implemented by a computer program recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present invention, a computer-readable storage medium may be provided, on which a computer program is stored, which, when executed by one or more computing devices, causes the one or more computing devices to implement any of the methods disclosed in this application.

[0102] For example, when the computer program is executed by one or more computing devices, the one or more computing devices are caused to perform the following steps: when the structures of corresponding layers of the multiple neural networks are the same, the corresponding layers of the multiple neural networks are spliced ​​into a splicing layer, and the original input tensors of the corresponding layers of the multiple neural networks are spliced ​​into a tensor with one dimension higher than an original input tensor; the calculation on the corresponding layer in the multiple neural networks is completed by inputting the one-dimensional tensor into the splicing layer to obtain the output tensor of the splicing layer; when the structures of multiple corresponding layers of the multiple neural networks connected in sequence are respectively the same, the first corresponding layer of the multiple corresponding layers of the multiple neural networks connected in sequence is spliced ​​into a first splicing layer; the original input tensors of the first corresponding layer of the multiple neural networks are spliced ​​into a tensor with one dimension higher than the original input tensor; the calculation on the first corresponding layer in the multiple neural networks is completed by inputting the one-dimensional tensor into the first splicing layer to obtain the output tensor of the first splicing layer.

[0103] The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in computer devices such as a client, a host, an agent device, a server, etc. It should be noted that when the computer program is run, it can also be used to perform additional steps in addition to the above-mentioned steps or perform more specific processing when executing the above-mentioned steps. The contents of these additional steps and further processing have been mentioned in the description of the relevant method with reference to Figures 1 to 4, so they will not be repeated here to avoid repetition.

[0104] It should be noted that the method and computing device for performing calculations of multiple neural networks according to exemplary embodiments of the present invention can completely rely on the operation of a computer program to implement corresponding functions, wherein the various units of the device or system correspond to each step in the functional architecture of the computer program, so that the entire device or system is called through a special software package (for example, a lib library) to implement the corresponding functions.

[0105] On the other hand, when Figures 1 to Figure 3B When the various units or devices mentioned are implemented in software, firmware, middleware or microcode, the program code or code segments for performing the corresponding operations can be stored in a computer-readable storage medium such as a storage medium, so that a computing device (such as a processor) can perform the corresponding operations by reading and running the corresponding program code or code segments.

[0106] For example, according to an embodiment of the present invention, a system includes one or more computing devices and one or more storage devices, wherein the one or more storage devices store a computer program, and when the computer program is executed by the one or more computing devices, the one or more computing devices implement any method disclosed in this application. For example, the one or more computing devices are configured to perform the following steps: when the structures of a corresponding layer of the multiple neural networks are the same, the corresponding layers of the multiple neural networks are spliced ​​into a splicing layer, and the original input tensors of the corresponding layers of the multiple neural networks are spliced ​​into a tensor with one dimension higher than the original input tensor; the calculation on the corresponding layer in the multiple neural networks is completed by inputting the one-dimensional tensor into the splicing layer to obtain the output tensor of the splicing layer; when the structures of the multiple corresponding layers of the multiple neural networks connected in sequence are respectively the same, the first corresponding layer of the multiple corresponding layers of the multiple neural networks connected in sequence is spliced ​​into a first splicing layer; the original input tensors of the first corresponding layer of the multiple neural networks are spliced ​​into a tensor with one dimension higher than the original input tensor; the calculation on the first corresponding layer in the multiple neural networks is completed by inputting the one-dimensional tensor into the first splicing layer to obtain the output tensor of the first splicing layer.

[0107] Specifically, the computing device can be deployed in a server or on a neuron device in a distributed network environment. In addition, the computing device can also include a video display (such as a liquid crystal display) and a user interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device can be connected to each other via a bus and / or a network.

[0108] Here, the computing device is not necessarily a single device, but may be any collection of devices or circuits that can execute the above instructions (or instruction sets) individually or in combination. The computing device may also be part of an integrated control computing device or a computing device manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0109] The computing device for executing the training method of the neural network or the named entity recognition method according to the exemplary embodiment of the present invention can be a processor, and such a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor, a microcontroller or a microprocessor. As an example and not a limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc. The processor can run instructions or codes stored in one of the storage devices, wherein the storage device can also store data. Instructions and data can also be sent and received over the network via a network interface device, wherein the network interface device can use any known transmission protocol.

[0110] The storage device may be integrated with the processor, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. Furthermore, the storage device may comprise a separate device, such as an external disk drive, a storage array, or any other storage device accessible to a database computing device. The storage device and processor may be operatively coupled or may be in communication with each other, for example, via an I / O port, a network connection, or the like, such that the processor can access files stored on the storage device.

[0111] It should be noted that the exemplary implementation of the present invention focuses on solving the current problems of slow computer response, high computational overhead, and low computational efficiency when calculating corresponding layers of the same structure for multiple neural networks. Specifically, in order to solve the above problems, the present invention adopts a technical solution of splicing corresponding layers of the same structure for multiple neural networks, splicing the original input tensors of the corresponding layers into a tensor with one dimension higher than the original input, and using the one-dimensional tensor and the spliced ​​corresponding layers to perform operations. Since a large number of matrix operations corresponding to the corresponding layers are converted into a smaller number of batch matrix operations corresponding to the spliced ​​corresponding layers, the operation startup speed is accelerated while ensuring the equivalence of the operations, the computational overhead is reduced, and the computational efficiency is improved.

[0112] While the exemplary embodiments of the present application have been described above, it should be understood that the above description is merely illustrative and not exhaustive, and the present application is not limited to the disclosed exemplary embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the present application. Therefore, the scope of protection of the present application should be determined by the scope of the claims.

Claims

1. A method for performing computations of a plurality of neural networks by at least one computer, wherein: The method comprises the following steps performed by the at least one computer: receiving a set of inputs, wherein the set of inputs comprises one of an image input, a text input, and a voice input; computing, by the plurality of neural networks, input tensors for layers of the plurality of neural networks based on the received set of inputs; When the structures of corresponding layers of the multiple neural networks are the same, the corresponding layers of the multiple neural networks are spliced ​​into a spliced ​​layer, and the original input tensors of the corresponding layers of the multiple neural networks are spliced ​​into a tensor with one dimension higher than the original input tensor; the calculation of the corresponding layers in the multiple neural networks is completed by inputting the one-dimensional tensor into the spliced ​​layer to obtain the output tensor of the spliced ​​layer, wherein the original input tensor of the corresponding layer indicates the input tensor of the corresponding layer before splicing; When the structures of the plurality of corresponding layers connected sequentially in the plurality of neural networks are respectively the same, the first corresponding layer of the plurality of corresponding layers connected sequentially in the plurality of neural networks is spliced ​​into a first splicing layer; the original input tensors of the first corresponding layers of the plurality of neural networks are spliced ​​into a tensor with one dimension higher than the original input tensor; the calculation of the first corresponding layer in the plurality of neural networks is completed by inputting the one-dimensional tensor into the first splicing layer to obtain the output tensor of the first splicing layer, wherein the original input tensor of the first corresponding layer indicates the input tensor of the first corresponding layer before splicing; performing one of image processing, text recognition, and speech recognition corresponding to the set of inputs based on calculation results of the plurality of neural networks, When a tensor with one dimension higher than the original input tensor is input to a concatenated layer formed by concatenating the corresponding layers of the plurality of neural networks, the matrix operations of the respective corresponding layers of the plurality of neural networks before concatenation are converted into batch matrix operations. The batch matrix calculations on corresponding layers with the same structure in the multiple neural networks are completed by calling the underlying parallel matrix operation library of the at least one computer.

2. The method of claim 1, wherein when the structures of the plurality of corresponding layers of the plurality of neural networks connected sequentially are respectively identical, the method further comprises: For each subsequent corresponding layer after the first corresponding layer in the plurality of neural networks, concatenate the subsequent corresponding layer of each neural network into a second concatenated layer; The calculation of the subsequent corresponding layer in the multiple neural networks is completed by inputting the output result of the previous corresponding layer of the subsequent corresponding layer into the second splicing layer to obtain the output result of the second splicing layer.

3. The method of claim 2, further comprising: For the last corresponding layer among the multiple corresponding layers connected sequentially in the multiple neural networks, the output tensor of the concatenated layer formed by concatenating the last corresponding layers of each neural network is transformed into a tensor with one dimension lower than the output tensor itself.

4. The method of claim 1, further comprising: When structures of corresponding layers of the multiple neural networks are the same, an output tensor of the concatenated layer is deformed into a tensor with one dimension lower than that of the output tensor itself.

5. The method according to claim 1, wherein The corresponding layer is any one of the following: Fully connected layer; Batch normalization layer; Activation function layer; Layer by element; Class decomposition machine layer.

6. The method according to claim 5, wherein: When the corresponding layers are fully connected layers, the concatenated layers are locally connected layers.

7. The method of claim 1, wherein: The parallel matrix operation library includes at least one of MKL, OpenBlas, CUDA, and cublas.

8. The method of claim 7, wherein: When the corresponding layer is a fully connected layer and the spliced ​​layer is a locally connected layer, the matrix multiplication in the locally connected layer is accelerated using a batch matrix multiplication function of the matrix operation library, wherein the batch matrix multiplication function includes at least one of the following functions: cblas_dgemm_batch function of the Mkl library; cublasDgemmStridedBatched function of the cublas library.

9. The method of claim 1, wherein: The multiple neural networks are homogeneous neural networks in deep neural networks.

10. A computing device for performing computations on a plurality of neural networks, wherein: The computing device comprises: A receiving unit, receiving a set of inputs, wherein the set of inputs includes one of image input, text input, and voice input; an input tensor calculation unit, which calculates input tensors of layers of the plurality of neural networks through the plurality of neural networks based on the received set of inputs; The network splicing unit is configured to: when the structures of corresponding layers of the multiple neural networks are the same, splice the corresponding layers of the multiple neural networks into a spliced ​​layer; when the structures of the multiple corresponding layers of the multiple neural networks connected in sequence are respectively the same, splice the first corresponding layer of the multiple corresponding layers of the multiple neural networks connected in sequence into a first spliced ​​layer; The input splicing unit is configured to: when the structures of corresponding layers of the multiple neural networks are the same, splice original input tensors of the corresponding layers of the multiple neural networks into a tensor with one dimension higher than the original input tensor; when the structures of multiple corresponding layers of the multiple neural networks connected sequentially are respectively the same, splice original input tensors of the first corresponding layer of the multiple neural networks into a tensor with one dimension higher than the original input tensor, wherein the original input tensor of the corresponding layer indicates the input tensor of the corresponding layer before splicing; The computing unit is configured to: when the structures of corresponding layers of the multiple neural networks are the same, complete the computation on the corresponding layer in the multiple neural networks by inputting the higher-dimensional tensor into the concatenated layer to obtain the output tensor of the concatenated layer; when the structures of the multiple corresponding layers of the multiple neural networks connected sequentially are respectively the same, complete the computation on the first corresponding layer in the multiple neural networks by inputting the higher-dimensional tensor into the first concatenated layer to obtain the output tensor of the first concatenated layer, where the original input tensor of the first corresponding layer indicates the input tensor of the first corresponding layer before concatenation; performing one of image processing, text recognition, and speech recognition corresponding to the set of inputs based on calculation results of the plurality of neural networks, When a tensor with one dimension higher than the original input tensor is input to a concatenated layer formed by concatenating the corresponding layers of the plurality of neural networks, the matrix operations of the respective corresponding layers of the plurality of neural networks before concatenation are converted into batch matrix operations. The computing unit is configured to complete batch matrix calculations on corresponding layers with the same structure in the multiple neural networks by calling the underlying parallel matrix operation library.

11. The computing device of claim 10, wherein: When the structures of the multiple corresponding layers connected sequentially in the multiple neural networks are respectively the same, the network splicing unit is configured to splice each subsequent corresponding layer after the first corresponding layer in the multiple neural networks into a second splicing layer; the calculation unit is configured to complete the calculation of the subsequent corresponding layer in the multiple neural networks by inputting the output result of the previous corresponding layer of the subsequent corresponding layer into the second splicing layer to obtain the output result of the second splicing layer.

12. The computing device of claim 11, further comprising: The output splicing unit is configured to transform the output tensor of the spliced ​​layer formed by splicing the last corresponding layer of the multiple corresponding layers connected sequentially in the multiple neural networks into a tensor with one dimension lower than the output tensor itself.

13. The computing device of claim 11, wherein: The computing device further includes: The output splicing unit is configured to, when the structures of corresponding layers of the multiple neural networks are the same, transform the output tensor of the splicing layer into a tensor with one dimension lower than the output tensor itself.

14. The computing device of claim 10, wherein: The corresponding layer is any one of the following: Fully connected layer; Batch normalization layer; Activation function layer; Layer by element; Class decomposition machine layer.

15. The computing device of claim 14, wherein: When the corresponding layers are fully connected layers, the concatenated layers are locally connected layers.

16. The computing device of claim 10, wherein: The parallel matrix operation library includes at least one of MKL, OpenBlas, CUDA, and cublas.

17. The computing device of claim 16, wherein: When the corresponding layer is a fully connected layer and the spliced ​​layer is a locally connected layer, the matrix multiplication in the locally connected layer is accelerated using a batch matrix multiplication function of the matrix operation library, wherein the batch matrix multiplication function includes at least one of the following functions: cblas_dgemm_batch function of the Mkl library; cublasDgemmStridedBatched function of the cublas library.

18. The computing device of claim 10, wherein: The multiple neural networks are homogeneous neural networks in deep neural networks.

19. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by one or more computing devices, the one or more computing devices are caused to implement the method according to any one of claims 1 to 9.

20. A system comprising one or more computing devices and one or more storage devices, wherein the one or more storage devices have a computer program recorded thereon, and when the computer program is executed by the one or more computing devices, the one or more computing devices are enabled to implement the method according to any one of claims 1 to 9.

Images