Transform network model structure improvement method and related equipment

Abstract

The embodiment of the invention provides an improvement method suitable for large model deployment and aiming at a Transform network structure and related equipment, so that the large model is easier to deploy during training and reasoning, and the training convergence speed is higher. The method provided by the embodiment of the invention comprises the following steps: replacing a full connection layer in a Transform module based on a time sequence dilated convolution block, determining an initial configuration parameter of the full connection layer in the Transform module, and determining a target configuration parameter of the time sequence dilated convolution block according to the initial configuration parameter; configuring convolution kernel sizes and void configuration parameters of all convolution layers in the time sequence void convolution block based on a user configuration file; obtaining the number of layers of the time sequence dilated convolution block, so that the receiving domain of the first layer corresponding to each output of the last layer of the time sequence dilated convolution block can cover the whole time sequence length of the input sequence; and setting the number of input channels and the number of output channels of all convolution layers in the time sequence cavity convolution block according to the model parameter value and the dimension of the middle layer of the original full connection layer.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of model training and reasoning, and in particular to a method for improving the structure of a Transformer network model and related equipment. Background Art

[0002] At present, artificial intelligence has achieved remarkable results in many fields, such as natural language processing, image recognition, speech recognition and generation, video processing, and the comprehensive application of multimodality. The success of these applications is usually based on a pre-trained base model, and the basic building blocks of this pre-trained large model are all based on the Transformer architecture. For example, the representation model (BERT, Bidirectional Encoder Representations from Transformers) uses the Transformer encoder part, while the GPT large model uses the Transformer decoder part. One of the biggest advantages of the Transformer architecture is that the model capacity can be increased simply by increasing the model structure parameter settings, thereby improving the generalization ability of the model. However, as the model becomes larger and larger, some of the original designs in the Transformer architecture have brought some difficult-to-solve problems to the training or reasoning deployment of large models. Summary of the invention

[0003] The embodiments of the present application provide a method for improving the structure of a Transformer network model and related equipment, which are used to make large models easier to deploy and more efficient during training and reasoning.

[0004] In a first aspect, an embodiment of the present application provides a method for improving a Transformer network model structure, which is applied to a computer device, wherein the computer device is deployed with a large model, and the large model is composed of a Transformer module. The method includes:

[0005] Based on the time series hole convolution block, the fully connected layer in the Transformer module is replaced, the initial configuration parameters corresponding to the fully connected layer in the Transformer module are determined, and the target configuration parameters corresponding to the time series hole convolution block are determined according to the initial configuration parameters; wherein the time series hole convolution block is used to replace the original fully connected layer in the Transformer module, and the target configuration parameters include model parameter values, time series length values, and original intermediate layer dimensions of the fully connected layer. The time series length value is used to characterize the time series length of the maximum input processed by the large model, and the model parameter value is used to characterize the vector dimension size of the Transformer module in the large model. The original intermediate layer dimension of the fully connected layer is used to express the vector dimension of the fully connected layer of the Transformer module in the large model after linear transformation and expansion;

[0006] Configuring the convolution kernel size and hole configuration parameters of all convolution layers in the temporal hole convolution block based on the user configuration file;

[0007] According to the temporal length value, the convolution kernel size and the hole configuration parameter, the number of temporal hole convolution block layers corresponding to the temporal hole convolution block is obtained, so that the receiving field of the first layer corresponding to each output of the last layer of the temporal hole convolution block can cover the temporal length value;

[0008] According to the model parameter value and the dimension of the original intermediate layer of the fully connected layer, the number of input channels and the number of output channels of all the convolutional layers in the temporal hole convolution block are set.

[0009] Optionally, configuring the hole configuration parameters of all convolutional layers in the temporal hole convolution block based on the user configuration file includes:

[0010] If the actual timing length value is greater than the timing length threshold configured in the user configuration file, the number of all convolutional layers in the timing hole convolution block and the convolutional layer numbers corresponding to the number of all convolutional layers are determined according to the timing length configured in the user configuration file; if the actual timing length value is not greater than the timing length threshold configured in the user configuration file, the number of all convolutional layers in the timing hole convolution block and the convolutional layer numbers corresponding to the number of all convolutional layers are determined according to the actual timing length value;

[0011] Based on a bottom-to-top exponential growth algorithm, the hole configuration parameters of all the convolutional layers in the temporal hole convolution block are set according to the convolutional layer number.

[0012] Optionally, setting the hole configuration parameters of all the convolution layers in the temporal hole convolution block according to the convolution layer number includes:

[0013] Determine a hole reset parameter corresponding to all the temporal hole convolution blocks in the large model; wherein the hole reset parameter is used to select the hole configuration parameter in all the temporal hole convolution blocks of the large model.

[0014] Optionally, the method further comprises:

[0015] Determine the residual connection blocks in all the temporal hole convolution blocks;

[0016] According to the residual connection block, a residual connection is established between an input channel located at a first layer of the temporal atrous convolution block and an output channel located at a last layer of the temporal atrous convolution block.

[0017] Optionally, setting the number of input channels and the number of output channels of all convolutional layers in the temporal atrous convolution block according to the model parameter value and the dimension of the original intermediate layer of the fully connected layer includes:

[0018] The number of input channels and the number of output channels in any temporal atrous convolution block are set to the number corresponding to the model parameter value, the first convolution layer of the temporal atrous convolution block uses each input vector dimension as an input channel, and the number of output channels of the first convolution layer is the original middle layer dimension of the fully connected layer; the last convolution layer uses the original middle layer dimension of the fully connected layer as the number of input channels, and the number of output channels is the vector dimension corresponding to the model parameter value; the original middle layer dimension of the fully connected layer is used as the number of input channels and the number of output channels of other convolution layers except the first convolution layer and the last convolution layer.

[0019] Optionally, the method further comprises:

[0020] Based on the convolution operation conditions, the input data of the temporal hole convolution block whose ranking order is the target ranking in the large model after channel connection is transposed, and the temporal dimension of the input data is adjusted to the target dimension.

[0021] A second aspect of an embodiment of the present application provides a method for improving a Transformer network model structure, which is applied to a computer device, wherein the computer device is deployed with a large model, and the large model is composed of a Transformer module. The method includes:

[0022] Based on the multiple cut fully connected layers, the original single fully connected layer in the Transformer module is replaced, the initial configuration parameters corresponding to the original fully connected layer in the Transformer module are determined, and the target configuration parameters corresponding to the multiple cut fully connected layers are determined according to the initial configuration parameters; wherein the target configuration parameters include model parameter values, the original fully connected layer intermediate layer dimensions and the original model configuration parameter header values, the model parameter values ​​are used to characterize the vector dimension size of the Transformer module in the large model, and the original fully connected layer intermediate layer dimensions are used to express the Transformer in the large model. The intermediate vector dimension after linear transformation and expansion of the vector dimension in the original fully connected layer of the transformer module, the original model configuration parameter header value is used to express the number of heads of the transformer module in the large model; the intermediate layer dimension of the original fully connected layer and the model configuration parameter value and the original model configuration parameter header value are used to express the number of heads of the transformer module in the large model and the number of fully connected layers of the cut fully connected layer. The model configuration parameter value is used to characterize the configuration parameters of the large model, and the original model configuration parameter header value is the number of the cut fully connected layer obtained after the original fully connected layer is cut;

[0023] Splitting the number of input channels of the first fully connected layer in the cut fully connected layer of the Transformer module into the ratio of the model parameter value to the value of the original model configuration parameter header, and determining the output of the first fully connected layer as the ratio of the model parameter value to the number of fully connected layers;

[0024] According to the ratio of the model parameter value to the number of fully connected layers and the value of the original model configuration parameter header, a change is made in the middle layer of the original fully connected layer to obtain a plurality of intermediate output sub-channels corresponding to the middle layer;

[0025] All intermediate output sub-channels are concatenated to obtain an intermediate output channel corresponding to the intermediate layer; wherein the intermediate output channel is associated with the model parameter value;

[0026] The output of the intermediate output channel is connected to the input channel of the fully connected layer located at the last layer in the cut fully connected layer, so that the output channel of the target fully connected layer located at the last layer is associated with the model parameter value.

[0027] A third aspect of the embodiment of the present application provides an improved system for a Transformer network model structure, including:

[0028] A determination unit, used to replace the fully connected layer in the Transformer module based on the time series hole convolution block, determine the initial configuration parameters corresponding to the fully connected layer in the Transformer module, and determine the target configuration parameters corresponding to the time series hole convolution block according to the initial configuration parameters; wherein the time series hole convolution block is used to replace the original fully connected layer in the Transformer module, and the target configuration parameters include model parameter values, time series length values, and original fully connected layer intermediate layer dimensions, the time series length value is used to characterize the maximum time series length processed by the large model, the model parameter value is used to characterize the vector dimension size of the large model, and the original fully connected layer intermediate layer dimension is used to express the vector dimension of the large model during the linear transformation of the vector dimension;

[0029] A configuration unit, configured to configure the convolution kernel size and hole configuration parameters of all convolution layers in the temporal hole convolution block based on a user configuration file;

[0030] An acquisition unit, configured to obtain the number of layers of a time series dilated convolution block corresponding to the time series dilated convolution block according to the time series length value, the convolution kernel size, and the dilated configuration parameter, so that the receiving field of the first layer corresponding to each output of the last layer of the time series dilated convolution block can cover the time series length value;

[0031] A setting unit is used to set the number of input channels and the number of output channels of all convolutional layers in the temporal hole convolution block according to the model parameter value and the dimension of the original intermediate layer of the fully connected layer.

[0032] The Transformer network model structure improvement system provided in the third aspect of the embodiment of the present application is used to execute the Transformer network model structure improvement method described in the first aspect.

[0033] A fourth aspect of the embodiments of the present application provides a system for improving a Transformer network model structure, including:

[0034] A determination unit is used to replace the original single fully connected layer in the Transformer module based on the multiple cut fully connected layers, determine the initial configuration parameters corresponding to the original fully connected layer in the Transformer module, and determine the target configuration parameters corresponding to the multiple cut fully connected layers according to the initial configuration parameters; wherein the target configuration parameters include model parameter values, the original fully connected layer intermediate layer dimensions and the original model configuration parameter header values, the model parameter values ​​are used to characterize the vector dimension size of the Transformer module in the large model, and the original fully connected layer intermediate layer dimensions are used to express the Tr in the large model The intermediate vector dimension after linear transformation and expansion of the vector dimension in the original fully connected layer of the ansformer module, the original model configuration parameter header value is used to express the number of heads of the Transformer module in the large model; the intermediate layer dimension of the original fully connected layer and the model configuration parameter value and the original model configuration parameter header value are used to express the number of heads of the Transformer module in the large model and the number of fully connected layers of the cut fully connected layer. The model configuration parameter value is used to characterize the configuration parameters of the large model, and the original model configuration parameter header value is the number of the cut fully connected layer obtained after the original fully connected layer is cut;

[0035] A splitting unit, used for splitting the number of input dimensions of the fully connected layer located at the first layer in the cut fully connected layer of the Transformer module into a ratio of the model parameter value to the value of the original model configuration parameter header, and determining that the output of the fully connected layer located at the first layer is the ratio of the model parameter value to the number of the fully connected layer;

[0036] An acquisition unit, configured to change the middle layer of the original fully connected layer according to the ratio of the model parameter value to the number of fully connected layers and the value of the original model configuration parameter header, to obtain a plurality of middle output sub-connection layers corresponding to the middle layer;

[0037] The acquisition unit is further used to splice all the intermediate output sub-connection layers to obtain an intermediate connection layer corresponding to the intermediate layer; wherein the intermediate connection layer is associated with the model parameter value;

[0038] A connection unit is used to connect the output of the intermediate connection layer with the input dimension of the fully connected layer located at the last layer in the cut fully connected layer, so that the output dimension of the target fully connected layer located at the last layer is associated with the model parameter value.

[0039] The Transformer network model structure improvement system provided in the fourth aspect of the embodiment of the present application is used to execute the Transformer network model structure improvement method described in the second aspect.

[0040] A fifth aspect of the embodiments of the present application provides a device for improving the Transformer network model structure, including:

[0041] CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply;

[0042] The memory is a short-term storage memory or a persistent storage memory;

[0043] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the method for improving the Transformer network model structure described in the first aspect or the second aspect.

[0044] The sixth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a computer, the computer executes the method for improving the Transformer network model structure described in the first aspect or the second aspect.

[0045] A seventh aspect of an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer, the computer executes the method for improving the Transformer network model structure described in the first aspect or the second aspect.

[0046] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: through the method for improving the Transformer network model structure disclosed in the embodiments of the present application, the fully connected layer in the existing Transformer network structure, which is not conducive to the training and reasoning deployment of large models, is improved, and the convolution layer of the temporal hole convolution block is used instead, so that the large model is easier to deploy during training and reasoning. At the same time, the computing hardware requirements for model deployment are reduced, the efficiency is higher, and the training accuracy of the large model is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0048] Figure 1 This is a design diagram of the fully connected layer of a Transformer structure of an existing solution;

[0049] Figure 2 A structural design diagram of a large model disclosed in an embodiment of the present application;

[0050] Figure 3 A flowchart of a method for improving a Transformer network model structure disclosed in an embodiment of the present application;

[0051] Figure 4 A flowchart of another method for improving the Transformer network model structure disclosed in an embodiment of the present application;

[0052] Figure 5 A flowchart of another method for improving the Transformer network model structure disclosed in an embodiment of the present application;

[0053] Figure 6 A flowchart of a method for improving a Transformer network model structure disclosed in an embodiment of the present application;

[0054] Figure 7 A performance comparison diagram of an existing solution disclosed in an embodiment of the present application and the technical solution of the present application;

[0055] Figure 8 A comparison chart of the training accuracy and convergence speed of an existing solution disclosed in an embodiment of the present application and the technical solution of the present application;

[0056] Fig. 9 A comparison chart of the actual translation effects of an existing solution disclosed in the embodiment of the present application and the technical solution of the present application;

[0057] Fig.10 A structural design diagram of another large model disclosed in an embodiment of the present application;

[0058] Fig.11 A comparison chart of operation data between an existing solution disclosed in an embodiment of the present application and the technical solution of the present application;

[0059] Fig.12 A comparison chart of the training accuracy and convergence speed of another existing solution disclosed in the embodiment of the present application and the technical solution of the present application;

[0060] Fig.13 A comparison chart of the actual translation effects of another existing solution disclosed in the embodiment of the present application and the technical solution of the present application;

[0061] Fig.14 A schematic diagram of the structure of an improved system for a Transformer network model structure disclosed in an embodiment of the present application;

[0062] Fig.15 A schematic diagram of the structure of an improved system for a Transformer network model structure disclosed in an embodiment of the present application;

[0063] Fig.16 A structural schematic diagram of a device for improving a Transformer network model structure disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0065] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0066] As described in the above background technology, as the model size becomes larger and larger, it becomes increasingly difficult to efficiently train and reason about the Transformer model. As the model becomes larger and larger, it is impossible to accommodate the entire model on a single GPU card or even all GPU cards on a single machine. Therefore, existing large model training or reasoning must be completed through various distributed parallel algorithms (such as data parallelism, model parallelism, etc.):

[0067] For a single large model, model parallelism is usually used to perform calculations on multiple GPUs across computing nodes. Common model parallelism includes pipeline parallelism and tensor parallelism. In order to reduce the total training and inference time, data parallelism is used, that is, the training data is evenly divided into several data subsets, and multiple large model instances (each instance uses the above model parallelism) are run to complete the training on these data subsets. However, in the Transformer structure, there are two connected huge fully connected layers at the end of each layer to complete two huge matrix multiplication operations. These two huge fully connected layers bring huge problems when using model parallelism to perform distributed training and inference of a single large model instance.

[0068] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0069] For a better understanding of the existing technical solutions, please refer to Figure 1 , Figure 1 This is a design diagram of the fully connected layer of a Transformer structure of an existing solution. Figure 1 The figure specifically shows the design and implementation of the original fully connected layer in Transformer. Specifically, in the prior art, the input channel of the large model is single, where the input is the model parameter value d_model. Therefore, in the middle layer (the blue diamond block can be understood as the model position where the fully connected layer is located), the model parameter value d_model is configured by configuring the parameter value d_ff, and matrix multiplication operations are performed in two fully connected layers, so as to output in a single output channel of the large model, thereby obtaining output data. Under normal conditions, d_ff is generally four times d_model, and its output is generally d_model. Therefore, combined with the above description, the fully connected layer takes up huge memory resources. Therefore, when the pipeline model is used in parallel, the number of Transformer sub-layers that can be accommodated by a single GPU card is limited, resulting in too many steps (stages) of the entire pipeline division, thereby increasing the window waiting period (bubble time) of each step of the pipeline, resulting in low GPU utilization. At the same time, the two adjacent fully connected layers perform matrix multiplication operations. Therefore, when tensor parallelism is used, the matrix multiplication process will require the exchange of a large number of intermediate calculation results between multiple GPUs, causing the GPU to fall into a waiting state due to waiting for data transmission, which will also lead to low GPU utilization.

[0070] To solve the above technical problems, please refer to Figure 3 , Figure 3 The flowchart of a method for improving the Transformer network model structure disclosed in an embodiment of the present application includes steps 301 to 303.

[0071] 301. Replace the fully connected layer in the Transformer module based on the temporal atrous convolution block, determine the initial configuration parameters corresponding to the fully connected layer in the Transformer module, and determine the target configuration parameters corresponding to the temporal atrous convolution block based on the initial configuration parameters.

[0072] Different from the existing solutions, multiple time-series hole convolution blocks are used to replace the original two fully connected layers. Specifically, in this embodiment, it is necessary to first determine the initial configuration parameters corresponding to the fully connected layer in the Transformer module, and then determine the target configuration parameters corresponding to the time-series hole convolution block based on the initial configuration parameters. It should be noted that the time-series hole convolution block is used to replace the original fully connected layer in the Transformer module. The target configuration parameters include model parameter values, time-series length values, and the original fully connected layer intermediate layer dimensions. The time-series length value is used to characterize the time-series length of the maximum input processed by the large model, and the model parameter value is used to characterize the vector dimension size of the Transformer module in the large model. The original fully connected layer intermediate layer dimension is used to express the vector dimension of the fully connected layer of the Transformer module in the large model after linear transformation and expansion.

[0073] In one specific embodiment, it can be combined with Figure 2 , Figure 2 This is a structural design diagram of a large model disclosed in the embodiment of this application. It should be noted that in the original Transformer architecture (i.e. Figure 1 ), the input of the fully connected layer is (number of samples n_batch, sequence length n_step, model size d_model). The entire model is processed in the d_model dimension. In this embodiment, combined with Figure 2 , replace the original two fully connected layers with time-series dilated convolution, thereby increasing the processing of the sequence length n_step dimension, and using convolution operations to learn the data features of this dimension. Among them, the number of samples n_batch can be understood as the number of input training samples (such as the number of long and short sentences, articles or sentences, etc.). The sequence length n_step can be understood as the number of words in a certain sample, that is, Figure 2 The corresponding x1,x2,...,x T-1 ,x T, where T is equivalent to the sequence length, which can be understood as the number of words in a sentence, or the number of words. The model size d_model is the model parameter value described above. For the convenience of understanding and description, it will not be described in detail later. It should also be noted that there are multiple convolutional layers in the temporal dilated convolution block (TCN, Temporal convolution Network). When processing the time sequence t, each convolutional layer only considers the input before the time sequence t, and does not consider the input after the time sequence t. Furthermore, this embodiment uses a one-dimensional temporal dilated convolution block to process one-dimensional data such as a language model, and uses a two-dimensional convolution to process two-dimensional data such as images. Furthermore, creating a temporal dilated convolution block requires the following parameters, and these parameter settings must meet the restrictions of the temporal dilated convolution block. Specifically, it is necessary to determine parameters such as the model parameter value, the time sequence length value, or the dimensions of the original fully connected layer intermediate layer (for example Figure 1 The vector dimension when transforming d_ff in, etc.).

[0074] It can also be understood that, in this embodiment, the parameters of all fully connected layers can be replaced by TCNs with the same configuration.

[0075] 302. Configure the convolution kernel size and dilation configuration parameters of all convolution layers in the temporal dilated convolution block based on the user configuration file.

[0076] Then, through the configuration file pre-configured by the user, the convolution kernel size and hole configuration parameters of all convolution layers in the temporal hole convolution block can be configured. It should be noted that the user configuration file can be individually designed according to the user's design requirements for large models, and this embodiment does not limit the specific convolution kernel size and control configuration parameters.

[0077] In one of the specific embodiments, the size of the convolution kernel in the time-series hole convolution block is first determined. Determine the maximum time length that the model can handle, and at the same time, it is also necessary to determine the size of d_model in the large model. Specifically, for the large model, the value of the convolution kernel size (convolution kernel value) can be defined as 2 or 3, which is used in all time-series hole convolution blocks. The time length is usually specified by configuration. In an achievable technical solution, the time length usually refers to n_step. For example, it is 1024 for GPT2, 3072 for GPT3, and so on.

[0078] Furthermore, multi-layer temporal convolutional layers often use an exponential growth method from bottom to top to specify the hole configuration parameters.

[0079] 303. According to the temporal length value, the convolution kernel size and the hole configuration parameter, the number of temporal atrous convolution block layers corresponding to the temporal atrous convolution block is obtained, so that the receiving field of the first layer corresponding to each output of the last layer of the temporal atrous convolution block can cover the temporal length value.

[0080] Then, according to the temporal length value, the convolution kernel size and the hole configuration parameters, the number of temporal atrous convolution block layers corresponding to the temporal atrous convolution block can be obtained, so that the receptive field of the first layer corresponding to each output of the last layer of the temporal atrous convolution block can cover the temporal length value.

[0081] In one specific embodiment, the total number of time-series hole convolution blocks required to cover the maximum time-series length specified in the configuration file can be calculated based on the maximum time-series length. Specifically, the calculation formula can be used. Among them, ω is the time series length, that is, the time series length value described above. k is the convolution kernel value, that is, the convolution kernel size. b is the basis of the exponential function (usually set in advance, that is, the hole configuration parameter), and n is the number of layers (that is, the number of layers of the time series hole convolution block). Combined with Figure 2 As shown, according to the time length value, the convolution kernel value and the basis of the corresponding exponential function, the corresponding n can be obtained, thereby determining the required number of time series empty convolution block layers. For example, when n = 3 (starting from 0), refer to Figure 2 It can be seen that each sequence length n_step requires a convolution operation, so the number of time-series hole convolution blocks at this time is 4. It should be noted that Figure 2 In the example, when n=0, since the number of words corresponding to each sequence length n_step needs to be convolved, that is, the hole parameter d=1, the convolution layer can be the size of the corresponding sequence length n_step. When n=1, the hole parameter d=2 is required, and the number can be understood as half the value of the sequence length n_step. So when n=3, the convolution layer can be Output the corresponding value in the output channel.

[0082] 304. According to the model parameter values ​​and the dimensions of the original fully connected layer intermediate layers, the number of input channels and the number of output channels of all convolutional layers in the temporal atrous convolution block are set.

[0083] Then, the number of input channels and output channels of all convolutional layers in the dilated convolution block can be set according to the model parameter values ​​and the dimensions of the original fully connected layer intermediate layers, and the output channel of any dilated convolution block can be connected to the input channel of the next dilated convolution block of any dilated convolution block, thereby completing the improvement of the Transformer network model structure.

[0084] In one of the specific embodiments, the number of input channels and the number of output channels of all temporal hole convolution blocks can be specified. Specifically, in this embodiment, the number of input channels and the number of output channels of all temporal hole convolution blocks can be specified as the size of the model parameter value d_model. That is, the input of the original fully connected layer is transposed so that d_model is transposed to the input channel dimension of the convolution layer and then input to the first layer of the temporal hole convolution block. Thereafter, the input channels and input channels of all layers except the last layer in the hole convolution block are set to the original middle layer dimension of the fully connected layer. The number of input channels of the last layer of the temporal hole convolution module is also set to the original middle layer dimension of the fully connected layer, and its output channel is set to the original middle layer dimension of the fully connected layer. The output of the last layer of the temporal hole convolution module is transposed again after the residual connection block operation, so that its dimensional arrangement is consistent with the dimensional arrangement of the fully connected layer in the original model.

[0085] The method for improving the Transformer network model structure disclosed in this embodiment improves the fully connected layer in the existing Transformer network structure, which is not conducive to the training and reasoning deployment of large models, and uses the convolution layer of the temporal hole convolution block instead, so that the large model is easier to deploy during training and reasoning. At the same time, the computing hardware requirements for model deployment are reduced, the efficiency is higher, and the training accuracy of the large model is improved.

[0086] When deploying a large model, you need to control the length of the time series in the model. Figure 4 , Figure 4 Based on the disclosed embodiment of this application Figure 3 The flowchart of another method for improving the Transformer network model structure of the embodiment shown includes steps 401 to 405.

[0087] 401. When the actual timing length value is greater than the timing length threshold configured in the user configuration file, the number of all convolutional layers in the timing hole convolution block and the convolutional layer numbers corresponding to the number of all convolutional layers are determined according to the timing length configured in the user configuration file.

[0088] Based on the above Figure 3 As shown, after determining the maximum time series length that the model can process, if the time series length value at this time is greater than the time series length threshold, the time series length can be configured according to the user configuration file, and the number of all convolutional layers in the time series hole convolution block and the convolutional layer number corresponding to the number of all convolutional layers can be determined. Furthermore, when the actual time series length value is not greater than the time series length threshold configured in the user configuration file, the number of all convolutional layers in the time series hole convolution block and the convolutional layer number corresponding to the number of all convolutional layers can be determined according to the actual time series length value.

[0089] In this embodiment, please refer to Figure 2 As described above, Input is the input channel, Output is the output channel, and multiple Hidden are hidden layers. From bottom to top, the convolution layer number of each temporal convolution layer is specified. It should be noted that in this embodiment, the definition generally starts from 0.

[0090] 402. Based on the bottom-up exponential growth algorithm, set the hole configuration parameters of all convolution layers in the temporal hole convolution block according to the convolution layer number.

[0091] Therefore, based on step 401 and the bottom-to-top exponential growth algorithm, the hole configuration parameters of all the temporal convolutional layers in the temporal hole convolution block can be set according to the convolutional layer number.

[0092] In one specific embodiment, the multi-layer temporal convolutional layer can adopt an exponential growth method from bottom to top to formulate the convolutional layer dilation parameter. Where dilation = b n It should be noted that b is the basis of the exponential function (i.e., the hole configuration parameter), which is given by Figure 2 As can be seen from the description, the dilation configuration parameter is generally set to 2. n is the layer number of the convolution layer, which generally starts from 0. Therefore, the dilation values ​​in multiple consecutive sequential dilated convolution blocks are 1, 2, 4...2. n Correspondingly, the value of dilation can be understood as the hole parameter d described in the above step 302.

[0093] 403. According to the temporal length value, the convolution kernel size and the hole configuration parameter, the number of temporal atrous convolution block layers corresponding to the temporal atrous convolution block is obtained, so that the receiving field of the first layer corresponding to each output of the last layer of the temporal atrous convolution block can cover the temporal length value.

[0094] In this embodiment, step 403 is the same as the aforementioned Figure 3 Step 303 is similar, and will not be described in detail here. However, it should be noted that in this embodiment, since it is necessary to determine the corresponding number of layers according to the maximum time series length, the convolution kernel size, and the hole configuration parameters, the number of layers of the time series hole convolution block of the corresponding time series hole convolution block is determined, so that after setting various parameter values ​​of the large model, such as the convolution kernel size and the related time series length value, the number of layers of the time series hole convolution block of the time series hole convolution block can be obtained through calculation and analysis.

[0095] 404. Determine the hole reset parameters corresponding to all the temporal hole convolution blocks in the large model.

[0096] Based on step 403, the hole reset parameters corresponding to all the temporal hole convolution blocks in the large model can be determined. It should be noted that the hole reset parameters are used to select the hole configuration parameters in all the temporal hole convolution blocks of the large model.

[0097] Based on the above step 403, further, when the number of layers of the sequential dilated convolutional block is greater than the preset layer number threshold, the dilated reset parameter M corresponding to all the sequential dilated convolutional blocks in the large model can also be determined. For example, when N>M, the dilated parameter d is reset to 1 every m layers according to the standard TCN implementation, and then continues to grow exponentially. For example, the M corresponding to each convolutional layer is calculated from 0, and dilation=b is calculated. m Set b=2, and increase in exponential form. When N=20, then when M=7, M needs to be reset to 0, so that d is reset to 1. It should be noted that the hole reset parameter is used to select the hole configuration parameters in all the time series hole convolution blocks of the large model.

[0098] In one of the specific embodiments, when the calculated number of layers of the temporal atrous convolution block is greater than the number threshold of the number of layers of the temporal atrous convolution block specified in the configuration file, in other words, when the calculated number of layers of the temporal atrous convolution block is too large, it can be specified that the dilation is reset to 1 every N layers, that is, the dilation parameter of the next convolution layer is set to 1 after every N layers.

[0099] Furthermore, it can be stipulated that the total number of layers of the temporal hole convolution block does not exceed a preset value, and does not need to cover the entire input temporal length.

[0100] 405. Set the hole reset parameters of any two temporal hole convolution blocks to preset hole configuration parameter values.

[0101] Therefore, based on step 404, the hole configuration parameters of any two temporal hole convolution blocks can be set to the preset hole configuration parameter values.

[0102] In one specific embodiment, in combination Figure 2 As shown, after the hole configuration parameters of any temporal hole convolution block are adjusted, it is necessary to reset the hole parameters of the temporal hole convolution block of the layer to the preset hole parameter value, that is, as described in step 404, reset the dilation to 1 every N blocks, etc. There is no specific limitation here.

[0103] Furthermore, in this embodiment, step 405 may not be performed, and no specific limitation is given here.

[0104] Through the improved method of the Transformer network model structure disclosed in this embodiment, under the premise of ensuring the generalization ability of the original solution model, not only the computing hardware requirements for model deployment are greatly reduced, but also the convergence speed during model training is accelerated. The reasoning performance of the model finally trained is equivalent to that of the original model.

[0105] When deploying a large model, you need to specify the number of input channels and output channels of all temporal dilated convolution blocks and establish connections between the input channels and output channels. Figure 5 , Figure 5 This is a flow chart of another method for improving the Transformer network model structure disclosed in an embodiment of the present application, including steps 501 to 504.

[0106] 501. Set the number of input channels and the number of output channels in any temporal atrous convolution block to the number corresponding to the model parameter value, so as to connect the input channels and output channels in all temporal atrous convolution blocks one by one according to the number of the model parameter value.

[0107] It should be noted that Figure 5 The embodiment shown is mainly Figure 3 For the expanded description and detailed description of step 303. Specifically, in this embodiment, if the improvement of the Transformer network model structure is to be implemented, it is necessary to specify the input channels and output channels of all temporal hole convolution blocks, and determine the corresponding number of input channels and the number of output channels. Then, since the number of input channels and the number of output channels are in a corresponding relationship, any one of the number of input channels and the number of output channels can be input, and the corresponding numerical value can be set to the number of model parameter values, so that the input channels and output channels in all temporal hole convolution blocks are connected one by one according to the number of model parameter values.

[0108] In one specific embodiment, in this embodiment, the number of input channels and the number of output channels of all temporal dilated convolution blocks can be specified as the model parameter d_model size. Further, for the convenience of describing step 501 in detail, please refer to Figure 6 The specific embodiments are shown in the figure.

[0109] Furthermore, it is also necessary to transpose the input data of the time-series hole convolution block whose ranking order is the target ranking in the large model after the channel connection based on the convolution operation conditions, and adjust the time dimension of the input data to the target dimension. Specifically, according to the requirements of the convolution operation, the input of the first time-series hole convolution block is transposed, and the time dimension is adjusted to the appropriate dimension. There is no restriction on the specific dimension here. Generally, the dimension can be adjusted to the dimension under the conditions of 1024 for GPT2 and 3072 for GPT3 as described above.

[0110] Furthermore, after each time-series atrous convolutional network calculation is completed, the calculation results need to be transposed again to restore the original dimensional order.

[0111] 502. The first convolution layer of the time series void convolution block takes each input vector dimension as an input channel, and the number of output channels of the first convolution layer is the original middle layer dimension of the fully connected layer; the last convolution layer takes the original middle layer dimension of the fully connected layer as the number of input channels, and the number of output channels is the vector dimension corresponding to the model parameter value; the original middle layer dimension of the fully connected layer is used as the number of input channels and the number of output channels of other convolution layers except the first convolution layer and the last convolution layer.

[0112] In parallel with step 501, the first convolution layer of the time series atrous convolution block can also use each input vector dimension as an input channel, and the number of output channels of the first convolution layer is the original dimension of the middle layer of the fully connected layer; the last convolution layer uses the original dimension of the middle layer of the fully connected layer as the number of input channels, and the number of output channels is the vector dimension corresponding to the model parameter value; the original dimension of the middle layer of the fully connected layer is used as the number of input channels and the number of output channels of other convolution layers except the first convolution layer and the last convolution layer. It should be noted that the other layers of the time series atrous convolution block are all the time series atrous convolution blocks except the time series atrous convolution block of the first layer and the time series atrous convolution block of the last layer.

[0113] Furthermore, in addition to step 501 and step 502, there are other technical solutions that may exist or may appear, which can be referred to step 503-step 504.

[0114] 503. Determine the residual connection blocks in all temporal hole convolution blocks.

[0115] Specifically, it is necessary to first determine the residual connection blocks in all temporal atrous convolution blocks.

[0116] In one specific embodiment, all the temporal dilated convolution blocks in the temporal dilated convolution block have a residual connection block, thereby connecting the input and output of the temporal dilated convolution block. Therefore, it is necessary to determine the residual connection blocks in all the temporal dilated convolution blocks.

[0117] 504. According to the residual connection block, a residual connection is established between an input channel of the first layer of the temporal atrous convolution block and an output channel of the last layer of the temporal atrous convolution block.

[0118] Therefore, based on step 503, a residual connection between the input channel of the temporal atrous convolution block located in the first layer and the output channel of the temporal atrous convolution block located in the last layer can also be established according to the residual connection block.

[0119] The improved method of the Transformer network model structure disclosed in this embodiment uses multiple time-series hole convolution blocks to replace the huge connection layer in the original design. Specifically, the original fully connected layer of the Transformer is replaced with a time-series hole convolution block, thereby greatly reducing the difficulty of large model deployment and greatly improving the convergence speed during model training.

[0120] Therefore, combined with the above Figures 3 to 5 For further implementation, please refer to Figures 7 to 9 .in, Figure 7 A performance comparison diagram of an existing solution disclosed in an embodiment of the present application and the technical solution of the present application; Figure 8 A comparison chart of the training accuracy and convergence speed of an existing solution disclosed in an embodiment of the present application and the technical solution of the present application; Fig. 9 This is a comparison chart of the actual translation effect of an existing solution disclosed in the embodiment of the present application and the technical solution of the present application. In the embodiment of the present application, taking the hole cardinality b=2, convolution kernel kernel_size=3, time series length n_step=72, model parameter d_model=512, number of heads n_head=8, number of blocks N=6 in transformer as an example, the application of transformer in English-German translation in the multi30 dataset is compared as an example to compare various data of the patent solution and the original transformer solution. Among them, Figure 7 (a) in the figure represents the parameter size of the two fully connected layers in the original transformer design. Figure 7 (b) in the figure shows the parameter size of using 7 one-dimensional time series atrous convolution blocks to cover 72 time series inputs in solution 1. Figure 7 (a) and Figure 7As can be seen from (b) in the figure, although multiple time-series hole convolution blocks cannot be calculated in parallel, since the parameter size of solution 1 (the existing parameter size is 1208020992, and the parameter size of solution 1 is 1121792) and the amount of calculation (the existing amount of calculation is 1.21G, and the amount of calculation of solution 1 is 0) are basically negligible compared with the original solution, all time-series hole convolution blocks deployed on a graphics card will not bring computing pressure, thus greatly reducing the requirements for computing hardware when deploying the model. For more information, please refer to Figure 8 , Figure 8 The figure shows the comparison of the accuracy and number of training rounds of the original Transformer architecture and the improved architecture of this patent when training and validating on the multi30k dataset. Figure 8 It can be seen that the improved scheme of this solution not only has significantly smaller parameters than the original solution, but also does not have the fully connected layer in the original solution, which has extremely high requirements for memory and computing resources. In addition, since the new solution processes data not only in the token dimension but also in the time series dimension, the model converges very quickly, and only half the training time (4epochs) is used to achieve the accuracy of the original solution (8epochs). However, since the corpus and vocabulary of the multi30 dataset itself are relatively small, it cannot reflect the capabilities of our model. However, in the end, the training and verification accuracy of our solution is (0.82, 1.4418), which are better than the original solution (1.21, 1.4998). If the corpus is larger, Scheme 1 will have greater advantages over the original solution. Therefore, please refer to Fig. 9 , Fig. 9 The figure shows the performance comparison of the original Transformer architecture and the improved architecture of this solution when performing inference on the multi30k dataset. Fig. 9 It can be seen that the final reasoning effect of Scheme 1 is similar to that of the original scheme and is slightly better than that of the original scheme.

[0121] Therefore, the scheme 1 of the embodiment of the present application makes the memory distribution of the model more even, without large parameter and gradient memory blocks, making it easier to deploy when the model pipeline is parallel. The parallelism of the parallel computing of the model is improved, and there is no large matrix operation, so that the intermediate calculation results do not need to be transferred on multiple computing devices when the model tensor is parallel. The model training parameters are reduced, making the model smaller. The amount of calculation of the model is reduced, making the model training and reasoning more efficient. The training accuracy, convergence speed and model reasoning accuracy of the model are guaranteed.

[0122] With the above Figures 3 to 5 Correspondingly, the present application embodiment also discloses another solution design, please refer to Figure 6 , Figure 6The flowchart of a method for improving the Transformer network model structure disclosed in an embodiment of the present application includes steps 601 to 605.

[0123] 601. Replace an original single fully connected layer in the Transformer module based on the multiple cut fully connected layers, determine initial configuration parameters corresponding to the original fully connected layer in the Transformer module, and determine target configuration parameters corresponding to the multiple cut fully connected layers based on the initial configuration parameters.

[0124] It should be noted that steps 601 to 605 in this embodiment can be understood as an expansion scheme corresponding to the above-mentioned scheme 1, or can be understood as another separate scheme 2. Specifically, in this embodiment, it is necessary to replace the original single fully connected layer in the Transformer module based on the multiple cut fully connected layers, determine the initial configuration parameters corresponding to the original fully connected layer in the Transformer module, and determine the target configuration parameters corresponding to the multiple cut fully connected layers based on the initial configuration parameters. It should be noted that the target configuration parameters include model parameter values, original fully connected layer intermediate layer dimensions and original model configuration parameter header values. The model parameter values ​​are used to characterize the vector dimension size of the Transformer module in the large model. The original fully connected layer intermediate layer dimensions are used to express the vector dimension after linear transformation and expansion of the original fully connected layer of the Transformer module in the large model. The original model configuration parameter header values ​​are used to express the number of heads of the Transformer module in the large model. There is a correlation between the original fully connected layer intermediate layer dimensions and the model configuration parameter values ​​and the original model configuration parameter header values ​​used to express the number of heads of the Transformer module in the large model and the number of fully connected layers of the cut fully connected layers. The model configuration parameter values ​​are used to characterize the configuration parameters of the large model. The original model configuration parameter header values ​​are the number of cut fully connected layers obtained after cutting the original fully connected layers.

[0125] In one specific embodiment, the original fully connected layer in the Transformer module can be cut, so that an originally large fully connected layer is cut into multiple small fully connected layers, that is, the cut fully connected layer described above. Then determine the fully connected value n_head (that is, the number of attention heads in the fully connected layer, that is, the original model configuration parameter head value) of the fully connected layer in the large model, the model configuration parameter value d_ff (associated with the middle layer value and n_head), and at the same time, the corresponding model parameter value d_model needs to be determined. Furthermore, it is also necessary to define the two fully connected layers in the original fully connected layer as middle layers. It should also be noted that the model configuration parameter value d_ff at this time can be understood as the middle layer dimension of the original fully connected layer mentioned above. In short, in this embodiment, since a large fully connected layer is split into multiple small fully connected layers, the middle layer dimension of its original fully connected layer is evenly distributed to each small fully connected layer. Therefore, from the perspective of the entire large model, the configuration parameter value d_ff is associated with the middle layer dimension of the original fully connected layer.

[0126] For easier understanding, please refer to Fig.10 , Fig.10 This is a structural design diagram of another large model disclosed in the embodiment of this application. Fig.10 It can be seen that any two adjacent fully connected layers between the first layer and the last layer can be used as intermediate layers. It should be noted that in other feasible technical solutions, all other fully connected layers except the first layer and the last layer can also be used as intermediate layers.

[0127] 602. In the cut fully connected layer of the Transformer module, the number of input dimensions of the fully connected layer located at the first layer is split into the ratio of the model parameter value to the original model configuration parameter header value, and the output of the fully connected layer located at the first layer is determined to be the ratio of the model parameter value to the number of fully connected layers.

[0128] Based on step 601, the number of input dimensions of the first fully connected layer in the cut fully connected layer of the Transformer module can be split into the ratio of the model parameter value to the original model configuration parameter header value, and the output of the first fully connected layer can be determined as the ratio of the model parameter value to the number of fully connected layers.

[0129] In one specific embodiment, Figure 1 For comparison, Fig.10In , due to the use of n_head independent connection layers, multiple independent fully connected layers are first used to transform the original single input into multiple smaller independent inputs. Specifically, under the premise that the input dimension of the large model is d_model, the input d_model is placed in multiple n_head for separate processing, thereby transforming the input from d_model to d_model / n_head. In practical applications, the number of independent connection layers and the size after transformation can be flexibly set according to other model parameters. Specifically, in Figure 2 In the figure, each blue diamond represents a fully connected layer, which is the corresponding row in the matrix, and the convolutional layer can include multiple fully connected layers, which are the corresponding columns in the matrix.

[0130] 603. According to the ratio of the model parameter value to the number of fully connected layers and the value of the original model configuration parameter header, a change is made in the middle layer of the original fully connected layer to obtain a plurality of middle sub-connected layers corresponding to the middle layer.

[0131] Then, according to the ratio of the model parameter value to the number of fully connected layers and the value of the original model configuration parameter header, changes can be made to the middle layer of the original fully connected layer to obtain multiple intermediate sub-connected layers corresponding to the middle layer.

[0132] In one specific embodiment, two fully connected layers are connected to each other. Fig.10 The output of "1" in is transformed twice continuously. Specifically, the output of each fully connected layer in the first layer is changed. Fig.10 In , the dimension of the middle layer is defined as (d_model / n_head)×(d_ff / d_model), where (d_model / n_head) is the output size of the fully connected layer in "1" above, and (d_ff / d_model) is the magnification of the middle fully connected layer in the original Transformer design. In practical applications, the dimension of the middle layer can be flexibly set according to other parameters of the model. Furthermore, d_ff is the configuration parameter value described above. Furthermore, the change in the middle dimension can be understood as taking (d_model / n_head) as rows and (d_ff / d_model) as columns, and converting the large matrix into multiple small matrices by multiplying them. The output of multiple small matrices in the middle layer is thus obtained as

[0133] 604. All the intermediate sub-connection layers are spliced ​​together to obtain an intermediate connection layer corresponding to the intermediate layer.

[0134] Then, all the intermediate sub-connection layers based on step 603 are spliced ​​to obtain an intermediate connection layer corresponding to the intermediate layer.

[0135] In one specific embodiment, the n_head independent outputs d_model / n_head obtained by the above "2" are concatenated to obtain a unique output of size d_model, which is the corresponding value output by the intermediate connection layer of the intermediate layer described above.

[0136] 605. Connect the output of the intermediate connection layer to the input dimension of the fully connected layer located at the last layer in the cut fully connected layer, so that the output dimension of the target fully connected layer located at the last layer is associated with the model parameter value.

[0137] Then, the output of the intermediate connection layer is connected to the input dimension of the fully connected layer at the last layer in the cut fully connected layer, so that the output dimension of the target fully connected layer at the last layer is associated with the model parameter value.

[0138] In one specific embodiment, the concatenated output is used as input to the last fully connected layer, so that the input and output dimensions of the fully connected layer are both d_model.

[0139] Through the improved method of the Transformer network model structure disclosed in this embodiment, multiple fully connected layers can be used to replace the original two connection layers. In the new fully connected layer design, more and smaller matrix operations are used, and each small matrix operation can be executed in parallel. In practical applications, the number of layers of the new fully connected layer and the input and output dimension sizes of each layer can be flexibly configured according to other model parameters and applications.

[0140] For ease of understanding and description, please refer to Figures 11 to 13 . Fig.11 A comparison chart of the operation data of an existing solution disclosed in an embodiment of the present application and the technical solution of the present application. Fig.12 A comparison chart of the training accuracy and convergence speed of another existing solution disclosed in an embodiment of the present application and the technical solution of the present application.

[0141] Fig.13 This is a comparison chart of the actual translation effect of another existing solution disclosed in the embodiment of this application and the technical solution of this application. Fig.10 Taking the design as an example, we use the original parameter configuration of Transformer d_model = 512, n_head = 8, d_ff = 4×d_model = 4×512 = 2048, N = 6 to analyze the difference between our improved solution and the original solution.

[0142] In terms of the memory size occupied, the original solution occupies a memory size of 2×d_model×d_ff=2×d_model×4×d_model=8*512*512, which can be processed in two blocks of memory. Under the large model configuration, the size of these two blocks of memory will also increase, resulting in a limited number of steps that can be deployed on each GPU card in the parallel pipeline, which increases the number of GPU graphics cards required for the entire pipeline. It is impossible to fully utilize the resources of each GPU and the bubble time (bubble time) of the entire pipeline caused by waiting becomes longer, thereby reducing the efficiency of resource utilization and reducing the efficiency of model training and reasoning. The above solution (subsequently described as Solution 2). The first to fourth steps are Fig.10 , "1", "2", "3", and "4" in .

[0143] Step 1: Memory usage:

[0144] n_head*(d_model*(d_model / n_head))=d_model*d_model=512*512, a total of 8 independent memories.

[0145] Step 2: Memory usage:

[0146] n_head*2*((d_model / n_head)*(d_ff / d_model)*(d_model / n_head))=d_model*d_model, a total of 16 independent memories.

[0147] Step 3: No memory usage.

[0148] Step 4: The memory size occupied is d_model*d_model, 1 block of memory.

[0149] Therefore, the memory size occupied is 3*512*512, which is 3 / 8 of the original solution. Since the memory occupied is small, it is easier to parallelize the pipeline. Since we use smaller and more memory blocks, it is easier to arrange these memory resources on multiple GPUs, and the computing resources of each GPU are effectively utilized, thereby improving the reasoning and training efficiency of the model when the pipeline is parallel.

[0150] In the process of calculating the degree of parallelism, the original solution is two consecutive matrix operations. In the case of tensor parallelism, the use of block matrix multiplication between multiple GPU graphics cards to complete these two matrix operations requires the exchange of intermediate temporary operation results of the block matrices between GPU graphics cards, which leads to reduced graphics card resource utilization. When performing tensor parallelism on these two matrices between n graphics cards, the amount of data that needs to be exchanged is approximately n*(n-1)*(d_model / n)=(n-1)*d_model. And Solution 2:

[0151] The matrices involved in the calculations in steps 1, 2, and 4 are relatively small and can be completed on one graphics card without the need to use tensor parallelism, and therefore, there is no need to exchange data between multiple graphics cards. Since steps 1 and 2 are independent calculations of n_head matrices, these n_head matrix calculations can be distributed to n graphics cards, thereby achieving the computing resource utilization efficiency of multiple GPU cards pursued by tensor parallelism. In scheme 2, only the third step requires data exchange, but the amount of data to be exchanged is very small, requiring only (n-1)*(d_model / n_head) data to be exchanged, which is approximately 1 / n_head compared to the data that needs to be exchanged in the standard scheme, so it can be ignored. In scheme 2, although step 4 cannot be parallelized, the matrix in the matrix calculation performed in this step is small, only 1 / 2*d_model / d_ff=1 / 8 of the original scheme, so it can be executed on one graphics card without considering the loss caused by the inability to use tensor parallelism in this calculation. In addition, from Fig.11 As can be seen from (a) and (b) in the figure, the overall computational complexity of the second solution is significantly reduced compared to the original solution. And as the model grows, the advantages of memory usage, parallel computing degree, and overall computational complexity become more and more obvious. Specifically, Fig.11 (a) in 1 shows the parameter size and computational complexity of the fully connected layer in GPT-3 using a standard Transformer design. (b) in 11 shows the parameter size and computational complexity of two implementations of the same size of fully connected layer. Fig.11 For (a) and (b), the first linear is the matrix multiplication of d_model / d_ff. The second is the matrix multiplication of d_ff / d_model. Dropout is the tuning parameter. Furthermore, total params is the parameter size, and total mult-adds is the amount of computation data. It can be clearly seen that Fig.11 The parameter size and computational data size of (b) are Fig.11 The size of the parameters and the amount of data in (a) are 3 / 8 of the size. Fig.12 , Fig.12The training accuracy of the German-English translator trained on the multi30k dataset using the encoder-decoder architecture of the standard and solution 2 Transformer implementations. Fig.12 It can be seen that the convergence speed and model accuracy achieved by the two implementations are similar. Fig.13 , Fig.13 This is a comparison of the translation results of the test data in the dataset using the standard implementation and our implementation. There are 5 examples in total. In each example, the first line is the German input, the second line is the standard translation result given by the dataset, the third line is the translation result of the standard implementation model, and the fourth line is the translation result of the model using our implementation. Fig.12 It can be seen that the translation results of the two implementations are also similar.

[0152] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0153] See also Fig.14 , Fig.14 A structural diagram of a system for improving the Transformer network model structure disclosed in an embodiment of the present application.

[0154] A determination unit 1401 is used to replace the fully connected layer in the Transformer module based on the time series hole convolution block, determine the initial configuration parameters corresponding to the fully connected layer in the Transformer module, and determine the target configuration parameters corresponding to the time series hole convolution block according to the initial configuration parameters; wherein the time series hole convolution block is used to replace the original fully connected layer in the Transformer module, the target configuration parameters include model parameter values, time series length values, and original fully connected layer intermediate layer dimensions, the time series length value is used to characterize the time series length of the maximum input processed by the large model, the model parameter value is used to characterize the vector dimension size of the Transformer module in the large model, and the original fully connected layer intermediate layer dimension is used to describe the vector dimension of the fully connected layer of the Transformer module in the large model after the linear transformation and expansion;

[0155] A configuration unit 1402 is used to configure the convolution kernel size and hole configuration parameters of all convolution layers in the temporal hole convolution block based on a user configuration file;

[0156] The acquisition unit 1403 is used to obtain the number of layers of the time series hole convolution block corresponding to the time series hole convolution block according to the time series length value, the convolution kernel size and the hole configuration parameter, so that the receiving field of the first layer corresponding to each output of the last layer of the time series hole convolution block can cover the time series length value;

[0157] The setting unit 1404 is used to set the number of input channels and the number of output channels of all convolutional layers in the temporal hole convolution block according to the model parameter value and the dimension of the original intermediate layer of the fully connected layer.

[0158] Exemplarily, the system includes:

[0159] The determining unit 1401 is specifically configured to determine the number of all convolutional layers in the time-dry convolution block and the convolutional layer numbers corresponding to the number of all convolutional layers according to the user configuration file timing length when the actual time-dry length value is greater than the time-dry length threshold configured in the user configuration file; when the actual time-dry length value is not greater than the time-dry length threshold configured in the user configuration file, determine the number of all convolutional layers in the time-dry convolution block and the convolutional layer numbers corresponding to the number of all convolutional layers according to the actual time-dry length value;

[0160] The setting unit 1404 is used to set the hole configuration parameters of all convolution layers in the temporal hole convolution block according to the convolution layer number based on the bottom-to-top exponential growth algorithm.

[0161] Exemplarily, the system further includes:

[0162] The determination unit 1401 is also used to determine the hole reset parameters corresponding to all the temporal hole convolution blocks in the large model; wherein the hole reset parameters are used to select the hole configuration parameters in all the temporal hole convolution blocks of the large model.

[0163] Exemplarily, the system further includes: an establishing unit 1405;

[0164] The determining unit 1401 is further used to determine the residual connection blocks in all the temporal hole convolution blocks;

[0165] The establishing unit 1405 is used to establish a residual connection between the input channel located at the first layer of the temporal dilated convolution block and the output channel located at the last layer of the temporal dilated convolution block according to the residual connection block.

[0166] Exemplarily, the system includes:

[0167] The setting unit 1404 is specifically used to set the number of input channels and the number of output channels in any time series hole convolution block to the number corresponding to the model parameter value, so as to connect the input channels and the output channels in all time series hole convolution blocks one by one according to the number of the model parameter value;

[0168] Or, setting unit 1404 is also used to use the first convolution layer of the temporal hole convolution block to use each input vector dimension as an input channel, and the number of output channels of the first convolution layer is the original middle layer dimension of the fully connected layer; use the original middle layer dimension of the fully connected layer as the number of input channels of the last convolution layer, and the number of output channels is the vector dimension corresponding to the model parameter value; use the original middle layer dimension of the fully connected layer as the number of input channels and the number of output channels of other convolution layers except the first convolution layer and the last convolution layer.

[0169] Exemplarily, the system further includes: an adjustment unit 1406;

[0170] The adjustment unit 1406 is used to transpose the input data of the temporal hole convolution block whose ranking order is the target ranking in the large model after channel connection based on the convolution operation condition, and adjust the temporal dimension of the input data to the target dimension.

[0171] See also Fig.15 , Fig.15 A structural diagram of a system for improving the Transformer network model structure disclosed in an embodiment of the present application.

[0172] A determination unit 1501 is used to replace an original single fully connected layer in a Transformer module based on multiple cut fully connected layers, determine initial configuration parameters corresponding to the original fully connected layer in the Transformer module, and determine target configuration parameters corresponding to the multiple cut fully connected layers according to the initial configuration parameters; wherein the target configuration parameters include model parameter values, dimensions of intermediate layers of the original fully connected layer, and values ​​of headers of the original model configuration parameters, the model parameter values ​​are used to characterize the vector dimension size of the Transformer module in the large model, the dimensions of the intermediate layers of the original fully connected layer are used to describe the intermediate vector dimensions after linear transformation and expansion of the vector dimensions in the original fully connected layer of the Transformer module in the large model, and the values ​​of headers of the original model configuration parameters are used to describe the number of heads of the Transformer module in the large model; there is an association between the dimensions of the intermediate layers of the original fully connected layer and the model configuration parameter values ​​and the values ​​of headers of the original model configuration parameters used to describe the number of heads of the Transformer module in the large model and the number of fully connected layers of the cut fully connected layers, the model configuration parameter values ​​are used to characterize the configuration parameters of the large model, and the values ​​of headers of the original model configuration parameters are the number of cut fully connected layers obtained after the original fully connected layer is cut;

[0173] A splitting unit 1502 is used to split the number of input dimensions of the first fully connected layer in the cut fully connected layer of the Transformer module into a ratio of a model parameter value to a value of an original model configuration parameter header, and determine that the output of the first fully connected layer is a ratio of the model parameter value to the number of fully connected layers;

[0174] The acquisition unit 1503 is used to change the middle layer of the original fully connected layer according to the ratio of the model parameter value to the number of fully connected layers and the value of the original model configuration parameter header to obtain multiple intermediate output sub-connection layers corresponding to the middle layer;

[0175] The acquisition unit 1503 is further used to concatenate all the intermediate output sub-connection layers to obtain an intermediate connection layer corresponding to the intermediate layer; wherein the intermediate connection layer is associated with the model parameter value;

[0176] The connection unit 1504 is used to connect the output of the intermediate connection layer with the input dimension of the fully connected layer located at the last layer in the cut fully connected layer, so that the output dimension of the target fully connected layer located at the last layer is associated with the model parameter value.

[0177] See below Fig.16 , a schematic diagram of a structure of an improved device for a Transformer network model structure disclosed in an embodiment of the present application includes:

[0178] CPU 1601, memory 1605, input / output interface 1604, wired or wireless network interface 1603 and power supply 1602;

[0179] The memory 1605 is a temporary storage memory or a permanent storage memory;

[0180] The CPU 1601 is configured to communicate with the memory 1605 and execute the instructions in the memory 1605 to perform the aforementioned Figures 3 to 6 A method for improving the Transformer network model structure in any of the illustrated embodiments.

[0181] The embodiment of the present application also provides a chip system, the chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the aforementioned Figures 3 to 6 A method for improving the Transformer network model structure in any of the illustrated embodiments.

[0182] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes instructions, when the instructions are executed on a computer, the computer executes the aforementioned Figures 3 to 6A method for improving the Transformer network model structure in any of the illustrated embodiments.

[0183] The present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the aforementioned Figures 3 to 6 A method for improving the Transformer network model structure in any of the illustrated embodiments.

[0184] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0185] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0186] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0188] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for improving the structure of a Transformer network model, characterized in that: Applied to a computer device, the computer device is deployed with a large model, the large model is composed of a Transformer module, and the method includes: Based on the time series hole convolution block, the fully connected layer in the Transformer module is replaced, the initial configuration parameters corresponding to the fully connected layer in the Transformer module are determined, and the target configuration parameters corresponding to the time series hole convolution block are determined according to the initial configuration parameters; wherein the time series hole convolution block is used to replace the original fully connected layer in the Transformer module, and the target configuration parameters include model parameter values, time series length values, and original intermediate layer dimensions of the fully connected layer. The time series length value is used to characterize the time series length of the maximum input processed by the large model, and the model parameter value is used to characterize the vector dimension size of the Transformer module in the large model. The original intermediate layer dimension of the fully connected layer is used to express the vector dimension of the fully connected layer of the Transformer module in the large model after linear transformation and expansion; Configuring the convolution kernel size and hole configuration parameters of all convolution layers in the temporal hole convolution block based on the user configuration file; According to the temporal length value, the convolution kernel size and the hole configuration parameter, the number of temporal hole convolution block layers corresponding to the temporal hole convolution block is obtained, so that the receiving field of the first layer corresponding to each output of the last layer of the temporal hole convolution block can cover the temporal length value; According to the model parameter value and the dimension of the original intermediate layer of the fully connected layer, the number of input channels and the number of output channels of all the convolutional layers in the temporal hole convolution block are set.

2. The method for improving a large model according to claim 1, characterized in that: The configuring the hole configuration parameters of all convolutional layers in the temporal hole convolution block based on the user configuration file includes: If the actual timing length value is greater than the timing length threshold configured in the user configuration file, the number of all convolutional layers in the timing hole convolution block and the convolutional layer numbers corresponding to the number of all convolutional layers are determined according to the timing length configured in the user configuration file; if the actual timing length value is not greater than the timing length threshold configured in the user configuration file, the number of all convolutional layers in the timing hole convolution block and the convolutional layer numbers corresponding to the number of all convolutional layers are determined according to the actual timing length value; Based on a bottom-to-top exponential growth algorithm, the hole configuration parameters of all the convolutional layers in the temporal hole convolution block are set according to the convolutional layer number.

3. The method for improving a large model according to claim 2, characterized in that: The step of setting the hole configuration parameters of all the convolution layers in the temporal hole convolution block according to the convolution layer number includes: Determine a hole reset parameter corresponding to all the temporal hole convolution blocks in the large model; wherein the hole reset parameter is used to select the hole configuration parameter in all the temporal hole convolution blocks of the large model.

4. The method for improving a large model according to claim 1, characterized in that: Before setting the number of input channels and the number of output channels of all convolutional layers in the temporal atrous convolution block, the method further includes: Determine the residual connection blocks in all the temporal hole convolution blocks; According to the residual connection block, a residual connection is established between an input channel of a first layer of the temporal atrous convolution block and an output channel of a last layer of the temporal atrous convolution block.

5. The method for improving the Transformer network model structure according to claim 1, characterized in that: The step of setting the number of input channels and the number of output channels of all convolutional layers in the temporal hole convolution block according to the model parameter value and the dimension of the original intermediate layer of the fully connected layer includes: The number of input channels and the number of output channels in any temporal atrous convolution block are set to the number corresponding to the model parameter value, the first convolution layer of the temporal atrous convolution block uses each input vector dimension as an input channel, and the number of output channels of the first convolution layer is the original fully connected layer intermediate layer dimension; the last convolution layer uses the original fully connected layer intermediate layer dimension as the input channel number, and the number of output channels is the vector dimension corresponding to the model parameter value; the original fully connected layer intermediate layer dimension is used as the input channel number and output channel number of other convolution layers except the first convolution layer and the last convolution layer.

6. The method for improving the Transformer network model structure according to claim 1, characterized in that: The method further comprises: Based on the convolution operation conditions, the input data of the temporal hole convolution block whose ranking order is the target ranking in the large model after channel connection is transposed, and the temporal dimension of the input data is adjusted to the target dimension.

7. A method for improving the structure of a Transformer network model, characterized in that: Applied to a computer device, the computer device is deployed with a large model, the large model is composed of a Transformer module, and the method includes: Based on the multiple cut fully connected layers, the original single fully connected layer in the Transformer module is replaced, the initial configuration parameters corresponding to the original fully connected layer in the Transformer module are determined, and the target configuration parameters corresponding to the multiple cut fully connected layers are determined according to the initial configuration parameters; wherein the target configuration parameters include model parameter values, the original fully connected layer intermediate layer dimensions and the original model configuration parameter header values, the model parameter values ​​are used to characterize the vector dimension size of the Transformer module in the large model, and the original fully connected layer intermediate layer dimensions are used to express the Transformer in the large model. The intermediate vector dimension after linear transformation and expansion of the vector dimension in the original fully connected layer of the transformer module, the original model configuration parameter header value is used to express the number of heads of the transformer module in the large model; the intermediate layer dimension of the original fully connected layer and the model configuration parameter value and the original model configuration parameter header value are used to express the number of heads of the transformer module in the large model and the number of fully connected layers of the cut fully connected layer. The model configuration parameter value is used to characterize the configuration parameters of the large model, and the original model configuration parameter header value is the number of the cut fully connected layer obtained after the original fully connected layer is cut; Splitting the number of input dimensions of the first fully connected layer in the cut fully connected layer of the Transformer module into the ratio of the model parameter value to the value of the original model configuration parameter header, and determining the output of the first fully connected layer as the ratio of the model parameter value to the number of fully connected layers; According to the ratio of the model parameter value to the number of fully connected layers and the value of the original model configuration parameter header, a change is made in the middle layer of the original fully connected layer to obtain a plurality of intermediate output sub-connected layers corresponding to the middle layer; All intermediate output sub-connection layers are concatenated to obtain an intermediate connection layer corresponding to the intermediate layer; wherein the intermediate connection layer is associated with the model parameter value; The output of the intermediate connection layer is connected to the input dimension of the fully connected layer located at the last layer in the cut fully connected layer, so that the output dimension of the target fully connected layer located at the last layer is associated with the model parameter value.

8. An improved system for the Transformer network model structure, characterized in that: Applied to a computer device, the computer device is deployed with a large model, the large model is composed of a Transformer module, and the system includes: A determination unit, used to replace the fully connected layer in the Transformer module based on the time series hole convolution block, determine the initial configuration parameters corresponding to the fully connected layer in the Transformer module, and determine the target configuration parameters corresponding to the time series hole convolution block according to the initial configuration parameters; wherein the time series hole convolution block is used to replace the original fully connected layer in the Transformer module, and the target configuration parameters include model parameter values, time series length values, and original fully connected layer intermediate layer dimensions. The time series length value is used to characterize the time series length of the maximum input processed by the large model, and the model parameter value is used to characterize the vector dimension size of the Transformer module in the large model. The original fully connected layer intermediate layer dimension is used to express the vector dimension of the fully connected layer of the Transformer module in the large model after linear transformation and expansion; A configuration unit, configured to configure the convolution kernel size and hole configuration parameters of all convolution layers in the temporal hole convolution block based on a user configuration file; An acquisition unit, configured to obtain the number of layers of a time series dilated convolution block corresponding to the time series dilated convolution block according to the time series length value, the convolution kernel size, and the dilated configuration parameter, so that the receiving field of the first layer corresponding to each output of the last layer of the time series dilated convolution block can cover the time series length value; A setting unit is used to set the number of input channels and the number of output channels of all convolutional layers in the temporal hole convolution block according to the model parameter value and the dimension of the original intermediate layer of the fully connected layer.

9. An improved system for the Transformer network model structure, characterized in that: Applied to a computer device, the computer device is deployed with a large model, the large model is composed of a Transformer module, and the system includes: A determination unit is used to replace the original single fully connected layer in the Transformer module based on the multiple cut fully connected layers, determine the initial configuration parameters corresponding to the original fully connected layer in the Transformer module, and determine the target configuration parameters corresponding to the multiple cut fully connected layers according to the initial configuration parameters; wherein the target configuration parameters include model parameter values, the original fully connected layer intermediate layer dimensions and the original model configuration parameter header values, the model parameter values ​​are used to characterize the vector dimension size of the Transformer module in the large model, and the original fully connected layer intermediate layer dimensions are used to express the Tr in the large model The intermediate vector dimension after linear transformation and expansion of the vector dimension in the original fully connected layer of the ansformer module, the original model configuration parameter header value is used to express the number of heads of the Transformer module in the large model; the intermediate layer dimension of the original fully connected layer and the model configuration parameter value and the original model configuration parameter header value are used to express the number of heads of the Transformer module in the large model and the number of fully connected layers of the cut fully connected layer. The model configuration parameter value is used to characterize the configuration parameters of the large model, and the original model configuration parameter header value is the number of the cut fully connected layer obtained after the original fully connected layer is cut; A splitting unit, used for splitting the number of input dimensions of the fully connected layer located at the first layer in the cut fully connected layer of the Transformer module into a ratio of the model parameter value to the value of the original model configuration parameter header, and determining that the output of the fully connected layer located at the first layer is the ratio of the model parameter value to the number of the fully connected layer; An acquisition unit, configured to change the middle layer of the original fully connected layer according to the ratio of the model parameter value to the number of fully connected layers and the value of the original model configuration parameter header, to obtain a plurality of middle output sub-connection layers corresponding to the middle layer; The acquisition unit is further used to splice all the intermediate output sub-connection layers to obtain an intermediate connection layer corresponding to the intermediate layer; wherein the intermediate connection layer is associated with the model parameter value; A connection unit is used to connect the output of the intermediate connection layer with the input dimension of the fully connected layer located at the last layer in the cut fully connected layer, so that the output dimension of the target fully connected layer located at the last layer is associated with the model parameter value.

10. A device for improving the structure of a Transformer network model, characterized in that: The device comprises: CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply; The memory is a transient storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the method for improving the Transformer network model structure as described in any one of claims 1 to 8.

Images