Tensor parallel method for image-text generation model
By performing tensor parallel calculation and multi-card parallel processing on the GPT Transformer model, combined with the deep diffusion network, the problem of large-scale model parameters exceeding the single card memory is solved, and efficient and stable graphic and text generation model operations and fast inference are achieved.
Patent Information
- Application Number
- CN202410958199.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-08
AI Technical Summary
The amount of parameters of large-scale deep learning models exceeds single-card memory, resulting in challenges in distributed training and inference, requiring more complex parallel technologies to support larger-scale models.
Using tensor parallel method for graphic and text generation models, the GPT Transformer model is calculated in parallel by constructing a hierarchical control unit to realize the hierarchical loading of the model, and the graphic and text generation model is trained in combination with the deep diffusion network, and multiple cards are used to parallelize the graphics and text generation model. Process user needs and perform cross-modal semantic alignment of text and images.
The model operation efficiency and inference speed are improved, and more efficient and stable graphic and text generation model operations are achieved to adapt to the rapid inference during peak demand.
Smart Images

Figure CN118917359A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of tensor parallelism of a graph-text generation model, and in particular to a tensor parallelism method for a graph-text generation model. Background Art
[0002] New deep learning models have continuously made breakthroughs in the performance of various tasks. These models rely on deep and wide network structures to "memorize" certain representations, and often increase the number and width of the model's network. Increasing the number of parameters can further improve the model's performance, so the number of model parameters has increased in recent years.
[0003] Large-scale models have brought new challenges to distributed training and reasoning: in the past, small-scale models that could fit on a single card could achieve excellent performance and scalability by training on a single machine with multiple cards. However, nowadays, the number of parameters of a single model or even an operator exceeds the memory of a single card, and more complex parallel technology is needed to distribute the parameters to multiple cards on each node, so that distributed training and reasoning can support larger-scale models. Summary of the invention
[0004] The purpose of the present invention is to solve the technical problems raised in the above background technology.
[0005] The present invention adopts the following technical solution: a tensor parallel method for a graphic and text generation model, comprising the following steps:
[0006] S1: Construct a hierarchical control unit to perform tensor parallel calculations on the GPT Transformer model to achieve hierarchical loading of the model;
[0007] S2: Use the multi-card parallel GPT Transformer model to perform natural language processing on user needs and achieve cross-membrane semantic alignment of text and images;
[0008] S3: Combine the deep diffusion network to train the image-text generation model.
[0009] Preferably, the GPT Transformer model consists of an Attention module and an MLP module. The Attention module consists of a self-attention layer and a dropout layer. The MLP module has two MLP layers. The first MLP layer changes the dimension from H to 4H, and the second MLP layer changes the dimension from 4H back to H. Nonlinear activation GeLU is used in between.
[0010] Preferably, the hierarchical control unit includes an MLP module control unit, an Attention module control unit and an output Embedding layer control unit, which perform tensor parallel calculations on the Transformer model respectively.
[0011] Preferably, the tensor parallel computing is performed by dividing the input matrix and the parameter matrix into blocks, and then performing calculations on different blocks separately.
[0012] Preferably, the tensor parallel computing cross-cuts the parameter matrix, dividing the dimension of the input matrix X and the dimension of the parameter weight matrix A into blocks, and the specific steps are as follows:
[0013] a. Assume that the input matrix X has a shape of (100,300), and split it horizontally into two matrices X1 and X2, each with a shape of (100,150);
[0014] b. Assume that the shape of the parameter weight matrix A is (300, 200), split it horizontally into two matrices A1 and A2, each with a shape of (150, 200);
[0015] The shape of cY is (100, 200), and the shape of Y1 and Y2 is (100, 200), where Y1 = X1·A1 and Y2 = X2·A2;
[0016] d. Perform parallel calculation on the above-split matrix, and the calculation formula is:
[0017]
[0018] Preferably, the tensor parallel computing performs longitudinal slicing on the parameter matrix, dividing the dimension of the input matrix X and the dimension of the parameter weight matrix A into blocks, and the specific steps are as follows:
[0019] a. Assume that the input matrix X has a shape of (100,300), and split it vertically into two matrices X1 and X2, each with a shape of (100,150);
[0020] b. Assume that the shape of the parameter weight matrix A is (300, 200), and split it horizontally into two matrices A1 and A2, each with a shape of (300, 100);
[0021] The shape of cY is (100, 200), and the shape of Y1 and Y2 is (100, 100), where Y1 = X1·A1 and Y2 = X2·A2;
[0022] d. Perform parallel calculation on the above-split matrix, and the calculation formula is:
[0023] X×A=X×[A1, A2]=[X·A1, X·A2].
[0024] Preferably, the calculation operation corresponding to the first MLP layer in the MLP module can be expressed as Y=GeLU(XA), each MLP layer is vertically split by column, corresponding to A=[A1,A2], Y=[Y1,Y2]=[GeLU(XA1),GeLU(XA2)]; the second MLP layer in the MLP module is horizontally split by row and directly connected to the first MLP layer, and the corresponding formula is as follows:
[0025]
[0026] Y (2) -Y (2) -X (2) -X (2) -A 2 .
[0027] Preferably, the tensor parallel segmentation method of the Attention module includes:
[0028] a. Split the three matrices Q / K / V in the Attention module by columns;
[0029] b. The result of the calculation is multiple independent parts;
[0030] c. Split the final weight matrix by rows and perform accumulation operation (all_reduce) on the results.
[0031] Preferably, the output Embedding layer is subjected to tensor parallel calculation by the output Embedding layer control unit, and the calculation formula is as follows:
[0032] h0=UW s +W p
[0033]
[0034] Among them, U=(u -k ,…,u -1 ) is the word that has been read and is used to predict the next word, n is the number of layers, W e is the word embedding matrix, w p is the position embedding matrix, W e It is also reused in the final output. The shape of the embedding layer is hidden-size(H)×vocabulary-size(v).
[0035] Compared with the prior art, the advantages and positive effects of the present invention are:
[0036] The present invention performs multi-level tensor parallel calculations on the GPT Transformer model to achieve offline hierarchical loading of the model, greatly improving the model operation efficiency and achieving faster reasoning speed during peak demand periods. In addition, the present invention uses a multi-card parallel GPT Transformer model to perform natural language processing on user needs, achieve cross-modal semantic alignment of text and images, and combine the deep diffusion network training image-text generation model to make the model operation more efficient and stable. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 A schematic diagram of cross-cutting a parameter matrix for a tensor parallel method for a graphic-text generation model proposed by the present invention;
[0038] Figure 2 A schematic diagram of the forward and backward calculation process of a tensor parallel method for a graphic and text generation model proposed by the present invention, which is a parameter matrix segmented by row;
[0039] Figure 3 A schematic diagram of longitudinally slicing a parameter matrix for a tensor parallel method for a graphic-text generation model proposed in the present invention;
[0040] Figure 4 A schematic diagram of the forward and backward calculation process of a tensor parallel method for a graphic and text generation model proposed by the present invention, which is a column-sliced parameter matrix;
[0041] Figure 5 A schematic diagram of the tensor parallel computing process of the Attention module of a tensor parallel method for a graphic-text generation model proposed in the present invention. DETAILED DESCRIPTION
[0042] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0043] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments of the following disclosure.
[0044] Embodiment 1
[0045] See also Figure 1-5 The present invention provides a technical solution: a tensor parallel method for a graph-text generation model, comprising the following steps:
[0046] S1: Construct a hierarchical control unit to perform tensor parallel computing on the GPT Transformer model:
[0047] The GPT Transformer model includes an Attention module and an MLP module. The Attention module consists of a self-attention layer and a dropout layer. The MLP module has two MLP layers. The first MLP layer changes the dimension from H to 4H, and the second MLP layer changes the dimension from 4H back to H. The nonlinear activation function GeLU is used in between. The connection of each layer also uses a residual connection similar to ResNet.
[0048] The hierarchical control unit mainly includes the MLP module control unit, the Attention module control unit and the output Embedding layer control unit, which perform tensor parallel calculations on the Transformer model.
[0049] S2: Use the multi-card parallel GPT Transformer model to perform natural language processing on user needs, divide the input matrix and parameter matrix into blocks, and then perform calculations on different blocks separately:
[0050] This slices the parameter matrix:
[0051] a. Assume that the input matrix X has a shape of (100,300), and split it horizontally into two matrices X1 and X2, each with a shape of (100,150);
[0052] b. Assume that the shape of the parameter weight matrix A is (300, 200), split it horizontally into two matrices A1 and A2, each with a shape of (150, 200);
[0053] The shape of cY is (100, 200), and the shape of Y1 and Y2 is (100, 200), where Y1 = X1·A1 and Y2 = X2·A2;
[0054] d. Perform parallel calculation on the above-split matrix, and the calculation formula is:
[0055]
[0056] Slice the parameter matrix:
[0057] a. Assume that the input matrix X has a shape of (100,300), and split it vertically into two matrices X1 and X2, each with a shape of (100,150);
[0058] b. Assume that the shape of the parameter weight matrix A is (300, 200), and split it horizontally into two matrices A1 and A2, each with a shape of (300, 100);
[0059] The shape of cY is (100, 200), and the shape of Y1 and Y2 is (100, 100), where Y1 = X1·A1 and Y2 = X2·A2;
[0060] d. Perform parallel calculation on the above-split matrix, and the calculation formula is:
[0061] X×A=X×[A1, A2]=[X·A1, X·A2].
[0062] S3: Combined with the deep diffusion network training image and text generation model:
[0063] The calculation operation corresponding to the first MLP layer in the MLP module can be expressed as Y = GeLU (XA). Each MLP layer is split vertically by column, and the corresponding A = [A1, A2].
[0064] Y = [Y1, Y2] = [GeLU (XA1), GeLU (XA2)]; the second MLP layer in the MLP module is split horizontally by row (Row) and directly connected to the first MLP layer. The corresponding formula is as follows:
[0065]
[0066] Y (2) -Y (2) -X (2) -X (2) -A 2 ;
[0067] The tensor parallel splitting methods of the Attention module include:
[0068] a. Split the three matrices Q / K / V in the Attention module by columns;
[0069] b. The result of the calculation is multiple independent parts;
[0070] c. Split the final weight matrix by row and perform accumulation operation (all_reduce) on the results;
[0071] The output Embedding layer is used to perform tensor parallel calculations on the output Embedding layer through the output Embedding layer control unit. The calculation formula is as follows:
[0072] h0=UWs +W p
[0073]
[0074] Among them, U=(u -k ,…,u -1 ) is the word that has been read and is used to predict the next word, n is the number of layers, W e is the word embedding matrix, w p is the position embedding matrix, W e It is also reused in the final output. The shape of the embedding layer is hidden-size(H)×vocabulary-size(v).
[0075] Embodiment 2
[0076] See also Figure 1-5 In this embodiment, the present invention provides an image generation system using the tensor parallel method, which generates a target image by calling a trained image generation model through input generation requirements. The image content embedding system includes an input module, an image generation module, and a work display module.
[0077] The input module is used to input user demand information, including type selection, picture description unit, style selection unit and style modification unit. The type selection unit includes but is not limited to image embedding and posture generation; the picture description unit is used for users to input image generation requirements through natural language; the style selection unit includes style type selections such as watercolor, ink painting, and oil painting, and style modification selections such as high detail, high quality, high definition, and movie effects. The input module also includes advanced setting options, in which users can input unwanted content on the screen and restrict the content of image generation.
[0078] The image generation module is used to call the image generation model to generate the target image, and specifically includes a task issuing unit, a hierarchical control unit, and an image generation unit. Among them, the task issuing unit is used to obtain user demand information; the hierarchical control unit includes an MLP module control unit, an Attention module control unit, and an output Embedding layer control unit, which are used to perform multi-level tensor parallel calculations on the model to achieve offline hierarchical loading of the model; the image generation unit is used to generate the target image using a multi-card parallel generation model according to user needs.
[0079] The work display module is used to display the image generation results. All historical images generated by the user through the system can be displayed in the image display module in the form of a waterfall flow. The present invention supports users to quickly search for target images based on generation time or keywords, making it convenient for users to quickly retrieve the desired personalized images, which is simple and efficient.
[0080] Furthermore, the image generation system provided in this embodiment adopts the above-mentioned tensor parallel method, which can effectively improve the model operation efficiency and help users to efficiently obtain the target image.
[0081] Embodiment 3
[0082] See also Figure 1-5 The present invention provides a copywriting generation system using the tensor parallel method. By inputting the demand description and calling the trained copywriting generation model, personalized copywriting can be generated. The copywriting generation system includes an input module, a model calling module and a dialogue display module.
[0083] The input module is used for users to input the required keywords of the copy content they want to generate through natural language.
[0084] The model calling module is used to call the copywriting generation model that applies the above multi-level tensor parallel computing method to generate personalized copywriting content. In addition to using common corpora during the training process, the copywriting generation model also uses a large number of promotional copywriting resources related to various industries, so that the model can fully learn the requirements and formats of advertising copywriting. In addition to general question-and-answer functions, the copywriting generation model can also help companies intelligently generate rich and diverse promotional copywriting based on keywords in major operating businesses.
[0085] The dialogue display module is used to display the copy generation results.
[0086] Furthermore, the copy generation system provided in this embodiment adopts the above-mentioned tensor parallel method, which can effectively improve the model operation efficiency, shorten the copy generation time, and at the same time has the function of generating personalized promotional copy, further improving user satisfaction.
[0087] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.
Claims
1. A tensor parallel method for image-text generation model, characterized in that: The following steps are involved: S1: Construct a hierarchical control unit to perform tensor parallel calculations on the GPT Transformer model to achieve hierarchical loading of the model; S2: Use the multi-card parallel GPT Transformer model to perform natural language processing on user needs and achieve cross-membrane semantic alignment of text and images; S3: Combine the deep diffusion network to train the image-text generation model.
2. The tensor parallel method for the image-text generation model according to claim 1, characterized in that: The GPTTransformer model consists of an Attention module and an MLP module. The Attention module consists of a self-attention layer and a dropout layer. The MLP module has two MLP layers. The first MLP layer changes the dimension from H to 4H, and the second MLP layer changes the dimension from 4H back to H. Nonlinear activation GeLU is used in the middle.
3. The tensor parallel method for image-text generation model according to claim 1, characterized in that: The hierarchical control unit includes an MLP module control unit, an Attention module control unit and an output Embedding layer control unit, which respectively perform tensor parallel calculations on the Transformer model.
4. The tensor parallel method for image-text generation model according to claim 3, characterized in that: The tensor parallel computing is performed by dividing the input matrix and the parameter matrix into blocks and then performing calculations on different blocks respectively.
5. The tensor parallel method for image-text generation model according to claim 4, characterized in that: The tensor parallel computing cross-cuts the parameter matrix and divides the dimensions of the input matrix X and the parameter weight matrix A into blocks. The specific steps are as follows: a. Assume that the input matrix X has a shape of (100, 300), and split it horizontally into two matrices X1 and X2, each with a shape of (100, 150); b. Assume that the shape of the parameter weight matrix A is (300, 200), and split it horizontally into two matrices A1 and A2, each with a shape of (150, 200); The shape of cY is (100, 200), and the shape of Y1 and Y2 is (100, 200), where Y1 = X1·A1 and Y2 = X2·A2; d. Perform parallel calculation on the above-split matrix, and the calculation formula is:
6. The tensor parallel method for image-text generation model according to claim 4, characterized in that: The tensor parallel computing slices the parameter matrix vertically, dividing the dimensions of the input matrix X and the dimensions of the parameter weight matrix A into blocks. The specific steps are as follows: a. Assume that the input matrix X has a shape of (100, 300), and split it vertically into two matrices X1 and X2, each with a shape of (100, 150); b. Assume that the shape of the parameter weight matrix A is (300, 200), and split it horizontally into two matrices A1 and A2, each with a shape of (300, 100); The shape of cY is (100, 200), and the shape of Y1 and Y2 is (100, 100), where Y1 = X1·A1 and Y2 = X2·A2; d. Perform parallel calculation on the above-split matrix, and the calculation formula is: X×A=X×[A1, A2]=[X·A1, X·A2].
7. The tensor parallel method for image-text generation model according to claim 3, characterized in that: The calculation operation corresponding to the first MLP layer in the MLP module can be expressed as Y=GeLU(XA). Each MLP layer is vertically split by column, and the corresponding A=[A1, A2], Y=[Y1, Y2]=[GeLU(XA1), GeLU(XA2)]; the second MLP layer in the MLP module is horizontally split by row and directly connected to the first MLP layer. The corresponding formula is as follows: AND (2) -AND (2) -X (2) -X (2) -TO 2 。 8. The tensor parallel method for image-text generation model according to claim 3, characterized in that: The tensor parallel splitting method of the Attention module includes: a. Split the three matrices Q / K / V in the Attention module by columns; b. The result of the calculation is multiple independent parts; c. Split the final weight matrix by rows and perform accumulation operation (al1_reduce) on the results.
9. The tensor parallel method for image-text generation model according to claim 1, characterized in that: The output Embedding layer control unit performs tensor parallel calculation on the output Embedding layer, and the calculation formula is as follows: h0=UW s +W p Among them, U=(u -k ,…,u -1 ) is the word that has been read and is used to predict the next word, n is the number of layers, W e is the word embedding matrix, w p is the position embedding matrix, W e It is also reused in the final output. The shape of the embedding layer is hidden-size(H)×vocabulary-size(v).
Citation Information
Patent Citations
Fragmentation parallel model training method and device, equipment and storage medium
CN116704291A
Solar cell
JP2018046055A
Cited By
Communication optimization method for diffusion model parallel training
CN120110931A
Mobile terminal LLM heterogeneous scheduling method and system based on hybrid parallel
CN120893587A
