Content generation method and apparatus

Abstract

This application discloses a content generation method and apparatus. The main technical solution includes: obtaining first weight information of a first model block from a first memory; generating first output data based on the first weight information and target data using the first model block, and obtaining second weight information of a second model block from the first memory; generating target content corresponding to the target data using the second model block based on the second weight information and the first output data; wherein both the first model block and the second model block are deployed in the second memory. This application enables the content generation model to perform inference on graphics cards with limited video memory by reloading the weight information of the model blocks in the content generation model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a content generation method and apparatus. Background Technology

[0002] With the rapid development of computer technology, content generation models have made significant progress. A content generation model consists of a text encoder, an image encoder, a diffusion model, and a decoder. The text encoder converts user prompts into text vectors, the image encoder converts user-uploaded images into image vectors, and the text and image vectors are cyclically used in the diffusion model to obtain content vectors. The decoder then decodes these content vectors to generate the content.

[0003] Current content generation models are typically deployed on GPUs with high VRAM usage to meet the requirements of repeated runs of the diffusion model. However, content generation models cannot be deployed on GPUs with limited VRAM, preventing them from performing inference. Summary of the Invention

[0004] In view of this, this application provides a content generation method and apparatus for inferring a content generation model on a graphics card with limited video memory, thereby generating target content.

[0005] This application provides the following solution:

[0006] In a first aspect, a content generation method is provided, the method comprising: obtaining first weight information of a first model block from a first memory; generating first output data based on the first weight information and target data using the first model block, and obtaining second weight information of a second model block from the first memory; generating target content corresponding to the target data based on the second weight information and the first output data using the second model block; wherein the first model block and the second model block are both deployed in the second memory.

[0007] Optionally, the second memory includes a first cache and a second cache, the first model block is deployed in the first cache, and the second model block is deployed in the second cache; obtaining the first weight information of the first model block from the first memory includes: transferring the first weight information from the first memory to the first cache; obtaining the second weight information of the second model block from the first memory includes: transferring the second weight information from the first memory to the second cache.

[0008] Optionally, after generating the first output data using the first model block, based on the first weight information and the target data, the method further includes: deleting the first model block and the first weight information from the first cache.

[0009] Optionally, the first model block and the second model block are both one of at least two model blocks concatenated in the content generation model. The content generation model is deployed in the second memory and also includes a third model block. Using the second model block, based on the second weight information and the first output data, the target content corresponding to the target data is generated, including: using the second model block, based on the second weight information and the first output data, generating the second output data, and transferring the third weight information of the third model block from the first memory to the first cache; using the third model block, based on the third weight information and the second output data, generating the target content.

[0010] Optionally, the transmission method for transferring the second weight information from the first memory to the second buffer, and / or the transmission method for transferring the third weight information of the third model block from the first memory to the first buffer, is asynchronous transmission.

[0011] Optionally, there are at least two second memories, each of which is equipped with a content generation model. The first memory stores the weight information of each model block in each content generation model. Generating target content corresponding to the target data includes: dividing the target data based on the number of second memories to obtain at least two data blocks that correspond one-to-one with the at least two second memories; for each of the at least two data blocks, obtaining the target first weight information of the target first model block deployed in the target second memory corresponding to the data block from the first memory; using the target first model block, generating third output data based on the target first weight information and the data block, and asynchronously obtaining the target second weight information of the target second model block deployed in the target second memory from the first memory; using the target second model block, generating the content block corresponding to the data block based on the target second weight information and the third output data; and fusing the content blocks corresponding to each of the at least two data blocks to obtain the target content.

[0012] Optionally, each model block includes: a self-attention module, a cross-attention module, and a feedforward neural network module. The weight information of each model block includes: the weight information of the self-attention module, the weight information of the cross-attention module, and the weight information of the feedforward neural network module. Using the target first model block, based on the target first weight information and the data block, a third output data is generated, and the target second weight information of the target second model block deployed in the target second memory is asynchronously retrieved from the first memory. This includes: using the self-attention module of the target first model block, based on the weight information of the self-attention module of the target first model block and the data block, a fourth output data is generated, and the weight information of the self-attention module of the target second model block is asynchronously retrieved from the first memory; using the cross-attention module of the target first model block, based on the weight information of the cross-attention module of the target first model block and the fourth output data, a fifth output data is generated; using the feedforward neural network module of the target first model block, based on the weight information of the feedforward neural network module of the target first model block and the fifth output data, a third output data is generated, and the weight information of the cross-attention module and the weight information of the feedforward neural network module of the target second model block are asynchronously retrieved from the first memory.

[0013] Optionally, using the target second model block, based on the target second weight information and the third output data, a content block corresponding to the data block is generated, including: using the self-attention module of the target second model block, based on the weight information of the self-attention module of the target second model block and the third output data, to generate sixth output data; using the cross-attention module of the target second model block, based on the weight information of the cross-attention module of the target second model block and the sixth output data, to generate seventh output data; and using the feedforward neural network module of the target second model block, based on the weight information of the feedforward neural network module of the target second model block and the seventh output data, to generate a content block.

[0014] Optionally, the content generation models deployed in at least two second memories process at least two data blocks in parallel; the input data of the self-attention module of the target first model block also includes the generated data of the self-attention modules of other first model blocks, which are first model blocks deployed in at least two second memories other than the target second memory; the input data of the self-attention module of the target second model block also includes the generated data of the self-attention modules of other second model blocks, which are second model blocks deployed in at least two second memories other than the target second memory.

[0015] Secondly, a content generation apparatus is provided, comprising: an acquisition unit configured to acquire first weight information of a first model block from a first memory; a first generation unit configured to generate first output data using the first model block, based on the first weight information and target data, and acquire second weight information of a second model block from the first memory; and a second generation unit configured to generate target content corresponding to the target data using the second model block, based on the second weight information and the first output data; wherein both the first model block and the second model block are deployed in the second memory.

[0016] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, implements the steps of the method of any one of the first aspects above.

[0017] Fourthly, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, perform the steps of the method of any one of the first aspects described above.

[0018] Fifthly, a computer program product is provided, comprising a computer program that, when executed, implements the steps of the method described in any of the first aspects above.

[0019] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0020] 1) This application deploys model blocks in the content generation model in a second memory location and stores the weight information of the model blocks in a first memory location. Therefore, the second memory location can be relatively small. During content generation model inference, the first weight information of the first model block is retrieved from the first memory location. Based on the first weight information and the target data, the first model block generates first output data. Simultaneously, the second weight information of the second model block is retrieved from the first memory location. Based on the second weight information and the first output data, the second model block generates the target content corresponding to the target data. Therefore, through this weight overloading method, even graphics cards with limited memory can complete the inference process of the content generation model.

[0021] 2) The second memory in this application also includes a first cache area and a second cache area. The first model block can be deployed in the first cache area, and the second model block can be deployed in the second cache area. During content generation model inference, the first weight information can be transferred from the first memory to the first cache area for caching, so that the first model block can generate the first output data based on the first weight information and the target data. During the process of the first model block generating the first output data, the second weight information can be transferred from the first memory to the second cache area for caching, so that after the first model block generates the first output data, the second model block can generate the target content corresponding to the target data based on the second weight information and the first output data. This reduces the transmission time of weight information and reduces the impact of weight information transmission on the time consumed by the content generation model in generating the target content.

[0022] 3) After the first model block in the first cache generates the first output data, this application can release the storage space of the first cache by deleting the first model block and the first weight information in the first cache. This allows the first cache to deploy other model blocks in the content generation model and cache the weight information of other model blocks, enabling inference of other model blocks. In other words, only two model blocks reside in the second memory for implementing the inference process of the content generation model. This greatly reduces the memory usage of the content generation model. Therefore, while ensuring that the content generation model can complete inference, it can also reduce the memory usage of the content generation model, thereby ensuring that graphics cards with small video memory can infer the content generation model.

[0023] 4) The content generation model of this application may also include a third model block. Since the storage space of the first cache has been released after the first model block completes inference, the third model block can be deployed in the first cache. During the process of the second model block generating the second output data, the third weight information of the third model block can be transferred from the first memory to the first cache, so that the third model block can generate the target content based on the third weight information and the second output data. That is to say, no matter how many concatenated model blocks are included in the content generation model, the second memory can complete the inference process of the content generation model through the first and second caches. Therefore, even graphics cards with small video memory can infer the content generation model.

[0024] 5) This application transmits the second weight information from the first memory to the second cache during the generation of the first output data of the first model block using asynchronous transmission, and / or transmits the third weight information from the first memory to the first cache during the generation of the second output data of the second model block. This can improve the flexibility and reliability of weight information transmission, thereby further reducing the impact of weight information transmission on the time consumption of the content generation model in generating the target content.

[0025] 6) There can be multiple second memories in this application, each of which deploys a content generation model, while the first memory stores the weight information of each model block in each content generation model. The target data can be segmented according to the number of second memories, so that the content generation model in each second memory processes a portion of the data. Then, the content blocks obtained by the content generation models in each second memory are merged to obtain the target content corresponding to the target data, thereby reducing the latency of generating the target content.

[0026] 7) The model block deployed in the second memory of this application includes a self-attention module, a cross-attention module, and a feedforward neural network module. The weight information stored in the first memory includes the weight information of the self-attention module, the weight information of the cross-attention module, and the weight information of the feedforward neural network module. During the process of the self-attention module of the target first model block generating the fourth output data, the weight information of the self-attention module of the target second model block can be asynchronously obtained from the first memory and transmitted to the second buffer. Furthermore, during the process of the cross-attention module and the feedforward neural network module of the target first model block generating the third output data, the weight information of the cross-attention module and the weight information of the feedforward neural network module of the target second model block can be asynchronously obtained from the first memory and transmitted to the second buffer. This reduces the impact of weight information transmission on the time consumption of the inference process of the target second model block.

[0027] 8) In the process of generating the sixth output data by the self-attention model of the target second model block, generating the seventh output data by the cross-attention module, and generating the content block by the feedforward neural network module, there is no need to wait for the weight information to be transmitted in the first memory. The corresponding weight information can be directly obtained in the second cache. Therefore, the impact of the transmission of weight information on the time consumption of the content generation model in generating the target content can be reduced.

[0028] 9) When there are multiple second memory locations, the content generation models deployed in multiple second memory locations process the corresponding data blocks in parallel to reduce the latency of generating the target content. Furthermore, the self-attention module of the target first model block can combine the generated data from the self-attention modules of other first model blocks to process the corresponding data, and the target second model block can combine the generated data from the self-attention modules of other second model blocks to process the corresponding data, thereby enhancing the representational ability of the content generation model and thus improving the quality of the generated target content.

[0029] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A schematic diagram of an architecture for a content generation model in related technologies.

[0032] Figure 2 This is a system architecture diagram applicable to the embodiments of this application.

[0033] Figure 3 A flowchart illustrating the content generation method provided in this application embodiment.

[0034] Figure 4 This is a reasoning diagram of a content generation model provided in this application.

[0035] Figure 5 A reasoning diagram illustrating another content generation model provided in this application.

[0036] Figure 6 A schematic block diagram of a content generation apparatus provided in an embodiment of this application.

[0037] Figure 7 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0039] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0040] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0041] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0042] Figure 1 An architectural diagram of a content generation model in related technologies, such as Figure 1 As shown, the content generation model consists of a text encoder, an image encoder, a diffusion model, and a decoder. The text encoder converts user prompts into text vectors, and the image encoder converts user-uploaded images into image vectors. These text and image vectors are cyclically used in the diffusion model's calculations to obtain content vectors. The decoder then decodes these content vectors to generate the content. Currently, content generation models are typically deployed on GPUs with high VRAM usage to meet the requirements of repeated runs of the diffusion model. However, they cannot be deployed on GPUs with limited VRAM, preventing inference operations on these machines.

[0043] In one related technique, to reduce the memory footprint of the content generation model, model quantization is used to save the weights in int8 / fp8 format, thus reducing overall GPU memory usage. For example, the original weights (fp16 precision) are converted into scaling factors and quantized weights (fp8 / int8). However, this operation may lose some information, resulting in a loss of weight information during dequantization, which leads to some degradation in the generation effect of the content generation model. This often requires fine-tuning or adjusting the quantization granularity to reduce the degree of degradation. However, even after quantization, the weights of the content generation model are still relatively large, making it impossible to deploy the content generation model on GPUs with limited GPU memory.

[0044] Another related technology uses xDiT to provide multi-GPU parallel inference solutions for diffusion models, including sequence parallelism (SP), tensor parallelism (TP), pipelined parallelism (PP), and data parallelism (DP), which can reduce the latency of generating a single video to some extent. Specifically, when using sequence parallelism for inference, communication latency is low, but the weight usage cannot be reduced, resulting in high VRAM usage. When using tensor parallelism for inference, weight usage can be reduced, but communication latency is relatively high, and for GPUs with limited VRAM, the proportion of communication time is even higher.

[0045] In view of this, this application provides a new approach to reasoning about content generation models. To facilitate understanding of this application, the system architecture on which this application is based is first described. Figure 2 The system architecture diagram applicable to the embodiments of this application is as follows: Figure 2 As shown, the system architecture may include: user devices and content generation devices located on the content generation application server.

[0046] The user equipment and the server can communicate with each other. The user equipment and the server can be connected directly or indirectly through wired or wireless communication, which is not limited herein.

[0047] User devices can include, but are not limited to, smart mobile terminals, smart home devices, wearable devices, and PCs (Personal Computers). Smart mobile devices can include mobile phones, tablets, laptops, PDAs (Personal Digital Assistants), and connected cars. Smart home devices can include smart TVs, smart refrigerators, and so on. Wearable devices can include smartwatches, smart glasses, virtual reality devices, augmented reality devices, and mixed reality devices (i.e., devices that support both virtual and augmented reality).

[0048] Content generation application servers can be standalone servers, server clusters, or cloud servers. Cloud servers, also known as cloud computing servers or cloud hosts, are a hosting product within the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.

[0049] Before performing content generation tasks, the content generation model used by the content generation device can be pre-loaded. Users can input target data through their devices, which then include this data in a content generation request and send it over the network to the content generation device on the server. The content generation device generates the target content corresponding to the target data and returns it to the user device over the network. The user device then displays the received target content to the user.

[0050] Apart from Figure 2 In addition to the architecture shown, target content corresponding to target data can also be generated by computer terminal devices with strong computing capabilities using the method provided in the embodiments of this application.

[0051] It should be understood that Figure 2The number of user devices and servers shown is merely illustrative. Depending on implementation needs, there can be any number of user devices and servers.

[0052] It should be noted that the terms "first" and "second" used in this disclosure do not have any limitations on size, order, or quantity, but are only used to distinguish them in name. For example, "first memory" and "second memory" are used to distinguish two types of memory, and "first model block," "second model block," and "third model block" are used to distinguish three types of model blocks, and so on.

[0053] Figure 3 This is a flowchart of a content generation method provided in an embodiment of this application. This method can be... Figure 2 The user equipment in the system shown executes this. For example... Figure 3 As shown, the method may include the following steps:

[0054] Step 301: Obtain the first weight information of the first model block from the first memory.

[0055] Step 302: Using the first model block, based on the first weight information and the target data, generate the first output data, and obtain the second weight information of the second model block from the first memory.

[0056] Step 303: Using the second model block, based on the second weight information and the first output data, generate the target content corresponding to the target data; wherein, both the first model block and the second model block are deployed in the second memory.

[0057] As can be seen from the above process, this application deploys the model blocks in the content generation model in the second memory and stores the weight information of the model blocks in the first memory. Therefore, the second memory can be relatively small. During content generation model inference, the first weight information of the first model block is retrieved from the first memory. Based on the first weight information and the target data, the first model block can generate the first output data. Simultaneously, the second weight information of the second model block is retrieved from the first memory. Based on the second weight information and the first output data, the second model block can generate the target content corresponding to the target data. Therefore, through this weight overloading method, even graphics cards with limited memory can complete the inference process of the content generation model.

[0058] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments.

[0059] First, the above step 301, namely "obtaining the first weight information of the first model block from the first memory", will be described in detail with reference to the embodiments.

[0060] In this embodiment, the first memory can be central processing unit (CPU) memory or paged memory. Preferably, the first memory is paged memory, which reduces the time spent accessing paged memory through CPU memory. The second memory is video memory, and the content generation model is deployed in the second memory.

[0061] The content generation model can include multiple concatenated model blocks. When loading the content generation model, the weight information of all model blocks included in the content generation model can be transferred to a pre-allocated first memory to reduce the weight information's usage in the second memory.

[0062] To further reduce the content generation model's footprint on the secondary memory, two model blocks from the content generation model can be permanently resident in the secondary memory. This ensures that the content generation model can perform normal inference without consuming too much space in the secondary memory. Optionally, the number of model blocks that can be permanently resident in the secondary memory can be determined based on the storage space available in the secondary memory. For example, if the secondary memory is large, three, four, or more model blocks can be permanently resident in the secondary memory.

[0063] In the embodiments of this application, the content may include visual content such as text content, images, or videos.

[0064] In one embodiment, the second memory includes a first cache and a second cache, with the first model block deployed in the first cache and the second model block deployed in the second cache. Obtaining the first weight information of the first model block from the first memory includes: transferring the first weight information from the first memory to the first cache.

[0065] Figure 4 A reasoning diagram of a content generation model provided in this application, such as Figure 4 As shown, the first memory contains the weight information of N model blocks in the content generation model (in... Figure 4 The weight information is shown in the diagram as the weight information of model block 0, model block 1, model block 2, ..., model block N-1. The second memory includes a first cache area and a second cache area, and the first model block (in...) Figure 4 Model block 0 is deployed in the first cache, and the second model block (in...) is deployed in the second cache. Figure 4 The model block 1 shown in the image is deployed in the second cache area. Figure 4 As shown in (a), the first weight information (i.e. the weight information of model block 0) can be transferred from the first memory to the first cache area, so that the first output data can be generated in the first cache area using the first model block based on the first weight information and the target data.

[0066] In some alternative implementations, the first weight information can be pre-transmitted from the first memory to the first cache for caching, so that the first model block can directly use the first weight information to process the target data when processing the target data, reducing the time the first model block waits for the transmission of the first weight information; alternatively, the first weight information can be transmitted from the first memory to the first cache when the first model block processes the target data.

[0067] The following describes step 302, namely, "using the first model block, generating the first output data based on the first weight information and the target data, and obtaining the second weight information of the second model block from the first memory," in detail with reference to the embodiments.

[0068] After obtaining the first weight information of the first model block from the first memory, the first model block can use the first weight information to calculate the target data and obtain the first output data. In order to reduce the impact of the transmission of the second weight information of the second model block on the time consumption of the content generation model in generating the target content, the second weight information of the second model block can be obtained from the first memory during the process of the first model block generating the first output data. That is, the second weight information is transmitted from the first memory to the second cache for caching.

[0069] Continue to refer Figure 4 ,like Figure 4 As shown in (a), the second weight information (i.e. the weight information of model block 1) can be transferred from the first memory to the second cache, so that the second model block can process the first output data based on the second weight information in the second cache.

[0070] It can be seen that if the time taken to generate the first output data of the first model block is greater than or equal to the time taken to transfer the second weight information from the first memory to the second cache, then the time taken to generate the first output data of the first model block can completely cover the time taken to transfer the second weight information from the first memory to the second cache. Therefore, the impact of the time taken to transfer the second weight information from the first memory to the second cache on the time taken for the content generation model to generate the target content can be ignored.

[0071] If the time taken to generate the first output data from the first model block is less than the time taken to transfer the second weight information from the first memory to the second cache, then the impact of the transfer time taken to transfer the second weight information from the first memory to the second cache on the time taken to generate the target content by the content generation model can also be reduced by covering part of the transfer time.

[0072] In one embodiment, the transmission method for transferring the second weight information from the first memory to the second cache is asynchronous transmission, which can improve the flexibility and reliability of weight information transmission and further reduce the impact of weight information transmission errors on the time consumption and quality of the content generation model in generating target content.

[0073] In one embodiment, after generating first output data using a first model block based on first weight information and target data, the method further includes: deleting the first model block and the first weight information from a first cache.

[0074] In the process of generating target content, the content generation model may use model blocks other than the first and second model blocks. Therefore, to ensure the normal inference of other model blocks, after the first model block generates the first output data, the first model block and the first weight information can be deleted from the first cache to free up space in the first cache. In other words, only two model blocks reside in the second memory for the inference process of the content generation model. This greatly reduces the content generation model's usage of the second memory. Therefore, while ensuring the content generation model can complete inference, it also reduces its usage of the second memory, thus ensuring that graphics cards with limited video memory can perform inference for the content generation model.

[0075] The following describes step 303, namely "using the second model block, based on the second weight information and the first output data, to generate the target content corresponding to the target data", in detail with reference to the embodiments.

[0076] In one embodiment, the first model block and the second model block are both one of at least two model blocks concatenated in the content generation model, and the content generation model further includes a third model block; using the second model block, based on the second weight information and the first output data, to generate target content corresponding to the target data includes: using the second model block, based on the second weight information and the first output data, to generate second output data, and transferring the third weight information of the third model block from the first memory to the first cache area; using the third model block, based on the third weight information and the second output data, to generate target content.

[0077] If the second model block fails to obtain the target content after processing the first output data based on the second weight information, the third model block continues inference. This third model block uses the third weight information to calculate the second output data generated by the second model block, thereby generating the target content. Since the space in the first buffer is released after the first model block generates the first output data, the third weight information of the third model block can be transferred from the first memory to the first buffer during the process of the second model block generating the second output data based on the second weight information and the first output data. This allows the target content to be generated in the first buffer using the third model block based on the third weight information and the second output data.

[0078] Continue to refer Figure 4 ,like Figure 4 As shown in (b), the third weight information (i.e., the weight information of model block 2) can be transferred from the first memory to the first cache, thereby enabling the transfer of the third weight information from the first memory to the first cache. Figure 4 After the second model block shown in (a) processes the first output data based on the second weight information to obtain the second output data, the target content can be generated in the first buffer using the third model block based on the third weight information and the second output data.

[0079] In one optional implementation, if the reasoning process of the content generation model is not completed after the reasoning of the third model block is finished, the space of the second cache can be released, and during the process of the third model block calculating the second output data based on the third weight information, the fourth weight information of the fourth model block is transferred from the first memory to the second cache, so that the output data of the third model block is calculated in the second cache using the fourth model block based on the fourth weight information, until the reasoning process of the content generation model is finished and the target content is output.

[0080] Taking a graphics card with 24GB of video memory as the second memory and 480x832 video data as the target data, the Tflops (trillions of floating-point operations per second) of each model block during the content generation model's inference of the target data on this graphics card can be calculated in the following way:

[0081]

[0082] Where b is the Batch Size, representing the number of samples processed in one forward and backward propagation; L represents the size of the image after encoding by the encoder; h represents the weight dimension in the self-attention module and the cross-attention module; S represents the length of the input text; and ffn_dim represents the weight dimension in the feedforward neural network module.

[0083] Taking b=2, h=5120, L=21*480 / 16*832 / 16=32760, S=512, and ffn_dim=13824 as an example, the model blocks of the content generation model have 84 Tflops, and the weight of each model block is 0.66G. The theoretical peak computing power is 165.2 Tflops. Under the PCIe (Peripheral Component Interconnect Express) 4.0 standard, the bandwidth speed of a 16-channel interface is 32GB / s. Therefore, the theoretically estimated computation time of the model block is 0.51s, while the weight information transmission time is 0.02s. It can be seen that when the content generation model is inferred on a graphics card with 24GB of video memory, the weight information transmission time can be completely masked by the computation process. Therefore, it is possible to infer the content generation model on graphics cards with smaller video memory and poorer communication bandwidth.

[0084] In one embodiment, the transmission method for transferring the third weight information of the third model block from the first memory to the first cache is asynchronous transmission, which can improve the flexibility and reliability of weight information transmission and further reduce the impact of weight information transmission errors on the time consumption and quality of the content generation model in generating target content.

[0085] In one embodiment, each model block includes: a self-attention module, a cross-attention module, and a feedforward neural network module, and the weight information of each model block includes: the weight information of the self-attention module, the weight information of the cross-attention module, and the weight information of the feedforward neural network module.

[0086] In the process of the single-GPU inference content generation model, in each model block of the content generation model, the self-attention module uses its weight information to calculate the target data and obtain the output data of the self-attention module; the cross-attention module uses its weight information to calculate the output data of the self-attention module and obtain the output data of the cross-attention module; and the feedforward neural network module uses its weight information to calculate the output data of the cross-attention module and obtain the target content.

[0087] In one optional implementation, for the multi-GPU inference content generation model process, each GPU-deployed content generation model processes one data block of the target data. For a single GPU, when the self-attention module of each model block of the deployed content generation model calculates the data block using its weight information, it needs to combine the output data of the corresponding model blocks in other GPUs to enhance the representation ability of the content generation model, thereby improving the quality of the generated target content. That is, the content generation models deployed in at least two second memories process at least two data blocks in parallel; the input data of the self-attention module of the target first model block also includes the generated data of the self-attention modules of other first model blocks, which are first model blocks deployed in at least two second memories other than the target second memory; the input data of the self-attention module of the target second model block also includes the generated data of the self-attention modules of other second model blocks, which are second model blocks deployed in at least two second memories other than the target second memory.

[0088] Taking three graphics cards as an example, content generation model 1 is deployed in graphics card 1, content generation model 2 in graphics card 2, and content generation model 3 in graphics card 3. Content generation model 1 includes a first model block A, a second model block B, and a third model block C; content generation model 2 includes a first model block D, a second model block E, and a third model block F; and content generation model 3 includes a first model block G, a second model block H, and a third model block I. When the self-attention module in the first model block A uses its weight information to calculate the data block, it needs to combine the output data of the first model block D and the first model block G. When the self-attention module in the second model block B uses its weight information to calculate the output data of the first model block A, it needs to combine the output data of the second model block E and the second model block H. When the self-attention module in the third model block C uses its weight information to calculate the output data of the second model block B, it needs to combine the output data of the third model block F and the third model block I, ultimately obtaining the content block generated by content generation model 1.

[0089] In one embodiment, there are at least two second memories, each of which is equipped with a content generation model. The first memory stores the weight information of each model block in each content generation model. Generating target content corresponding to target data includes: dividing the target data based on the number of second memories to obtain at least two data blocks that correspond one-to-one with the at least two second memories; for each of the at least two data blocks, obtaining the target first weight information of the target first model block deployed in the target second memory corresponding to the data block from the first memory; using the target first model block, generating third output data based on the target first weight information and the data block, and asynchronously obtaining the target second weight information of the target second model block deployed in the target second memory from the first memory; using the target second model block, generating a content block corresponding to the data block based on the target second weight information and the third output data; and fusing the content blocks corresponding to each of the at least two data blocks to obtain the target content.

[0090] While a single second memory can ensure normal inference of the content generation model during the generation of a single video (i.e., single-GPU inference of the content generation model), the latency of generating a single video cannot be improved. Therefore, multiple second memories can be used for inference of the content generation model (i.e., multi-GPU inference of the content generation model), thereby reducing the latency of generating a single video. Thus, the target data can be divided according to the number of second memories, resulting in at least two data blocks corresponding one-to-one with at least two second memories. The content generation model deployed in each second memory processes one data block, obtaining the corresponding content block. Then, each content block is merged to obtain the target content.

[0091] Optionally, the target data can be segmented according to the sequence dimension of the target data.

[0092] Figure 5 A reasoning diagram for another content generation model provided in this application, such as Figure 5 As shown, the second memory unit is 3 (that is, 3 graphics cards). Figure 5 Taking second memory A, second memory B, and second memory C as an example, the target data can be divided into three databases (in...). Figure 5 (Shown as data block A, data block B, and data block C), the first memory stores the content generation model deployed in each of the second memories (in... Figure 5 The image shows the weight information of N model blocks in content generation model A, content generation model B, and content generation model C. Figure 5The weight information is shown in the diagram as the weight information of model block 0, model block 1, model block 2, ..., model block N-1. Each second memory includes a first cache and a second cache. The content deployed in each second memory generates the first model block of the model (in...). Figure 5 Model block 0 is deployed in the first cache, and the second model block (in...) is deployed in the second cache. Figure 5 The first weight information (i.e., the weight information of model block 0) of each first model block can be transferred from the first memory to the first cache. Then, in the first cache of the second memory, the first model block generates third output data based on the first weight information and the corresponding data block. Asynchronously, the second weight information (i.e., the weight information of model block 1) of each second model block is transferred from the first memory to the second cache. Then, in the second cache of the second memory, the second model block generates the content block corresponding to each data block based on the second weight information and the third output data. Figure 5 The content blocks are shown as content block A, content block B, and content block C. Then, the merging module is used to merge each content block to obtain the target content.

[0093] In one embodiment, during the multi-GPU inference content generation model, the transmission of weight information competes for bandwidth with the communication between the GPU and the weight information transmission process. Therefore, it is necessary to reduce the granularity of weight information transmission to mitigate the impact of bandwidth contention between the weight information transmission process and the GPU on the target content generated by the content generation model. Specifically, weight information is transmitted separately for each model block containing the self-attention module, cross-attention module, and feedforward neural network module.

[0094] For example, during the process of the self-attention module of the first model block calculating the corresponding data block based on its weight information, the weight information of the self-attention module of the second model block is asynchronously obtained from the first memory. During the process of the cross-attention module and the feedforward neural network module of the first model block calculating the output data of the self-attention module based on its weight information, the weight information of the cross-attention module and the feedforward neural network module of the second model block is asynchronously obtained from the first memory.

[0095] For example, during the process of the self-attention module of the first model block calculating the corresponding data block based on its weight information, the weight information of the self-attention module of the second model block is asynchronously retrieved from the first memory. During the process of the cross-attention module of the first model block calculating the output data of the self-attention module based on its weight information, the weight information of the cross-attention module of the second model block is asynchronously retrieved from the first memory. During the process of the feedforward neural network module of the first model block calculating the output data of the cross-attention module based on its weight information, the weight information of the feedforward neural network module of the second model block is asynchronously retrieved from the first memory.

[0096] In one optional implementation, using the target first model block, based on the target first weight information and data blocks, a third output data is generated, and the target second weight information of the target second model block deployed in the target second memory is asynchronously obtained from the first memory. This includes: using the self-attention module of the target first model block, based on the weight information of the self-attention module of the target first model block and data blocks, a fourth output data is generated, and the weight information of the self-attention module of the target second model block is asynchronously obtained from the first memory; using the cross-attention module of the target first model block, based on the weight information of the cross-attention module of the target first model block and the fourth output data, a fifth output data is generated; using the feedforward neural network module of the target first model block, based on the weight information of the feedforward neural network module of the target first model block and the fifth output data, a third output data is generated, and the weight information of the cross-attention module and the weight information of the feedforward neural network module of the target second model block are asynchronously obtained from the first memory.

[0097] For the target first model block deployed in the target second memory, the data block corresponding to the target second memory is processed. After obtaining the target first weight information from the first memory, the target first model block can use the target first weight information to calculate the data block, thereby obtaining the generated data of the target first model block, which is the third output data.

[0098] Further, continue to refer to Figure 5 Taking the target second memory as second memory A as an example, then the target first model block is the first model block of content generation model A (that is... Figure 5 The self-attention module of the first model block of the content generation model A in the second memory A, based on its weight information and combined with the output data of the self-attention modules of the first model blocks of other content generation models, calculates the data block A corresponding to the second memory A to generate the fourth output data. In order to reduce latency, the second model block of the content generation model A can be asynchronously retrieved from the first memory during the generation of the fourth output data (i.e., ... Figure 5The weight information of the self-attention module of model block 1) in the second memory A is transmitted to the second buffer in the second memory A for caching; the cross-attention module of the first model block of the content generation model A calculates the fourth output data based on its weight information to generate the fifth output data, and the feedforward neural network module of the first model block of the content generation model A calculates the fifth output data based on its weight information to generate the generated data of the first model block of the content generation model A, that is, the third output data. In order to reduce latency, the weight information of the cross-attention module and the feedforward neural network module of the second model block of the content generation model A can be asynchronously obtained from the first memory and transmitted to the second buffer in the second memory A for caching.

[0099] In one embodiment, using the target second model block, based on the target second weight information and the third output data, to generate a content block corresponding to the data block, includes: using the self-attention module of the target second model block, based on the weight information of the self-attention module of the target second model block and the third output data, to generate sixth output data; using the cross-attention module of the target second model block, based on the weight information of the cross-attention module of the target second model block and the sixth output data, to generate seventh output data; and using the feedforward neural network module of the target second model block, based on the weight information of the feedforward neural network module of the target second model block and the seventh output data, to generate a content block.

[0100] After the first target model block generates the third output data, the second target model block can calculate the third output data based on the second target weight information, thereby generating the output data of the content generation model, which is the content block A corresponding to data block A.

[0101] Further, continue to refer to Figure 5 Continuing with the example of the second memory being the target second memory A, the target second model block is the second model block of the content generation model A (that is...). Figure 5 In the second memory A, model block 1), the self-attention module of the second model block of content generation model A calculates the output data of the first model block of content generation model A, i.e., the third output data, based on its weight information in the second cache and the output data of the self-attention modules of the second model blocks of other content generation models, to generate the sixth output data; the cross-attention module of the second model block of content generation model A calculates the sixth output data based on its weight information in the second cache to generate the seventh output data; and the feedforward neural network module of the second model block of content generation model A calculates the seventh output data based on its weight information in the second cache to generate the output data of content generation model A, i.e., the content block A corresponding to data block A.

[0102] Optionally, if inference cannot be completed using only two model blocks (i.e., the first model block and the second model block) during the process of content generation model corresponding to each second memory inference, other model blocks (e.g., the third model block, the fourth model block, the fifth model block, etc.) can be added until the content generation model inference is completed and the corresponding content block is generated.

[0103] For example, after the first model block of content generation model A generates the third output data, the space of the first cache area in the second memory A is released; during the process of generating the sixth output data, the self-attention module of the second model block of content generation model A asynchronously obtains the weight information of the self-attention module of the third model block of content generation model A from the first memory and transmits it to the first cache area in the second memory A for caching; during the process of the cross-attention module and the feedforward neural network module of the second model block of content generation model A processing the sixth output data, they asynchronously obtain the weight information of the cross-attention module and the feedforward neural network module of the third model block of content generation model A from the first memory and transmit it to the first cache area in the second memory A for caching. After the second model block of content generation model A completes its inference, the self-attention module of the third model block of content generation model A calculates the output data of the second model block of content generation model A based on its weight information in the first buffer and the output data of the self-attention modules of the third model blocks of other content generation models, generating the eighth output data; the cross-attention module of the third model block of content generation model A calculates the eighth output data based on its weight information in the first buffer, generating the ninth output data; and the feedforward neural network module of the third model block of content generation model A calculates the ninth output data based on its weight information in the first buffer, generating the output data of content generation model A, that is, the content block A corresponding to data block A.

[0104] In one alternative implementation, the release of space in the cache area of ​​the second memory can also be processed asynchronously. For example, during the model block inference process in the second cache area, the storage space of the first cache area can be released asynchronously, and during the model block inference process in the first cache area, the storage space of the second cache area can be released asynchronously, thereby further reducing the latency of generating the target content.

[0105] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0106] According to another embodiment, a content generation apparatus is provided. Figure 6 This is a schematic block diagram of a content generation apparatus provided in an embodiment of this application. The apparatus is disposed in... Figure 2 User equipment in the architecture shown. Figure 6 As shown, the device 600 mainly includes: an acquisition unit 601, a first generation unit 602, and a second generation unit 603. The main functions of each component are as follows:

[0107] The acquisition unit 601 is configured to acquire the first weight information of the first model block from the first memory.

[0108] The first generation unit 602 is configured to generate first output data using a first model block, based on first weight information and target data, and to obtain second weight information of a second model block from a first memory.

[0109] The second generation unit 603 is configured to generate target content corresponding to the target data based on the second weight information and the first output data using the second model block; wherein the first model block and the second model block are both deployed in the second memory.

[0110] As one possible implementation, the second memory includes a first cache area and a second cache area, with the first model block deployed in the first cache area and the second model block deployed in the second cache area; when the acquisition unit 601 acquires the first weight information of the first model block from the first memory, it is also configured to transfer the first weight information from the first memory to the first cache area; when the first generation unit 602 acquires the second weight information of the second model block from the first memory, it is also configured to transfer the second weight information from the first memory to the second cache area.

[0111] As one possible implementation, the device 600 further includes a deletion unit (not shown in the figure), configured to delete the first model block and the first weight information in the first buffer after generating first output data based on the first weight information and the target data using the first model block.

[0112] As one possible implementation, the first model block and the second model block are both one of at least two model blocks in a content generation model that are connected in series. The content generation model is deployed in the second memory and also includes a third model block. The second generation unit 603 is further configured to use the second model block to generate second output data based on the second weight information and the first output data, and to transfer the third weight information of the third model block from the first memory to the first cache. The third model block is used to generate target content based on the third weight information and the second output data.

[0113] Furthermore, the transmission method for transferring the second weight information from the first memory to the second cache, and / or the transmission method for transferring the third weight information of the third model block from the first memory to the first cache, is asynchronous transmission.

[0114] As one possible implementation, there are at least two second memories, each of which is equipped with a content generation model. The first memory stores the weight information of each model block in each content generation model. When generating target content corresponding to the target data, the second generation unit 603 is also configured to segment the target data based on the number of second memories to obtain at least two data blocks that correspond one-to-one with the at least two second memories. For each of the at least two data blocks, the first target weight information of the target first model block deployed in the target second memory corresponding to the data block is obtained from the first memory. Using the target first model block, based on the target first weight information and the data block, third output data is generated, and the second target weight information of the target second model block deployed in the target second memory is obtained asynchronously from the first memory. Using the target second model block, based on the target second weight information and the third output data, the content block corresponding to the data block is generated. The content blocks corresponding to each of the at least two data blocks are fused to obtain the target content.

[0115] As one possible implementation, each model block includes: a self-attention module, a cross-attention module, and a feedforward neural network module. The weight information of each model block includes: the weight information of the self-attention module, the weight information of the cross-attention module, and the weight information of the feedforward neural network module. The second generation unit 603, while using the target first model block, based on the target first weight information and the data block, to generate third output data, and asynchronously obtaining the target second weight information of the target second model block deployed in the target second memory from the first memory, is also configured to utilize the self-attention module of the target first model block, based on the self-attention of the target first model block... The system uses the weight information and data blocks of the force module to generate fourth output data, and asynchronously retrieves the weight information of the self-attention module of the target second model block from the first memory. Using the cross-attention module of the target first model block, based on the weight information of the cross-attention module of the target first model block and the fourth output data, it generates fifth output data. Using the feedforward neural network module of the target first model block, based on the weight information of the feedforward neural network module of the target first model block and the fifth output data, it generates third output data, and asynchronously retrieves the weight information of the cross-attention module and the weight information of the feedforward neural network module of the target second model block from the first memory.

[0116] As one possible implementation, when the second generation unit 603 generates the content block corresponding to the data block using the target second model block and based on the target second weight information and the third output data, it is also configured to generate the sixth output data using the self-attention module of the target second model block and based on the weight information of the self-attention module of the target second model block and the third output data; generate the seventh output data using the cross-attention module of the target second model block and based on the weight information of the cross-attention module of the target second model block and the sixth output data; and generate the content block using the feedforward neural network module of the target second model block and based on the weight information of the feedforward neural network module of the target second model block and the seventh output data.

[0117] Furthermore, the content generation models deployed in at least two second memories process at least two data blocks in parallel; the input data of the self-attention module of the target first model block also includes the generated data of the self-attention modules of other first model blocks, which are first model blocks deployed in at least two second memories other than the target second memory; the input data of the self-attention module of the target second model block also includes the generated data of the self-attention modules of other second model blocks, which are second model blocks deployed in at least two second memories other than the target second memory.

[0118] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. The system and device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0120] In addition, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the steps of any of the methods in the foregoing method embodiments.

[0121] And an electronic device, comprising:

[0122] One or more processors; and

[0123] A memory associated with one or more processors, the memory being used to store program instructions that, when read and executed by one or more processors, perform the steps of any of the methods in the foregoing method embodiments.

[0124] This application also provides a computer program product, including a computer program that, when executed, implements the steps of any of the methods in the foregoing method embodiments.

[0125] in, Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Specifically, it may include a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, and a memory 720. The processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, and memory 720 can be communicatively connected via a communication bus 730. The input / output interface 713 can also be referred to as an I / O interface 713.

[0126] The processor 710 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0127] The memory 720 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 720 can store the operating system 721 for controlling the operation of the electronic device 700, and the basic input / output system (BIOS) 722 for controlling the low-level operations of the electronic device 700. Additionally, it can store a web browser 723, a data storage management system 724, and a content generation device 600, etc. The aforementioned content generation device 600 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 720 and is called and executed by the processor 710.

[0128] Input / output interface 713 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0129] Network interface 714 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0130] Bus 730 includes a pathway for transmitting information between various components of the device, such as processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, and memory 720.

[0131] It should be noted that although the above-described device only shows the processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, memory 720, bus 730, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0132] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0133] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A content generation method, characterized in that, The content generation method includes: Obtain the first weight information of the first model block from the first memory; Using the first model block, based on the first weight information and the target data, generate the first output data, and obtain the second weight information of the second model block from the first memory; Using the second model block, based on the second weight information and the first output data, the target content corresponding to the target data is generated; Both the first model block and the second model block are deployed in the second memory.

2. The content generation method according to claim 1, characterized in that, The second memory includes a first cache area and a second cache area, the first model block is deployed in the first cache area, and the second model block is deployed in the second cache area; The step of obtaining the first weight information of the first model block from the first memory includes: transferring the first weight information from the first memory to the first cache area; The step of obtaining the second weight information of the second model block from the first memory includes: transferring the second weight information from the first memory to the second cache.

3. The content generation method according to claim 2, characterized in that, After generating the first output data using the first model block based on the first weight information and the target data, the content generation method further includes: Delete the first model block and the first weight information in the first cache area.

4. The content generation method according to claim 3, characterized in that, The first model block and the second model block are both one of at least two model blocks in a chain in the content generation model, the content generation model is deployed in the second memory, and the content generation model also includes a third model block; The step of generating target content corresponding to the target data using the second model block, based on the second weight information and the first output data, includes: Using the second model block, based on the second weight information and the first output data, generate the second output data, and transfer the third weight information of the third model block from the first memory to the first cache area; The target content is generated using the third model block, based on the third weight information and the second output data.

5. The content generation method according to claim 4, characterized in that, The transmission method for transferring the second weight information from the first memory to the second cache area, and / or the transmission method for transferring the third weight information of the third model block from the first memory to the first cache area, is asynchronous transmission.

6. The content generation method according to any one of claims 1 to 4, characterized in that, The number of second memory is at least two, and each second memory is used to deploy a content generation model. The first memory stores the weight information of each model block in each content generation model. The generation of the target content corresponding to the target data includes: Based on the number of the second memory, the target data is divided to obtain at least two data blocks that correspond one-to-one with at least two second memory blocks; For each of the at least two data blocks, obtain the target first weight information of the target first model block deployed in the target second memory corresponding to the data block from the first memory; using the target first model block, based on the target first weight information and the data block, generate third output data, and asynchronously obtain the target second weight information of the target second model block deployed in the target second memory from the first memory; using the target second model block, based on the target second weight information and the third output data, generate the content block corresponding to the data block; The content blocks corresponding to each of the at least two data blocks are merged to obtain the target content.

7. The content generation method according to claim 6, characterized in that, Each model block includes: a self-attention module, a cross-attention module, and a feedforward neural network module. The weight information of each model block includes: the weight information of the self-attention module, the weight information of the cross-attention module, and the weight information of the feedforward neural network module. The step of generating third output data based on the target first model block, the target first weight information, and the data block, and asynchronously obtaining the target second weight information of the target second model block deployed in the target second memory from the first memory, includes: Using the self-attention module of the first target model block, based on the weight information of the self-attention module of the first target model block and the data block, fourth output data is generated, and the weight information of the self-attention module of the second target model block is asynchronously obtained from the first memory; Using the cross-attention module of the first target model block, and based on the weight information of the cross-attention module of the first target model block and the fourth output data, fifth output data is generated. Using the feedforward neural network module of the first target model block, and based on the weight information of the feedforward neural network module of the first target model block and the fifth output data, third output data is generated. The weight information of the cross-attention module and the weight information of the feedforward neural network module of the second target model block are asynchronously obtained from the first memory.

8. The content generation method according to claim 7, characterized in that, The step of generating a content block corresponding to the data block using the target second model block, based on the target second weight information and the third output data, includes: Using the self-attention module of the target second model block, and based on the weight information of the self-attention module of the target second model block and the third output data, a sixth output data is generated; Using the cross-attention module of the target second model block, and based on the weight information of the cross-attention module of the target second model block and the sixth output data, the seventh output data is generated; The content block is generated using the feedforward neural network module of the target second model block, based on the weight information of the feedforward neural network module of the target second model block and the seventh output data.

9. The content generation method according to claim 8, characterized in that, The content generation models deployed in the at least two second memories process the at least two data blocks in parallel. The input data of the self-attention module of the target first model block also includes the generated data of the self-attention modules of other first model blocks, wherein the other first model blocks are the first model blocks deployed in the other second memory besides the target second memory in the at least two second memory; The input data of the self-attention module of the target second model block also includes the generated data of the self-attention modules of other second model blocks, wherein the other second model blocks are second model blocks deployed in other second memories besides the target second memory in the at least two second memories.

10. A content generation apparatus, characterized in that, The content generation device includes: The acquisition unit is configured to acquire the first weight information of the first model block from the first memory; The first generation unit is configured to use the first model block to generate first output data based on the first weight information and target data, and to obtain the second weight information of the second model block from the first memory; The second generation unit is configured to use the second model block to generate target content corresponding to the target data based on the second weight information and the first output data; Both the first model block and the second model block are deployed in the second memory.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the content generation method according to any one of claims 1 to 9.

12. An electronic device, characterized in that, include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the content generation method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed, it implements the steps of the content generation method according to any one of claims 1 to 9.

Images