Content generation method, apparatus, device, medium, and product

Abstract

This paper provides a content generation method, apparatus, device, medium, and product applied in the computer field. The method includes: acquiring first data; processing the first data based on a first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing a second feature vector based on first compression information, the first compression information being determined based on the second feature vector, the first compression information including multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; acquiring second data; processing the first feature vector and the second data based on the first model to obtain the content. This paper solves the problem of excessive cache memory usage, and the corresponding beneficial effect is to reduce memory usage while improving model inference efficiency.

Description

Technical Field

[0001] This article relates to the field of computer processing technology, and in particular to a content generation method, apparatus, device, medium and product. Background Technology

[0002] Currently, data reasoning and generation based on models suffer from problems such as high memory consumption and slow reasoning speed. Summary of the Invention

[0003] This paper provides a content generation method, apparatus, device, medium, and product that solves the problems of slow model inference speed and large memory consumption in the prior art, and achieves the effect of improving model inference efficiency while ensuring the accuracy of model inference.

[0004] Firstly, a content generation method, the method comprising: Get the first data; The first data is processed based on the first model to obtain a first feature vector. The first feature vector is obtained by compressing the second feature vector based on the first compression information. The first compression information is determined based on the second feature vector and includes multiple different compression parameters. The compression parameters are used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector. Obtain the second data, and process the first feature vector and the second data based on the first model to obtain the content.

[0005] Secondly, a content generation apparatus, the apparatus comprising: The media data acquisition module is used to acquire the initial data. The feature compression module is used to process the first data based on the first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing the second feature vector based on the first compression information, the first compression information is determined based on the second feature vector, and the first compression information includes multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; The content generation module is used to acquire second data, process the first feature vector and the second data based on the first model, and obtain the content.

[0006] Thirdly, an electronic device comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement any of the content generation methods described above.

[0007] Fourthly, a storage medium containing computer-executable instructions, which, when executed by a computer processor, cause the processor to perform any of the content generation methods described above.

[0008] Fifthly, a computer program product includes a computer program that, when executed by a processor, causes the processor to perform any of the content generation methods described above.

[0009] The beneficial effects of the above method are as follows: First data is acquired; the first data is processed based on a first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing a second feature vector based on first compression information, the first compression information being determined based on the second feature vector, and the first compression information including multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; second data is acquired, and the first feature vector and second data are processed based on the first model to obtain content. This solves the problems of slow model inference speed and large memory consumption in the prior art, achieving the effect of improving model inference efficiency while ensuring model inference accuracy.

[0010] It should be understood that the descriptions in this section are not intended to identify key or important features of this document, nor are they intended to limit its scope. Other features of this document will become readily apparent from the following description. Attached Figure Description

[0011] The above and other features, advantages, and aspects of this document will become more apparent when viewed in conjunction with the accompanying drawings and the following specific implementation process. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0012] Figure 1 This is a schematic diagram of an exemplary system architecture that can be applied to this paper; Figure 2 This is a flowchart illustrating a content generation method presented in this paper. Figure 3 A flowchart illustrating another content generation method presented in this article; Figure 4 This is a flowchart illustrating another content generation method presented in this article; Figure 5 This is a flowchart illustrating another content generation method presented in this article; Figure 6 A schematic diagram of the structure of a content generation device provided in this article; Figure 7 This is a schematic diagram of the structure of an electronic device provided in this article. Detailed Implementation

[0013] The present invention will now be described in more detail with reference to the accompanying drawings. While some implementations of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the implementations described herein. Rather, these implementations are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and implementations are for illustrative purposes only and are not intended to limit the scope of this invention.

[0014] It should be understood that the steps described in the method herein may be performed in different orders and / or in parallel. Furthermore, implementations of the method may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.

[0015] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". Definitions of other terms are given below.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this article are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "one" and "more" used in this document are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments herein are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0019] It is understandable that before using this article, users should be informed of the type, scope of use, and usage scenarios of the personal information involved in this article in an appropriate manner in accordance with relevant laws and regulations, and their authorization should be obtained.

[0020] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as electronic devices, applications, servers, or storage media, that perform the operations described herein, based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation method described in this article. Other methods that comply with relevant laws and regulations may also be applied to the implementation method described in this article.

[0023] It is understood that the data involved in this article (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] It should be noted that the collection, updating, analysis, processing, use, transmission, and storage of user personal information described herein comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security. The collection, updating, analysis, processing, use, transmission, and storage of user personal information described herein are all conducted with the user's knowledge and consent, and comply with relevant privacy protection regulations.

[0025] In this paper, the application scenarios of this technical solution can be illustrated by example: This solution can be applied to any scenario that requires processing long sequences and compressing and caching the resulting feature vectors so that the compressed and cached feature vectors can be reused after autoregression. By determining the compression parameters corresponding to different sub-feature vectors in the feature vector, the corresponding sub-feature vectors can be compressed, thereby achieving the goal of reusing the compressed and cached feature vectors during the autoregressive processing of the first model. This not only reduces cache memory but also improves processing efficiency. In the case where the first model consists of multiple network layer processing modules, each network layer processing module can perform the step of determining the first feature vector.

[0026] Before detailing the technical solution, let's first explain the basic architecture and core processing logic of the first model involved. The first model can employ an autoregressive neural network model, whose basic structure consists of multiple sequentially connected network layer processing modules stacked together. Each network layer processing module mainly contains two core sub-layers: a multi-head self-attention sub-layer and a feedforward neural network sub-layer. These two sub-layers work together to enable the first model to perform deep semantic modeling on the input sequence data (e.g., text or images) and generate output content unit by unit.

[0027] The complete data input to the first model (such as a whole sentence, a complete image, or a complete audio clip) is first transformed into an ordered sequence representation, with each unit being the basic element constituting this sequence. Specifically, the original input data is segmented into a series of discrete basic processing units according to its modality: for text, each unit typically corresponds to a word; for images, each unit typically corresponds to an image patch; and for audio, each unit typically corresponds to a time frame. These units are arranged in the order they appear in the original input, collectively forming the input sequence. All subsequent processing by the first model, including embedding representation, positional encoding, and self-attention computation, is performed unit-by-unit at the level of this sequence composed of each unit.

[0028] To understand the implementation process of key-value caching compression in the subsequent schemes of this paper, we can first clarify the three basic concepts in the self-attention mechanism: query, key, and value, which are usually collectively referred to as "Q, K, V". During the self-attention computation, for each input unit, the first model generates corresponding query vector, key vector, and value vector through linear transformations. The query represents the current unit inquiring about contextual information, expressing what the current unit "wants to focus on." The key serves as an index identifier for all units in the sequence, used to match with the query to measure the relevance between the current unit and each context unit. The value carries the actual semantic content of the unit; once the units that need attention are determined through matching the query and key, the first model extracts the values ​​of these units and performs weighted aggregation. Through the interactive computation between query, key, and value, the self-attention mechanism can dynamically aggregate contextual information from other related units for each unit in the sequence, thereby capturing long-distance dependencies.

[0029] In the autoregressive generation process of the first model, since each newly generated unit is only related to the K and V of previously generated units, and the K and V of previous units remain unchanged and are reused in subsequent generation rounds, K and V caching is widely used to eliminate redundant computation and improve inference efficiency. This method caches the K and V of already generated units for use in subsequent generation steps. These cached K and V values ​​constitute the uncompressed second key-value information in this paper.

[0030] However, as the length of the generated sequence increases, the amount of cached key and value data also grows linearly, leading to excessive memory consumption and becoming a major bottleneck in long context deployments. This paper addresses the issue of key and value cache size by eliminating solutions that perform cache compression on key and value data.

[0031] Figure 1 A schematic diagram of an exemplary system architecture to which this article can be applied is shown. References Figure 1 The system 100 includes a client 101, a network 102, and a server 103.

[0032] Client 101 serves as the terminal for user operation and interaction, supporting both server-side and local deployment for standalone execution of the model. This client is responsible for receiving user input instructions, such as initiating content generation requests, and simultaneously displaying the generated media data results (i.e., the final output content in this document) to the user. Client 101 can be various devices with computing and display capabilities, such as smartphones, tablets, and personal computers.

[0033] Network 102, serving as the data transmission channel between client 101 and server 103, is responsible for transmitting content generation requests initiated by client 101 to server 103, and simultaneously transmitting the second media data generated and processed by server 103 back to client 101, ensuring accurate and timely data transmission between client 101 and server 103. Network 102 can be in various forms, such as wired or wireless networks.

[0034] Server 103 is the core processing unit of the entire system, containing both a CPU and a GPU. The CPU is responsible for overall task scheduling and resource management, coordinating the work between various components. For example, it receives requests from the network, parses and performs preliminary processing on the requests, and allocates relevant computational tasks to the GPU. The GPU possesses powerful parallel computing capabilities and plays a crucial role in the content generation process. When server 103 receives a content generation request from client 101, the CPU transmits the first media data to the GPU. The GPU processes the first data using a pre-trained first model. This first model incorporates a hybrid dimension compression mechanism for key-value caching during inference. By assigning differentiated compression parameters to different sub-key-value information, it reduces memory consumption while maintaining the model's generation performance. After completing the computation, the GPU returns the generated content to the CPU, which then sends the result back to client 101 via network 102 for user viewing and use. Through the collaborative work of client 101, network 102, and server 103, an efficient and low-memory-consumption content generation service is achieved in long-context scenarios.

[0035] Figure 2 This is a flowchart illustrating a content generation method applicable to scenarios where key-value (KV) information is compressed according to determined compression parameters during the generation of relevant content. This method can be executed by a content generation device, which can be implemented in software and / or hardware, or optionally, through an electronic device, such as a client.

[0036] like Figure 2 As shown, the content generation method may specifically include: S210, Obtain the first data.

[0037] The first data can be input information applicable to various content generation fields, such as various types of media data including text, images, audio, and video. For example, assuming the first data is text, it can include the beginning of an article to be continued, sentences to be translated, or questions to be answered; assuming the first data is an image, it can include images to be repaired, visual images used to generate text descriptions, or paintings to be converted in style.

[0038] S220. Process the first data based on the first model to obtain the first feature vector.

[0039] The first feature vector is obtained by compressing the second feature vector based on the first compression information. The first compression information is determined based on the second feature vector and includes multiple different compression parameters. These compression parameters indicate the compression ratio of at least some sub-feature vectors in the second feature vector. The second feature vector is the feature vector obtained when the first model first processes the first data.

[0040] The first model is used to generate output content associated with the input data. Optionally, the first model may include multiple sequentially connected network layer processing modules, which are sequentially connected constituent units in the first model used for processing data layer by layer. The network layer processing modules are used to cache the first feature vector after data compression.

[0041] The first feature vector is the key-value information obtained by compressing the second feature vector based on the obtained first compression information. The second feature vector is used to characterize the key-value information obtained by the first model after generating the first data.

[0042] The first compression information is compression guidance information determined based on the second feature vector itself. The first compression information includes multiple different compression parameters, each indicating the compression ratio applied to at least a portion of the sub-feature vectors in the second feature vector during the compression process. This compression ratio determines the amount of information retained by the corresponding sub-feature vector after compression. Sub-feature vectors are components obtained after partitioning the second feature vector, each corresponding to a local vector segment within the second feature vector. During compression, different compression parameters in the first compression information are applied to different sub-feature vectors, allowing each sub-feature vector to be independently compressed according to its corresponding compression ratio.

[0043] In this paper, the sub-feature vectors in the second feature vector can be understood as: the feature vector corresponding to each data unit after the first model processes the first data. A data unit can be understood as the token information corresponding to the first data.

[0044] In this paper, a first model can be used to process the first data to generate a first feature vector. During the generation process, the first model internally processes the first data to obtain a second feature vector and acquires the first compression information used to compress the second feature vector. Subsequently, the relevant server or processor can compress the second feature vector it generates based on the first compression information to obtain the compressed first feature vector, which is then cached.

[0045] S220. Obtain the second data. Based on the first model, process the first feature vector and the second data to obtain the content.

[0046] The second data is the data obtained from the autoregressive processing of the first model.

[0047] The output of the first model includes N second data points concatenated based on the output order. The first second data point is determined by the first model based on the semantic features of the first second data point. The i-th second data point is determined by the first model based on the semantic features of the first second data point to the (i-1)-th second data point. 1 < i ≤ N and N is a natural number.

[0048] In this paper, the second data obtained from the autoregressive processing of the first model can be obtained. Then, the first feature vector and the second data are processed based on the first model to obtain the content. In the autoregressive processing, when the first model processes and determines the i-th second data, it calls and uses the cached first feature vector to participate in the calculation. The first model analyzes the semantic features of the first data to obtain the first second data. For any i-th second data from the second second data to the Nth second data, the first model performs a comprehensive analysis of the semantic features from the first second data to the i-th minus one second data to complete the process. Finally, the content is obtained when all the second data has been processed.

[0049] It is understandable that the first feature vector in this paper can be a feature vector that is only related to the first data; or it can be a feature vector that is jointly determined by the first data and the second data.

[0050] The beneficial effects of the above technical solution are: acquiring first data; processing the first data based on a first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing a second feature vector based on first compression information, the first compression information being determined based on the second feature vector, the first compression information including multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; acquiring second data, processing the first feature vector and the second data based on the first model to obtain content, thus solving the problems of slow model inference speed and large memory consumption in the prior art, and achieving the effect of improving model inference efficiency while ensuring the accuracy of model inference.

[0051] Figure 3 This is a flowchart illustrating a content generation method provided in this paper. Based on the above scheme, the process for determining the first key-value information can also be explained; its specific implementation can be found in the detailed description herein. Technical features that are the same as or similar to those described above will not be repeated here.

[0052] like Figure 3 As shown, the content generation method of this article may specifically include: S310, Obtain the first data.

[0053] S320. Obtain the third feature vector of the second data based on the first model.

[0054] The third feature vector refers to the feature representation obtained after processing the second data based on the first model, used to characterize the semantic or structural information of the second data. For example, the third feature vector refers to the item value information of the second data. The third feature vector can be newly generated key-value information based on the second data.

[0055] In this paper, the first model generates the second data autoregressively while processing the first data; then, the second data is processed based on the first model, and the corresponding key-value information is generated for the second data under the self-attention mechanism of the first model. This key-value information is the third feature vector.

[0056] S330. Obtain the compression parameters of the third feature vector, and compress the third feature vector based on the compression parameters to update the first feature vector.

[0057] The compression parameter of the third feature vector refers to the compression guidance information determined based on the third feature vector itself. This compression parameter is used to indicate the compression ratio applied when compressing the third feature vector.

[0058] In this paper, the compression parameters of the third feature vector can be obtained, and the third feature vector can be compressed based on the compression parameters. By transforming the compression processing mechanism in the model, the data dimension or storage size of the third feature vector is reduced while maintaining key information. Finally, the compressed result is used to update the first feature vector, so that the first feature vector can continuously accumulate key value information from the historical second data in compressed form during the autoregressive process.

[0059] This can be understood as follows: during the autoregressive processing of the first model, after each acquisition of the second data to determine its third eigenvector, the compression ratio of the third eigenvector is re-determined. The method for determining the compression ratio is the same as that for determining the compression ratio of the second eigenvector. It should also be noted that when determining the third eigenvector of the second data, the key-value (KV) information of the second data can be directly determined, and combined with the cached KV information to determine the output content.

[0060] S340. Based on the third feature vector and the first feature vector, obtain the content.

[0061] The first input data includes at least the first media data or the second output data of the previous network layer processing module, and the first output data is the second input data input to the next network layer processing module or the first sub-media data output.

[0062] The content generation method presented in this paper, when processing the first feature vector and the second data based on the first model, obtains the third feature vector of the second data based on the first model, thereby effectively extracting the features of the second data in the current generation step; then, by obtaining the compression parameters of the third feature vector and compressing the third feature vector based on the compression parameters to update the first feature vector, the first feature vector can dynamically integrate newly generated feature information during the autoregressive processing, effectively reducing the storage resources occupied by historical features and the computational complexity in subsequent steps while maintaining the continuous transmission of key information.

[0063] In this paper, the first feature vector can be obtained in two ways: the first method aims to keep the first feature vector associated with the first data unchanged; the second method aims to change the first feature vector as the output of the second data changes during the first model's autoregressive processing.

[0064] The following section elaborates on two methods for determining the first eigenvector: The first approach: When the first model processes the first data for the first time, the second feature vector corresponding to the first data can be determined, which is the KV information obtained directly from the processing. By analyzing and processing the second feature vector, compression parameters for compressing each sub-feature vector in the second feature vector can be obtained. Then, each sub-feature vector in the second feature vector is compressed according to the compression parameters to obtain the first feature vector, which is the compressed KV information.

[0065] In the autoregression process, after obtaining the second data, the key-value information corresponding to the second data can be determined based on the first model, and the first feature vector can be reused. That is, in the autoregression process, the first feature vector determined initially remains unchanged.

[0066] The second approach involves processing the first data using the first model to output the first second data point and determine its corresponding first feature vector, i.e., key-value (KV) information. After obtaining the first second data point, both the second and first data points can be input into the first model to obtain the third feature vector (KV information) of the first second data point. The cached first and third feature vectors are then used to obtain the second second data point.

[0067] The compression parameters of the third feature vector can be determined based on the first and third feature vectors, and compression can be performed based on the compression parameters to update the first compression information. Alternatively, the compression parameters of the third feature vector can be determined based on the second and third feature vectors corresponding to the first feature vector, and compression can be performed to update the first compression information.

[0068] During the autoregression process, the first feature vector can be updated in the manner described above until the autoregression stops and the content is obtained.

[0069] Figure 4 This is a flowchart illustrating a content generation method provided in this paper. Based on the above scheme, the process of determining the first feature vector can also be explained; its specific implementation can be found in the detailed description herein. Technical features that are the same as or similar to those described above will not be repeated here.

[0070] like Figure 4 As shown, the content generation method of this article may specifically include: S410, Obtain the first data.

[0071] S420. Process the first data based on the first model to obtain the second feature vector.

[0072] In this paper, during the processing of first data by the first model, the first data can be used as the first input data. In the example where the first model includes a processing model with multiple network layers, the first input data includes at least the first data or the second output data of the previous network layer processing module, and the second feature vector is the second input data input to the next network layer processing module or the first second data output.

[0073] Specifically, the content of the first input data depends on the current processing stage of the network layer module within the entire first model: In the first scenario, if the current network layer processing module is the first network layer processing module in the first model, this module has not yet received any processing results from the preceding modules. Therefore, the first input data it receives comes from external input, i.e., the initially acquired first data. In this case, the first input data is the first data itself.

[0074] In the second scenario, if the current network layer processing module is not the first network layer processing module (i.e., it belongs to an intermediate or subsequent layer in the first model), its input no longer comes from the original first data, but from the processing result of the preceding network layer processing module. The preceding network layer processing module outputs a result after processing the data; this result is the second data. This second data becomes the input of the current network layer processing module. Therefore, in this case, the first input data is the second data from the preceding network layer processing module.

[0075] Here, the second feature vector refers to the data obtained after the network layer processing module processes the first input data. The second feature vector can be either the second input data to the next network layer processing module or the first second output data. Whether the second feature vector specifically refers to the second input data to the next network layer processing module or the first second output data depends on the current processing stage of the network layer processing module in the entire first model and the destination of its output. In the first scenario, if the current network layer processing module is not the last network layer processing module in the first model, meaning there are other network layer processing modules that need to continue processing the data, the output data of this module will not be directly used as the final output. Instead, it will be passed as input to the next network layer processing module for further processing. In this case, the second feature vector output by the current network layer processing module is the second input data to the next network layer processing module.

[0076] In the second scenario, if the current network layer processing module is the last network layer processing module in the first model, meaning there are no other network layer processing modules to continue processing data, the output data of this module is no longer passed to deeper layers, but is instead used as part of the final output of the entire model. Since the model generates output unit by unit in an autoregressive manner, in this case, the second feature vector output by this module is the first second data generated by the model in the current round.

[0077] During the first model's processing of the first data, the input source of the current network layer processing module can be determined first: if the current network layer processing module is the first processing module in the first model, then the received first input data is the first data; if the current network layer processing module is not the first processing module, then the first input data it receives is the second data output by the previous network layer processing module. Subsequently, the network layer processing module performs a predetermined neural network operation on the received first input data. After the operation is completed, the destination of its output data is determined according to the position of the current network layer processing module in the first model: if there are subsequent network layer processing modules after the current network layer processing module, then the generated second feature vector will be passed as input to the next network layer processing module, and at this time, the second feature vector is the second input data input to the next network layer processing module; if the current network layer processing module is the last processing module in the first model, then the second feature vector it generates will be the final output result of the model in the current generation round, and at this time, the second feature vector is the first second data output by the model.

[0078] S430, Obtain the first compressed information of the second feature vector.

[0079] The second feature vector includes at least one sub-feature vector. Each sub-feature vector in the second feature vector corresponds to a key-value pair of a unit or an attention head in the sequence. The first compression information refers to the compression information used to compress the at least one sub-feature vector.

[0080] Optionally, during the first model processing, it can be monitored whether the first model has completed processing the first input data and generated a second feature vector. When the first model outputs a second feature vector, it can respond to this output event by acquiring the second feature vector generated during the current processing; and then process based on the acquired second feature vector to determine the first compression information corresponding to at least one sub-feature vector.

[0081] Optionally, the second feature vector is obtained in the following way: The input data is linearly transformed using multiple learning matrices in the network layer processing module to obtain the second feature vector, and then the second feature vector is normalized to obtain the normalized second feature vector.

[0082] The multiple learning matrices refer to three sets of trainable parameter matrices pre-set within the network layer processing module for linear transformation of the input data, typically denoted as Wq, Wk, and Wv. These three sets of learning matrices perform different transformation functions: one set transforms the input data into a query representation, another into a key representation, and the third into a value representation. By applying these three sets of learning matrices to the input data for linear transformation, a corresponding second feature vector can be obtained. This second feature vector is composed of the transformed query representation, key representation, and value representation.

[0083] In this paper, multiple pre-set learning matrices in the network layer processing module are used to perform linear transformations on the input data to obtain the second feature vector. After obtaining the second feature vector, normalization is performed on it to adjust the numerical range and unify the distribution, ultimately resulting in a normalized second feature vector. The benefits of this approach are: it lays a data foundation for accurately evaluating the importance of each sub-feature vector and determining reasonable first compression information; it ensures that the compression operation is based on normalized input, thereby better preserving key information during compression and improving the balance between compression effect and model performance.

[0084] S440. Based on different compression parameters in the first compression information, the sub-feature vectors in the second feature vector are compressed to obtain the first feature vector.

[0085] The second feature vector reflects the result of linear processing of the input data. The sub-feature vector is related to token information, which in turn is related to the input data. Token information refers to the vectorized representation of the basic data unit obtained after word segmentation or block segmentation of the input data. For example, when the first data is text, the token information is the embedding vector corresponding to each word after word segmentation. When the first data is an image, the token information is the embedding vector obtained by linear mapping of each image block after block segmentation.

[0086] In this paper, we first obtain first compression information for at least one sub-feature vector, which guides the specific execution method of the compression operation. Then, we identify each sub-feature vector contained in the second feature vector. Finally, based on the compression parameters specified by the first compression information, we perform compression processing on the corresponding sub-feature vectors associated with the token information in the second feature vector, and obtain the compressed first feature vector through this compression operation.

[0087] S450. Obtain the second data. Based on the first model, process the first feature vector and the second data to obtain the content.

[0088] The beneficial effects of the above content generation method are as follows: by processing the first data based on the first model to obtain the second feature vector, the structured feature extraction of the input content is realized; then, by obtaining the first compression information of the second feature vector, and compressing each sub-feature vector in the second feature vector according to different compression parameters in the first compression information to obtain the first feature vector, the compression process can apply differentiated compression ratios according to the characteristics of different sub-feature vectors, thereby retaining key information while compressing non-key parts more significantly, improving the compactness and storage efficiency of feature expression, and maintaining the effectiveness and integrity of feature information while reducing computational overhead for subsequent processing steps.

[0089] Figure 5 This is a flowchart illustrating a content generation method provided in this paper. Based on the above scheme, the process of determining the first compressed information associated with the network layer processing module can also be described; its specific implementation can be found in the detailed description herein. Technical features that are the same as or similar to those described above will not be repeated here. Figure 5 As shown, the content generation method of this article may specifically include: S510, Obtain the first data.

[0090] S520. Based on multiple second compression information, the second feature vector is compressed to obtain multiple fourth feature vectors.

[0091] Each second compression information corresponds to a fourth feature vector, and the second compression information is related to the candidate compression ratio.

[0092] Here, the fourth feature vector refers to the compressed key-value information corresponding to each of the second compressed information. The candidate compression ratio refers to a set of pre-defined compression ratios that can be selected when compressing the second feature vector.

[0093] In the specific implementation, for each of these multiple second compression information pieces, compression processing is performed on the second feature vector based on its indicated compression method and corresponding candidate compression ratio. Thus, each time the second feature vector is compressed using a second compression information piece, a corresponding compressed result is generated, which is the fourth feature vector. Finally, after multiple compression processes, multiple fourth feature vectors corresponding to the multiple second compression information pieces are obtained. Each fourth feature vector represents the key-value information obtained after compressing the second feature vector at a specific candidate compression ratio.

[0094] S530. Based on multiple fourth feature vectors, second feature vectors, first conditions, and second conditions, obtain the first compressed information of the second feature vector.

[0095] The first condition characterizes the accuracy requirement that the output of the first model must meet, and the second condition characterizes the memory requirement that the cached first feature vector must meet. The first condition can be understood as the condition that the total accuracy loss of the network layer processing module must meet after compressing at least some of the sub-feature vectors in the second feature vector. The second condition can be understood as the memory limit information condition.

[0096] In the specific implementation process, multiple sets of fourth feature vectors that have been generated are obtained, and the original uncompressed second feature vectors of the network layer processing module are also obtained. Subsequently, the first and second conditions are introduced as evaluation criteria. Based on the above multiple sets of fourth feature vectors, the original second feature vectors, the first and second conditions, a comprehensive analysis and calculation are performed. By comparing the accuracy loss and memory usage under different compression strategies, and under the premise of satisfying the first and second conditions, the corresponding first compression information is determined for each sub-feature vector in the second feature vector.

[0097] This process ensures that the compression method of each sub-feature vector in the overall compression scheme is determined after trade-offs. Therefore, the beneficial effect of the above method is that the total accuracy loss after compression of all sub-feature vectors can meet the requirements of the first condition, while the total memory usage after compression is also controlled within the range specified by the memory limit information. Thus, the model's generation performance is maintained to the maximum extent while effectively reducing the memory usage of the key-value cache.

[0098] Specifically, to assess the impact of compressing a specific sub-feature vector, an isolation analysis approach was adopted. This approach examines the error introduced by compressing a single sub-feature vector, assuming other sub-feature vectors remain unchanged. This error refers to the difference between the attention mechanism's output and the original output caused by the change in the attention score and value vector corresponding to that sub-feature vector due to the compression operation. The analysis focuses on the change in attention output corresponding to the query within a local window and quantifies it as follows: (1) In the formula, Representing a local window, it is a set of queries, specifically referring to queries contained within a context range near the current processing unit, used to approximate the impact of compression operations on the global scope within a local range. This represents all queries within the local window Q. The attention score represents the changed attention score, specifically the new attention weight assigned to the t-th token information by query q after compressing only the token information corresponding to the t-th sub-feature vector. This represents the changed value, specifically the new value vector obtained after compressing the token information corresponding to the t-th sub-feature vector. This represents the original attention score, specifically the original attention weight assigned to the corresponding token information by query q in its original, uncompressed state. The original value refers specifically to the matrix composed of the value vectors corresponding to all token information in the original state without any compression. The physical meaning of formula (1) is: the degree of influence on the output result of the model's attention mechanism after compressing the token information corresponding to the t-th sub-feature vector. This metric directly reflects the change in the model's attention output in a local range after a specific sub-feature vector is compressed. The larger the change, the more important the sub-feature vector is to maintain the model's accuracy, and more information should be retained or a lower compression ratio should be used during compression; conversely, the sub-feature vector is relatively minor and can be processed with a higher compression ratio.

[0099] Considering that the local window only provides a partial sample of the query, a conservative score estimation strategy is adopted to ensure that the key sub-feature vectors retain as much original information as possible. This strategy considers the upper bound of the error, which can be expressed as: (2) Formula (2) above represents the upper bound of the total error. Specifically, it refers to decomposing the total error introduced by compressing a specific sub-feature vector into the sum of the attention score change term and the value vector change term. This is used to approximate the impact of compression on model accuracy without having to fully recompute the attention mechanism.

[0100] To simplify computation and facilitate batch processing, it is assumed that all sub-feature vectors are compressed according to a specific compression ratio, thus obtaining an attention score for the query. Based on the low-rank nature of key-value information and the sparsity of attention scores, this change in attention score is considered a slight perturbation. Combining the above analysis, a scoring metric for evaluating the impact of compression on each sub-feature vector is constructed by combining two computational terms. The expression for the scoring metric can be expressed as: (3) The physical meaning of the above formula (3) is: to quantify the total impact of the token information corresponding to the t-th sub-feature vector on the output of the model's attention mechanism after compression, and this impact is expressed in the form of an upper bound on the error. Specifically, the formula decomposes the total error introduced by compression into two independent components and sums them up: The first term in the formula The term representing the change in attention score measures the error impact on the attention output caused by the change in the attention score of the t-th token information due to compression, even if the value vector itself remains unchanged. The second term in the formula The value vector change term measures the error impact on the attention output caused by the change in the value vector content of the t-th token information due to compression, even if the attention score remains unchanged.

[0101] Summing these two items over all queries within the local window Q yields the final result. The value represents the upper bound estimate of the total error introduced by compressing the t-th sub-feature vector. The larger this value, the more significant the impact of compressing this sub-feature vector on the model accuracy, meaning the higher the importance of this sub-feature vector.

[0102] Finally, based on multiple sets of fourth feature vectors, the second feature vectors of the network layer processing module, the first condition, and the memory limit information, combined with the degree of influence of each sub-feature vector after compression reflected by the scoring metric, the corresponding first compression information is determined for each sub-feature vector in the second feature vector. Specifically: Formula (3) provides a quantified importance score for each sub-feature vector by calculating the sum of the attention score change term and the value vector change term after compressing a specific sub-feature vector. This score directly reflects the degree of impact of the compressed sub-feature vector on the model accuracy. Subsequently, when exploring compression schemes based on multiple sets of fourth feature vectors, it is necessary to comprehensively weigh the score results calculated by this formula for each sub-feature vector, the memory limit information of the network layer processing module, and the first condition. For sub-feature vectors with higher scores, it indicates that they are more critical to maintaining model accuracy. Therefore, a lower compression ratio or more dimensions should be retained during compression to ensure that the overall compression scheme meets the first condition. For sub-feature vectors with lower scores, a higher compression ratio can be used to further reduce memory usage, thereby meeting the requirements of the memory limit information. In this way, the first compression information that can achieve the best compression effect under the premise of meeting the first condition and the memory limit information is finally determined for each sub-feature vector in the second feature vector.

[0103] Optionally, the specific implementation of obtaining the first compressed information of the second feature vector based on multiple fourth feature vectors, second feature vectors, first conditions, and second conditions may include: (1) Based on the precision loss function, multiple fourth feature vectors and second feature vectors are processed to obtain multiple sub-precision losses.

[0104] Among them, multiple sub-precision losses are used to characterize the loss values ​​of at least some sub-feature vectors in the second feature vector under different second compression information.

[0105] The precision loss function refers to a pre-defined mathematical formula or algorithmic rule used to quantize multiple sets of fourth and second feature vectors to calculate the degree of precision loss introduced by the compression operation. Sub-precision loss refers to the numerical value of the precision loss introduced by a sub-feature vector when compressing a sub-feature vector within the second feature vector using a specific set of second compression information.

[0106] In this paper, a predetermined precision loss function is first obtained. This function quantifies the degree of precision loss introduced by the compression operation. Multiple sets of fourth feature vectors and the second feature vectors from the network layer processing module are taken as input and fed into this precision loss function for processing. During processing, for each sub-feature vector in the second feature vector, the precision loss corresponding to compression using different second compression information is evaluated.

[0107] Optionally, the specific implementation of obtaining the sub-precision loss of the sub-feature vector under different second compression information may include: Step 1: For multiple fourth feature vectors, based on the first sub-function in the precision loss function, process the sub-feature vectors associated with the same data unit in the fourth and second feature vectors to obtain attention weight change information; and based on the attention weight change information and some sub-feature vectors in the second feature vector, obtain attention loss information.

[0108] Here, the first sub-function refers to the pre-defined portion of the accuracy loss function specifically used to calculate the error introduced by changes in attention weights. Attention weight change information refers to information used to characterize the change in attention weights corresponding to the i-th sub-feature vector before and after compression. Attention loss information refers to the numerical value used to quantify the accuracy loss caused by changes in attention weights.

[0109] In this paper, given multiple sets of fourth feature vectors, the i-th sub-feature vector to be processed is selected. Then, the first sub-function of the precision loss function is invoked, specifically for handling calculations related to attention weights. The i-th sub-feature vector from the fourth feature vector and the i-th sub-feature vector from the second feature vector are input to the first sub-function for processing. This processing yields attention weight change information, which characterizes the change in attention weights corresponding to the i-th sub-feature vector before and after compression. After obtaining the attention weight change information, it is further processed in conjunction with the value information in the second feature vector. This process ultimately determines the attention loss information, which quantifies the precision loss numerically introduced solely by the attention weight change.

[0110] The second step is to process the value information of the sub-feature vector in the fourth feature vector and the value information of the second feature vector based on the second sub-function in the precision loss function to obtain the value change loss information.

[0111] The second sub-function is specifically used to calculate the error introduced by changes in the value vector. Value change loss information refers to the numerical value used to quantify the accuracy loss caused by changes in the value vector.

[0112] In this paper, the second sub-function of the precision loss function can be invoked. This second sub-function is specifically designed to handle calculations related to changes in the value vector. Subsequently, the value information corresponding to the i-th sub-feature vector is extracted from the fourth feature vector, and the value information from the second feature vector is also obtained. These two sets of value information are then fed into the second sub-function for processing. Through the predefined operational logic of the second sub-function, the value information before and after compression is compared and quantified to ultimately determine the value change loss information. This information is used to quantify the precision loss value introduced solely by the change in the value vector corresponding to the i-th sub-feature vector.

[0113] Step 3: Based on the attention loss information and value change loss information, obtain the sub-precision loss of the sub-feature vector.

[0114] In this paper, we can obtain the already determined attention loss information and value change loss information, and integrate these two loss information by combining them into a whole evaluation result through a predefined combination method. The result of this integrated processing is the sub-precision loss corresponding to the output data of the network layer processing module after the i-th sub-feature vector is compressed. This sub-precision loss comprehensively reflects the total impact of compressing the sub-feature vector on the model's output accuracy.

[0115] Optionally, through the above steps, for each sub-feature vector, all candidate second compression information are traversed, and its sub-precision loss under each candidate compression method is calculated respectively, so as to obtain the correspondence between the sub-precision loss of each sub-feature vector and the candidate compression method.

[0116] The beneficial effect of the above-mentioned precision loss determination process is that it enables a comprehensive measurement of the impact of compression operations on the model output precision from two dimensions: changes in attention weights and changes in value vectors, thereby obtaining the accurate sub-precision loss of each sub-feature vector under different second compression information.

[0117] (2) Based on multiple sub-precision losses associated with at least some sub-feature vectors and a first condition, and based on multiple compressed memories and a second condition associated with at least some sub-feature vectors, obtain the first compressed information of at least some sub-feature vectors in the second feature vector.

[0118] Among these, multiple compressed memories are associated with the second compressed information. Optionally, the number of multiple sub-precision losses is the same as the number of second compressed information, and the number of multiple compressed memories is the same as the number of second compressed information.

[0119] Here, compressed memory refers to the amount of memory space occupied in the cache by the compressed result corresponding to the sub-feature vector after the sub-feature vector in the second feature vector is compressed.

[0120] In this paper, multiple sub-precision losses associated with each sub-feature vector are obtained, the number of which is the same as the number of second compression information, with each sub-precision loss corresponding to a specific second compression information. Simultaneously, multiple compressed memory locations associated with each sub-feature vector are also obtained, the number of which is also the same as the number of second compression information, with each compressed memory location corresponding to a specific second compression information. Then, a first condition and memory limit information are introduced as decision-making criteria. Based on the multiple sub-precision losses of each sub-feature vector, combined with the first condition, the overall compression scheme is evaluated to determine whether it meets the precision loss requirements when using different second compression information. Simultaneously, based on the multiple compressed memory locations of each sub-feature vector, combined with the memory limit information, the overall compression scheme is evaluated to determine whether it meets the memory usage requirements when using different second compression information. By comprehensively balancing the sub-precision loss and compressed memory of each sub-feature vector under different second compression information, and ensuring that the total precision loss after compression of all sub-feature vectors meets the first condition and the total compressed memory meets the memory limit information, the corresponding first compression information is selected for each sub-feature vector in the second feature vector.

[0121] The beneficial effects of determining the first compression information for each sub-feature vector are as follows: it enables refined screening and decision-making for the compression scheme of each sub-feature vector, ensuring that the total memory usage of all compressed sub-feature vectors does not exceed the memory limit information under the premise that the first condition is met overall. Thus, while compressing the key-value cache to reduce memory pressure, it effectively maintains the generation performance of the model.

[0122] Optionally, the specific implementation of obtaining the first compression information of at least some sub-feature vectors in the second feature vector based on multiple sub-precision losses associated with at least some sub-feature vectors and a first condition, and based on multiple compressed memories and a second condition associated with at least some sub-feature vectors, may include the following steps: Step 1: Based on the first operator, convert the multiple sub-precision losses and the first condition associated with at least some sub-feature vectors, as well as the multiple compressed memories and the second condition associated with at least some sub-feature vectors, into the first function.

[0123] Here, the first operator refers to the undetermined coefficients introduced when transforming a constrained optimization problem into an unconstrained one. Its magnitude directly determines the weight trade-off between the accuracy loss and memory usage terms in the optimization objective. Optionally, the first operator can be any first operator. Specifically, for each sub-feature vector, a compression method needs to be selected from multiple candidate compression methods that minimizes the sum of the sub-accuracy loss and the first operator multiplied by the compressed memory. That is, finding the method that minimizes the sum of the sub-accuracy loss and the compressed memory usage among all candidate dimensions. Find the minimum d, and thus obtain the optimal compression choice for the sub-feature vector under the current first operator.

[0124] In fact, when the value of the first operator is small, it indicates a lighter penalty for memory usage. In this case, the optimization process tends to prioritize preserving precision, thus favoring compression methods that consume more memory after compression but suffer less sub-precision loss. Conversely, when the value of the first operator is large, it indicates a heavier penalty for memory usage. In this case, the optimization process tends to prioritize saving memory, thus favoring compression methods that consume less memory after compression but may suffer greater sub-precision loss. By introducing the first operator and constructing a first function that includes sub-precision loss and compressed memory, a balance can be achieved between total precision loss and compressed memory while satisfying the memory limit information.

[0125] The first function characterizes the balance between total precision loss and compressed memory when the memory limit is met. This can be understood as, within the constraints of the memory limit information, reflecting the optimal state or mutual constraint relationship achievable by the total precision loss when the compressed memory is limited to the value specified by the memory limit information, through a mathematical description of the relationship between the total precision loss and compressed memory. Specifically, this balance information reflects how to achieve an equilibrium between the total precision loss and compressed memory while satisfying the memory limit information; that is, it is impossible to further reduce the total precision loss without exceeding the memory limit information, nor is it possible to further reduce the compressed memory without increasing the total precision loss.

[0126] In this paper, multiple sub-precision losses associated with each sub-feature vector are associated with a first condition, and multiple compressed memory values ​​associated with each sub-feature vector are associated with memory limit information. Based on a first operator, these relationships are integrated and transformed into a first function. The purpose of this function is to transform the problem of solving the sub-feature vector compression scheme under the constraints of satisfying the first condition and memory limit information into an unconstrained mathematical expression. Through this transformation, the first function can mathematically represent the state information when the total precision loss and compressed memory reach a mutual constraint and balance under the premise of satisfying the memory limit information, that is, it reflects the equilibrium relationship between the total precision loss and compressed memory within the limit of the memory limit information.

[0127] Specifically, the expression for the first function can be represented as: (4) In the formula, The symbol represents the penalty coefficient per unit dimension for the first operator, which represents the precision loss cost of occupying 1 additional unit dimension, used to balance memory usage and precision loss; the inner layer in formula (4) Indicates in fixed Next, for each sub-feature vector, select di to minimize the accuracy loss and dimensionality penalty; the outer layer in formula (4) , indicating adjustment This ensures that the final total dimension perfectly matches the memory budget B.

[0128] Step 2: Based on the first operator in the first function, obtain the compression parameters of at least some sub-feature vectors in the first compressed information.

[0129] In this paper, the value of the first operator can be kept unchanged and used as a fixed parameter in the first function for solving the function. During the solution process, for each sub-feature vector, its comprehensive performance under various candidate compression methods is evaluated, and the candidate compression method that makes the first function reach its optimal state is selected. This selection result is the solution result for each sub-feature vector, which at least contains the first compression information determined for each sub-feature vector, that is, the specific compression method that should be adopted for each sub-feature vector.

[0130] Based on the above formula (4), when When fixed, the inner optimization can be broken down into N independent subproblems (each sub-feature vector is decided independently), and the formula can be expressed as: (5) In the formula, D represents the set of candidate compression dimensions; This represents the accuracy loss when the i-th sub-feature vector is compressed to dimension d (calculated using the previous Lt formula); This represents the memory penalty corresponding to dimension d. The larger the value, the heavier the penalty, and the more inclined it is to compress dimensions. The physical meaning of is expressed in the current The optimal compression dimension of the i-th sub-feature vector is the first compressed information (selected from the candidate set).

[0131] The beneficial effect of determining the first compression information of each sub-feature vector is that it realizes the mathematical modeling and optimization of the trade-off between accuracy loss and memory usage under the premise of satisfying the memory limit information. This allows the first compression information determined for each sub-feature vector to achieve the optimal state of total accuracy loss while strictly satisfying the memory limit information constraint. Thus, it achieves an effective balance between compressed memory and model accuracy as a whole, and preserves the model's generation performance to the maximum extent.

[0132] Optionally, in this paper, after obtaining the solution results of the sub-feature vectors, the following steps are also included: (1)When the compressed memory obtained after compressing the second feature vector based on the first compression information does not meet the second condition, adjust the operator information of the first operator.

[0133] In this article, the sub-feature vector can be compressed according to the first compression information included in the solution result of the sub-feature vector, and the occupied compressed memory after compression can be calculated. Subsequently, this compressed memory is compared with the pre-set memory upper limit information. If the occupied compressed memory after compression is greater than the value specified by the memory upper limit information, it indicates that the compression scheme corresponding to the current solution result fails to meet the memory occupancy limit requirements, and this scheme needs to be adjusted. At this time, the numerical value of the first operator used in the first function is adjusted, and the weight relationship between the precision loss and the memory occupancy is adjusted by changing the size of the first operator.

[0134] (2)Based on the adjusted first operator, re-execute the step of obtaining the compression parameter.

[0135] In this article, after adjusting the first operator, according to the adjusted first operator, the step of obtaining the solution result of the sub-feature vector under the condition that the first operator in the first function remains unchanged can be re-executed, and through iterative optimization, the first compression information that not only meets the requirements of the memory upper limit information but also minimizes the total precision loss can be finally obtained.

[0136] The above verification of whether the compressed memory is greater than the memory upper limit information and the implementation process of adjusting the first operator can be achieved through the following method: Calculate the sum of all current : ; If > B: The total dimension exceeds the budget, then increase (increase the penalty to make more sub-feature vectors select smaller dimensions); if < B: The total dimension is less than the budget, then decrease (reduce the penalty to make more tokens select larger dimensions) until S(λ)≈B, and the at this time is the final compression ratio.

[0137] The beneficial effect of the above processing method is that by introducing a feedback adjustment mechanism based on the comparison result of the compressed memory and the memory upper limit information, the dynamic optimization of the first operator is realized, ensuring that the first compression information of each sub-feature vector finally determined can effectively balance the compressed memory and the model accuracy on the premise of strictly meeting the memory upper limit information, so as to take into account both the memory resource limitation and the maintenance of the model generation performance in actual deployment.

[0138] S540. Process the first data based on the first model to obtain the first feature vector.

[0139] The first feature vector is obtained by compressing the second feature vector based on the first compression information. The first compression information is determined based on the second feature vector and includes multiple different compression parameters. The compression parameters are used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector.

[0140] S550. Obtain the second data. Based on the first model, process the first feature vector and the second data to obtain the content.

[0141] The content generation method presented in this paper, when determining the first compression information associated with the network layer processing module, compresses the second feature vector based on multiple second compression information related to candidate compression ratios to obtain corresponding sets of fourth feature vectors. Subsequently, it comprehensively evaluates the multiple sets of fourth feature vectors, the original second feature vectors, the total precision loss constraint, and the memory limit information to determine the corresponding first compression information for each sub-feature vector in the second feature vector. This process ensures that the compression method of each sub-feature vector in the overall compression scheme is determined after systematic trade-offs, so that the total precision loss generated after the compression of all sub-feature vectors can meet the requirements of the first condition, while the total memory usage after compression is also controlled within the range specified by the memory limit information. Thus, while effectively reducing the memory usage of the key-value cache, it maximizes the maintenance of the model's generation performance.

[0142] In this paper, optionally, in order to achieve the effect of batch and fast calling during the content generation process, the following steps may be included: for multiple network layer processing modules, classify the sub-feature information based on the first compressed information of at least one sub-feature information in the network layer processing module; cache the sub-feature information belonging to the same category.

[0143] Specifically, when processing the first media data to generate the second media data, each network layer processing module generates and caches first key-value information for generating subsequent sub-media data. This first key-value information is obtained by compressing the second key-value information with first compression information, and it contains several sub-feature information. Based on the first compression information corresponding to these sub-feature information, they can be categorized and then stored in the cache together. This optimizes the organization of data in the cache, allowing for batch retrieval of the required data from the corresponding category when generating the i-th sub-media data and needing to access previously cached key-value information, thus achieving the effect of fast batch retrieval.

[0144] Figure 6 This is a schematic diagram of the structure of a content generation device provided in this article, such as... Figure 6As shown, the device includes: a data acquisition module 610, a feature compression module 620, and a content generation module 630.

[0145] The data acquisition module 610 is used to acquire first data; the feature compression module 620 is used to process the first data based on a first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing a second feature vector based on first compression information, the first compression information being determined based on the second feature vector, the first compression information including multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; and the content generation module 630 is used to acquire second data, process the first feature vector and the second data based on the first model to obtain the content.

[0146] Optionally, based on the above-described device, the feature compression module 620 is further configured to obtain a third feature vector of the second data based on the first model; obtain compression parameters of the third feature vector; and compress the third feature vector based on the compression parameters to update the first feature vector.

[0147] Optionally, based on the above-mentioned device, the feature compression module 620 is further configured to process the first data based on the first model to obtain a second feature vector; obtain first compression information of the second feature vector; and compress the sub-feature vectors in the second feature vector based on different compression parameters in the first compression information to obtain the first feature vector.

[0148] Optionally, based on the above-mentioned device, the content generation device may also include a compressed information determination module; The compression information determination module is used to compress the second feature vector based on multiple second compression information to obtain multiple fourth feature vectors, wherein each second compression information corresponds to a fourth feature vector, and the second compression information is related to the candidate compression ratio; based on the multiple fourth feature vectors, the second feature vector, the first condition, and the second condition, the first compression information of the second feature vector is obtained; wherein the first condition is used to characterize the accuracy condition that the output content of the first model must meet, and the second condition is used to characterize the memory condition that the cached first feature vector must meet.

[0149] Based on the above-described apparatus, optionally, a compression information determination module is used to process the plurality of fourth feature vectors and the second feature vector based on a precision loss function to obtain a plurality of sub-precision losses, wherein the plurality of sub-precision losses are used to characterize the loss values ​​of at least some sub-feature vectors in the second feature vector under different second compression information; based on the plurality of sub-precision losses associated with the at least some sub-feature vectors and a first condition, and based on the plurality of compressed memories associated with the at least some sub-feature vectors and the second condition, to obtain first compression information of at least some sub-feature vectors in the second feature vector; wherein the plurality of compressed memories are related to the second compression information.

[0150] Based on the above device, optionally, a compressed information determination module is used to process the sub-feature vectors associated with the same data unit in the fourth feature vector and the second feature vector based on the first sub-function in the precision loss function for multiple fourth feature vectors, so as to obtain attention weight change information. Based on the attention weight change information and some sub-feature vectors in the second feature vector, attention loss information is obtained; based on the second sub-function in the accuracy loss function, the value information of the sub-feature vectors in the fourth feature vector and the value information of the second feature vector are processed to obtain value change loss information. Based on the attention loss information and the value change loss information, the sub-precision loss of the sub-feature vector is obtained.

[0151] Based on the above-mentioned device, optionally, the compression information determination module is further configured to convert, based on the first operator, multiple sub-precision losses and the first condition associated with the at least some sub-feature vectors, as well as multiple compressed memories and the second condition associated with the at least some sub-feature vectors, into a first function; and based on the first operator in the first function, obtain the compression parameters of the at least some sub-feature vectors in the first compression information.

[0152] Based on the above-mentioned device, optionally, the compression information determination module is further configured to adjust the operator information of the first operator if the compressed memory obtained after compressing the second feature vector based on the first compression information does not meet the second condition; and re-execute the step of obtaining the compression parameters based on the adjusted first operator.

[0153] The beneficial effects of the above-mentioned device are as follows: acquiring first data; processing the first data based on a first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing a second feature vector based on first compression information, the first compression information being determined based on the second feature vector, the first compression information including multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; acquiring second data, processing the first feature vector and the second data based on the first model to obtain the content, thus solving the problems of slow model inference speed and large memory consumption in the prior art, and achieving the effect of improving model inference efficiency while ensuring model inference accuracy.

[0154] The content generation apparatus provided herein can execute any of the content generation methods provided herein, and has the corresponding functional modules and beneficial effects of executing the methods.

[0155] It is worth noting that the various units and modules included in the above-mentioned device are divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of this document.

[0156] Figure 7 This is a schematic diagram of the structure of an electronic device provided in this article. Figure 7 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0157] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0158] In particular, the processes described in the flowchart above can be implemented as computer software programs. For example, this document includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an implementation, the computer program can be downloaded and installed from a network via communication device 609, or installed from storage device 608, or installed from ROM 602. When the computer program is executed by processing device 601, it performs the functions defined in the methods herein.

[0159] The beneficial effects of the aforementioned electronic device are as follows: It acquires first data; processes the first data based on a first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing a second feature vector based on first compression information, the first compression information being determined based on the second feature vector, and the first compression information including multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; it acquires second data, processes the first feature vector and the second data based on the first model to obtain the content, thus solving the problems of slow model inference speed and large memory consumption in the prior art, and achieving the effect of improving model inference efficiency while ensuring model inference accuracy.

[0160] The names of messages or information exchanged between multiple devices in this document are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0161] The electronic device provided herein and the content generation method provided above belong to the same inventive concept. Technical details not described in detail herein can be found in the content generation method provided above. Furthermore, the electronic device and the content generation method provided above have the same beneficial effects.

[0162] This article provides a computer storage medium on which a computer program is stored, which, when executed by a processor, implements the content generation method described above.

[0163] It should be noted that the computer-readable medium described above can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this document, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0164] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0165] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0166] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: Get the first data; The first data is processed based on the first model to obtain a first feature vector. The first feature vector is obtained by compressing the second feature vector based on the first compression information. The first compression information is determined based on the second feature vector and includes multiple different compression parameters. The compression parameters are used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector. Obtain the second data, and process the first feature vector and the second data based on the first model to obtain the content.

[0167] Computer program code for performing the operations described herein may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0168] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of the systems, methods, and computer program products according to this document. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the figures. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0169] The units described herein can be implemented in software or hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0170] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0171] In the context of this document, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0172] The beneficial effects of the aforementioned computer storage medium are as follows: First data is acquired; the first data is processed based on a first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing a second feature vector based on first compression information, the first compression information being determined based on the second feature vector, and the first compression information including multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; second data is acquired; the first feature vector and second data are processed based on the first model to obtain the content, thus solving the problems of slow model inference speed and large memory consumption in the prior art, and achieving the effect of improving model inference efficiency while ensuring model inference accuracy.

[0173] The above description is merely a preferred implementation and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the description herein is not limited to technical solutions formed by specific combinations of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions described herein.

[0174] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Multitasking and parallel processing may be advantageous in certain contexts. Similarly, while some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this paper. Certain features described in the context of a single implementation may also be implemented in combination within a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0175] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A content generation method comprising: obtaining first data; processing the first data based on a first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing a second feature vector based on first compression information, the first compression information is determined based on the second feature vector, and the first compression information comprises a plurality of different compression parameters, the compression parameters being used to indicate compression ratios of at least part of sub-feature vectors in the second feature vector; obtaining second data, and processing the first feature vector and the second data based on the first model to obtain the content.

2. The method of claim 1, wherein the processing the first feature vector and the second data based on the first model comprises: obtaining a third feature vector of the second data based on the first model; obtaining compression parameters of the third feature vector, and compressing the third feature vector based on the compression parameters to update the first feature vector.

3. The method of claim 1, wherein the processing the first data based on the first model to obtain the first feature vector comprises: processing the first data based on the first model to obtain a second feature vector; obtaining first compression information of the second feature vector; compressing sub-feature vectors in the second feature vector based on different compression parameters in the first compression information to obtain the first feature vector.

4. The method of any one of claims 1-3, wherein the first compression information is obtained by: compressing the second feature vector based on a plurality of second compression information, to obtain a plurality of fourth feature vectors, wherein, each second compression information corresponding to a fourth feature vector, the second compression information being associated with a candidate compression ratio; obtaining the first compression information of the second feature vector based on the plurality of fourth feature vectors, the second feature vector, a first condition, and a second condition; wherein the first condition is used to represent an accuracy condition to be met by an output content of the first model, and the second condition is used to represent a memory condition to be met by caching the first feature vector.

5. The method of claim 4, wherein the obtaining the first compression information of the second feature vector based on the plurality of fourth feature vectors, the second feature vector, the first condition, and the second condition comprises: processing the plurality of fourth feature vectors and the second feature vector based on an accuracy loss function to obtain a plurality of sub-accuracy losses, wherein the plurality of sub-accuracy losses are used to represent loss values of at least part of sub-feature vectors in the second feature vector under different second compression information; obtaining the first compression information of at least part of sub-feature vectors in the second feature vector based on the plurality of sub-accuracy losses associated with the at least part of sub-feature vectors and the first condition, and based on a plurality of compression memories associated with the at least part of sub-feature vectors and the second condition; wherein the plurality of compression memories are associated with the second compression information.

6. The method of claim 5, wherein the processing the plurality of fourth feature vectors and the second feature vector based on the accuracy loss function to obtain the plurality of sub-accuracy losses comprises: For multiple fourth feature vectors, based on the first sub-function in the precision loss function, the sub-feature vectors associated with the same data unit in the fourth feature vector and the second feature vector are processed to obtain attention weight change information; Based on the attention weight change information and some sub-feature vectors in the second feature vector, attention loss information is obtained; Based on the second sub-function in the accuracy loss function, the value information of the sub-feature vector in the fourth feature vector and the value information of the second feature vector are processed to obtain value change loss information; Based on the attention loss information and the value change loss information, the sub-precision loss of the sub-feature vector is obtained.

7. The method according to claim 5, wherein obtaining first compression information of at least a portion of the sub-feature vectors in the second feature vector based on a plurality of sub-precision losses associated with the at least partial sub-feature vectors and a first condition, and based on a plurality of compressed memories associated with the at least partial sub-feature vectors and the second condition, comprises: Based on the first operator, the multiple sub-precision losses and the first condition associated with the at least some sub-feature vectors, as well as the multiple compressed memories and the second condition associated with the at least some sub-feature vectors, are converted into a first function; Based on the first operator in the first function, compression parameters for at least some sub-feature vectors in the first compressed information are obtained.

8. The method according to claim 7, further comprising: If the compressed memory obtained after compressing the second feature vector based on the first compression information does not meet the second condition, the operator information of the first operator is adjusted. Based on the adjusted first operator, the step of obtaining the compression parameters is executed again.

9. A content generation apparatus, comprising: The media data acquisition module is used to acquire the initial data. The feature compression module is used to process the first data based on the first model to obtain a first feature vector, wherein the first feature vector is obtained by compressing the second feature vector based on the first compression information, the first compression information is determined based on the second feature vector, and the first compression information includes multiple different compression parameters, the compression parameters being used to indicate the compression ratio of at least some sub-feature vectors in the second feature vector; The content generation module is used to acquire second data, process the first feature vector and the second data based on the first model, and obtain the content.

10. An electronic device, the electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the content generation method according to any one of claims 1-8.

11. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the content generation method of any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the content generation method as described in any one of claims 1-8.

Images