Method and device for training copywriting generation model, video copywriting generation model and visual copywriting generation model

Abstract

The embodiment of the invention provides copywriting generation, video copywriting generation and visual copywriting generation model training methods and devices.The copywriting generation method comprises the steps that style information and a visual data description text of a target style are obtained, and the style information and the visual data description text are input into a visual copywriting generation model, the visual copywriting generation model comprises a gating network, a plurality of low-rank expert networks with preset styles and a style fusion network; determining at least one target low-rank expert network from a plurality of low-rank expert networks based on the style information by using a gating network; respectively carrying out visual semantic understanding on the visual data description text by utilizing at least one target low-rank expert network, and correspondingly obtaining at least one style copywriting; and generating a visual data copywriting of the target style based on the at least one style copywriting by using the style fusion network. Multi-style flexible combination and fine control are realized, and the diversity and style matching degree of the generated copywriting are improved.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of artificial intelligence technology, and in particular to a method and apparatus for training text generation, video text generation, and visual text generation models. Background Technology

[0002] In today's era of increasingly important digital marketing and content creation, consumers' demand for diverse copywriting styles is constantly growing. To attract different interest groups, businesses often need to prepare promotional materials in various styles, including but not limited to formal, humorous, artistic, or professional writing styles, to suit different product positioning and market strategies. With the development of artificial intelligence technology, especially the increasing prevalence of large-scale models in the field of natural language processing, the automatic generation of diverse copywriting styles has become possible.

[0003] However, in the pursuit of multi-style copy generation, training with multiple styles can easily lead to interference between style features. This results in generated copy that fails to accurately reflect the target style and may also incorporate other unexpected style elements, causing performance degradation. Training with a large amount of mixed-style data places extremely high demands on computational resources.

[0004] Therefore, there is an urgent need for a text generation method that can accurately reflect the target style. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a text generation method. One or more embodiments of this specification also relate to a video text generation method, a visual text generation model training method, a text generation device, a video text generation device, a visual text generation model training device, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a text generation method is provided, comprising:

[0007] The style information and visual data description text of the target style are obtained and input into the visual copy generation model. The visual copy generation model includes a gating network, a low-rank expert network with multiple preset styles, and a style fusion network.

[0008] Using gating networks, at least one target low-rank expert network is identified from multiple low-rank expert networks based on style information.

[0009] Using at least one target low-rank expert network, visual semantic understanding is performed on visual data description text to obtain at least one style copy.

[0010] Using a style fusion network, visual data copy for the target style is generated based on at least one style copy.

[0011] According to a second aspect of the embodiments of this specification, a video text generation method is provided, comprising:

[0012] The style information of the target style and the video description text of the target video are obtained, and the style information and video description text are input into the visual copywriting generation model. The visual copywriting generation model includes a gating network, a low-rank expert network with multiple preset styles, and a style fusion network.

[0013] Using gating networks, at least one target low-rank expert network is identified from multiple low-rank expert networks based on style information.

[0014] Using at least one target low-rank expert network, visual semantic understanding is performed on the video description text to obtain at least one style of video script;

[0015] Using a style fusion network, a video script of the target style is generated based on at least one style video script.

[0016] According to a third aspect of the embodiments of this specification, a method for training a visual text generation model is provided, comprising:

[0017] The system acquires sample style information, sample visual data description text, and corresponding label visual data text for multiple preset styles. The sample style information and sample visual description text are then input into the text generation model, which includes a gating network, multiple low-rank expert networks, and a style fusion network.

[0018] Using a gated network, at least one predictive low-rank expert network is determined from multiple low-rank expert networks based on sample style information;

[0019] Using at least one predictive low-rank expert network, visual semantic understanding is performed on the visual data description text of the samples, and at least one predictive style copy is obtained accordingly.

[0020] Using a style fusion network, predictive visual data copy is generated based on at least one predictive style copy.

[0021] Calculate the difference loss based on the predicted visual data text and the tag visual data text;

[0022] Based on the difference loss, the parameters of the gating network and multiple low-rank expert networks are iteratively trained until the training stops, thus obtaining the visual text generation model.

[0023] According to a fourth aspect of the embodiments of this specification, a copywriting generation apparatus is provided, comprising:

[0024] The first acquisition module is configured to acquire style information and visual data description text of the target style, and input the style information and visual data description text into the visual copywriting generation model. The visual copywriting generation model includes a gating network, a low-rank expert network with multiple preset styles, and a style fusion network.

[0025] The first determination module is configured to use a gated network to determine at least one target low-rank expert network from multiple low-rank expert networks based on style information.

[0026] The first semantic understanding module is configured to use at least one target low-rank expert network to perform visual semantic understanding on visual data description text, and obtain at least one style copy accordingly.

[0027] The first generation module is configured to use a style fusion network to generate visual data copy for a target style based on at least one style copy.

[0028] According to a fifth aspect of the embodiments of this specification, a video script generation apparatus is provided, comprising:

[0029] The second acquisition module is configured to acquire style information of the target style and video description text of the target video, and input the style information and video description text into the visual copywriting generation model. The visual copywriting generation model includes a gated network, a low-rank expert network with multiple preset styles, and a style fusion network.

[0030] The second determination module is configured to use a gated network to determine at least one target low-rank expert network from multiple low-rank expert networks based on style information.

[0031] The second semantic understanding module is configured to use at least one target low-rank expert network to perform visual semantic understanding on the video description text and obtain at least one style video script accordingly.

[0032] The second generation module is configured to use a style fusion network to generate a video script of the target style based on at least one style video script.

[0033] According to a sixth aspect of the embodiments of this specification, a visual text generation model training apparatus is provided, comprising:

[0034] The third acquisition module is configured to acquire sample style information of multiple preset styles, sample visual data description text, and corresponding label visual data copy of multiple preset styles, and input the sample style information and sample visual description text into the copy generation model. The copy generation model includes a gated network, multiple low-rank expert networks, and a style fusion network.

[0035] The third determination module is configured to use a gating network to determine at least one predictive low-rank expert network from multiple low-rank expert networks based on sample style information.

[0036] The third semantic understanding module is configured to use at least one predictive low-rank expert network to perform visual semantic understanding on the sample visual data description text, and obtain at least one predictive style copy accordingly.

[0037] The third generation module is configured to use a style fusion network to generate predictive visual data copy based on at least one predictive style copy.

[0038] The calculation module is configured to calculate the difference loss based on the predicted visual data text and the label visual data text;

[0039] The training module is configured to iteratively train the parameters of the gating network and multiple low-rank expert networks based on the difference loss until the training stops, thereby obtaining a visual text generation model.

[0040] According to a seventh aspect of the embodiments of this specification, a computing device is provided, comprising:

[0041] Memory and processor;

[0042] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, they implement the steps of the first aspect of the text generation method, the second aspect of the video text generation method, or the third aspect of the visual text generation model training method.

[0043] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions, which, when executed by a processor, implement the steps of the first aspect of the text generation method, the second aspect of the video text generation method, or the third aspect of the visual text generation model training method.

[0044] According to a ninth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the first aspect of the text generation method, the second aspect of the video text generation method, or the third aspect of the visual text generation model training method.

[0045] One embodiment of this specification implements the acquisition of style information and visual data description text for a target style, and inputs the style information and visual data description text into a visual copywriting generation model. The visual copywriting generation model includes a gating network, multiple low-rank expert networks with preset styles, and a style fusion network. Using the gating network, based on style information, at least one target low-rank expert network is determined from the multiple low-rank expert networks. Using the at least one target low-rank expert network, visual semantic understanding is performed on the visual data description text, resulting in at least one style copywriting. Using the style fusion network, visual data copywriting for the target style is generated based on the at least one style copywriting. The gating network dynamically selects and combines multiple preset style expert networks according to the input style information, enabling the model to flexibly respond to diverse style requirements. Each expert network processes the visual description text in parallel, generating intermediate copywriting for the corresponding style, preserving the expressive characteristics of different styles. The style fusion network weighted and integrates the multiple outputs, avoiding style rigidity caused by a single expert and improving the richness and accuracy of the generated copywriting. The overall architecture achieves refined style control and synergistic optimization of generation quality. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a text generation method provided in one embodiment of this specification;

[0047] Figure 2 This is a flowchart illustrating a video text generation method provided in one embodiment of this specification;

[0048] Figure 3 This is a flowchart illustrating a visual text generation model training method provided in one embodiment of this specification;

[0049] Figure 4a This is a schematic diagram of a task material selection interface provided in one embodiment of this specification;

[0050] Figure 4b This is a schematic diagram of a video loading interface provided in one embodiment of this specification;

[0051] Figure 4c This is a schematic diagram illustrating the updating of a video loading interface according to one embodiment of this specification;

[0052] Figure 4d This is a schematic diagram of an updated video loading interface provided in one embodiment of this specification;

[0053] Figure 4e This is a schematic diagram of a video browsing interface provided in one embodiment of this specification;

[0054] Figure 4fThis is a flowchart illustrating the processing steps of a text generation method provided in one embodiment of this specification.

[0055] Figure 5 This is a schematic diagram of the structure of a visual text generation model provided in one embodiment of this specification;

[0056] Figure 6 This is a front-end diagram illustrating a video text generation method provided in one embodiment of this specification;

[0057] Figure 7 This is a schematic diagram of the structure of a document generation device provided in one embodiment of this specification;

[0058] Figure 8 This is a schematic diagram of the structure of a video text generation device provided in one embodiment of this specification;

[0059] Figure 9 This is a schematic diagram of the structure of a visual text generation model training device provided in one embodiment of this specification;

[0060] Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0061] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0062] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0063] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0064] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0065] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0066] Agent: An intelligent agent that can autonomously perceive its environment and take actions to achieve specific goals.

[0067] Caption: A brief textual description of the image / video content.

[0068] FFN (FeedForward Network): A feedforward neural network is a multi-layered neural network that does not contain recurrence or feedback. It is an important sub-component at each position in the transformer structure.

[0069] LoRA (Low-Rank Adaptation) is an efficient fine-tuning method that adapts to new tasks by adjusting only a very small number of parameters through low-rank decomposition of the weight matrix.

[0070] Transformers (Transformer Models): A deep learning model architecture based on self-attention mechanism, widely used in natural language processing tasks such as BERT and GPT.

[0071] MoE (Mixture of Experts): A hybrid model that distributes input to one of multiple "expert" subnetworks for processing, thereby improving model capacity and efficiency.

[0072] Embedding: Embedding refers to the technique of mapping high-dimensional sparse data (such as words) to a low-dimensional dense vector space, which makes it easier for machine learning models to process.

[0073] MLP (Multi-Layer Perceptron): A multilayer perceptron is a feedforward neural network containing multiple layers of fully connected neurons.

[0074] Softmax: A normalization function often used in the classification output layer of a model to transform a real vector into a probability distribution.

[0075] With the deepening application of artificial intelligence technology in the field of content generation, users' demand for personalized and stylized text generation is increasing. Especially against the backdrop of rapid development of video content, higher requirements are placed on the ability to generate intelligent copy that adapts to diverse scenarios.

[0076] However, traditional methods face significant challenges in generating multi-style copy. Full-scale fine-tuning of the entire model incurs high training and storage costs; training with mixed data of multiple styles can easily lead to mutual interference of style features, resulting in blurred styles or degraded expressions in the generated results; at the same time, it is still difficult to balance generation quality and style accuracy while controlling the model size.

[0077] To address the aforementioned issues, existing technologies attempt to decouple the learning capabilities of different styles to independent sub-models or adaptation modules by introducing a modular structure, and then dynamically select or combine outputs using gating mechanisms to reduce training overhead and improve style control capabilities.

[0078] However, such methods still have limitations. The gating mechanism lacks fine-grained control over the style allocation strategy, making it difficult to accurately match complex or mixed style requirements. Insufficient coordination between multiple style modules may lead to inconsistent output. At the same time, the unbalanced load between modules affects the overall stability and scalability of the model, which restricts its widespread application in real-world scenarios.

[0079] To address the aforementioned issues, this specification provides a video text generation method, a visual text generation model training method, a text generation device, a video text generation device, a visual text generation model training device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0080] See Figure 1 , Figure 1 This is a flowchart of a text generation method provided in one embodiment of this specification, which specifically includes the following steps.

[0081] Step 102: Obtain the style information and visual data description text of the target style, and input the style information and visual data description text into the visual copy generation model. The visual copy generation model includes a gating network, a low-rank expert network with multiple preset styles, and a style fusion network.

[0082] The style information of the target style is an input signal used to indicate the style to be expressed in the generated copy. This signal can be a style descriptive word in text form or an encoded vector representing a specific style. For example, style information can be a clear style semantic identifier such as "humorous," "formal," "literary," or "concise."

[0083] Visual data description text is a semantic textual description of visual content such as videos and images, containing key information such as the main subject, actions, scene, and relationships between objects. For example, a video description text could be "A young man is running on a city street, with skyscrapers at dusk in the background, and upbeat music playing."

[0084] The visual copywriting generation model is a neural network model used to generate copywriting content that conforms to a specified style. Its structure includes a gating network, multiple low-rank expert networks with preset styles, and a style fusion network. This model is built on a pre-trained large language model, with the backbone parameters kept frozen, and stylization capabilities are injected only through the low-rank expert networks.

[0085] In practical applications, the system first receives the target style information and the visual data description text to be processed from the user input. Then, the style information and visual data description text are vectorized to generate corresponding embedded representations. The vectorized inputs are jointly fed into the visual copywriting generation model as the initial input signal. The gating network within the model receives the embedded representations of the style information for activation decisions in the subsequent expert network; the low-rank expert network receives the semantic information of the visual data description text, preparing for stylized semantic understanding; and the style fusion network integrates the outputs of multiple experts in subsequent stages.

[0086] For input processing, one option is to extract features from style information and visual data description text separately using independent encoders, and then concatenate or fuse them using cross-attention. Another option is to concatenate the two in a specific order into a single input sequence and then feed them into a shared encoder for processing. For the model input format, one option is to use a structure of [CLS] style information [SEP] description text [SEP]. Another option is to map them separately through a multimodal embedding layer and then align them in the feature space. The above implementation methods can be flexibly chosen according to the specific model architecture and are not limited thereto.

[0087] For example, in a video content creation task, a user wants to generate "motivational" text for a video of someone running in the city. The system receives the user's input style information "motivational" and the visual data description text generated by the video understanding module: "A young man is running on a city street, with skyscrapers at dusk in the background, and upbeat music playing." The system converts "motivational" into a corresponding style embedding vector and encodes the description text into a semantic vector. Then, these two vectors are concatenated into a unified input sequence, formatted as [CLS]motivational[SEP]A young man is running on a city street, with skyscrapers at dusk in the background, and upbeat music playing [SEP], and fed into the visual text generation model. The model begins processing the input. The gating network analyzes the style feature distribution corresponding to "motivational" semantically, preparing to activate relevant expert networks; multiple low-rank expert networks with preset styles receive the semantic information of the description text, waiting to be selected and invoked; the style fusion network is in standby mode, ready to receive subsequent multi-expert outputs and integrate them. Once the entire input process is complete, the system enters the expert selection and stylization generation stage.

[0088] Step 104: Using a gating network, based on style information, determine at least one target low-rank expert network from multiple low-rank expert networks.

[0089] Gated networks are neural network modules used to generate expert selection weights based on input style information. Their function is to map style semantics to the activation distribution of expert networks. This network can be built on a fully connected structure or a lightweight attention mechanism, receiving the embedded representation of style information as input and outputting the activation weights of each low-rank expert network.

[0090] Style information, which is the semantic input indicating the style to be generated for the target, has been encoded as a vector representation in the preceding steps and serves as the input signal to the gating network to guide the selection process of the expert network.

[0091] Low-rank expert networks are parameterized subnetworks built based on low-rank matrix factorization techniques. Each expert network corresponds to a specific style of expressive ability and is used to perform stylized semantic transformation on the input text. Multiple low-rank expert networks constitute an expert library, supporting parallel modeling of different styles.

[0092] In practical applications, the system embeds style information into the gating network. The gating network calculates a set of expert weights using internal parameters, representing the activation strength of each low-rank expert network under the current style. The system then filters expert networks based on these weights, selecting several experts with weights higher than a set threshold or those ranked highly as target low-rank expert networks. Weight allocation can be normalized using the softmax function to ensure the interpretability of the output distribution. For the expert selection mechanism, one option is to directly select all experts with non-zero weights; another option is to use a top-k selection strategy, retaining only the k experts with the highest weights to control computational overhead. For the weight generation process, one option is to normalize the gating network's output score after temperature coefficient adjustment; another option is to introduce sparsity constraints to encourage a more focused weight distribution in the gating network output. These implementation methods can be flexibly configured according to model performance and efficiency requirements.

[0093] For example, in the video script generation task, the system has received the style information "motivation" and the corresponding visual description text, and completed the vectorization input. The gating network receives the embedding vector of "motivation," calculates it through the internal fully connected layer, and outputs a set of expert weights, for example: [0.05, 0.82, 0.03, 0.76, 0.01, 0.68], corresponding to six preset style experts (such as formal, passionate, humorous, striving, artistic, and determined). Based on this weight distribution, the system selects the experts with weights higher than 0.7, namely the three experts of passionate, striving, and determined, as the target low-rank expert network. These three experts will be activated and participate in subsequent stylization processing, while the remaining experts remain inactive. The output of the gating network reflects the semantic association strength between the composite style "motivation" and multiple single-style experts, providing a selection basis for subsequent parallel stylization processing.

[0094] If the style selection mechanism relies on manual configuration or simple matching rules, it is difficult to establish a dynamic mapping between style semantics and model parameters, leading to inaccurate expert network selection and an inability to adapt to the expression needs of complex or mixed styles. To address this, a further method utilizes a gating network to determine at least one target low-rank expert network from multiple low-rank expert networks based on style information. This includes: using a gating network, based on the remapping relationship between style and multiple low-rank expert networks, mapping style information to the expert network weights of multiple low-rank expert networks, where the remapping relationship is obtained by the gating network through pre-training on labeled data of multiple preset styles; and using a gating network, based on the expert network weights, determining at least one target low-rank expert network from multiple low-rank expert networks.

[0095] The expert network weights are numerical vectors output by the gating network, representing the activation strength or contribution of each low-rank expert network under the current style, and are used in the subsequent expert selection and fusion process.

[0096] The remapping relationship is a nonlinear mapping function established within the gating network from style semantics to expert weight distribution. This function is obtained through supervised training and can map a single or composite style to a weight combination of multiple related experts, achieving fine-grained expert allocation.

[0097] In practical applications, the gating network receives the embedded representation of style information and calculates a set of expert network weights based on its internally trained remapping relationships. This remapping relationship is obtained through supervised training of the gating network using labeled data of a preset style. During training, the system provides style labels and a subset of experts to be activated, enabling the gating network to learn a reasonable weight allocation pattern from style to expert. The generated expert network weights reflect the degree of fit of each low-rank expert network to the current style. Based on these weights, the system selects at least one target low-rank expert network from multiple low-rank expert networks. The selection method can be based on a weight threshold or a ranking mechanism. For weight generation, one option is to directly output the raw score from the gating network and then normalize it using a softmax function; another option is to introduce a temperature coefficient to adjust the distribution smoothness, or to add sparsity constraints to promote weight focus. For target expert determination, one option is to select experts with weights higher than a preset threshold; another option is to use a top-k strategy to retain the k experts with the highest weights. These implementation methods can be flexibly adjusted according to the model's accuracy and efficiency requirements.

[0098] In the embodiments of this specification, style information is mapped to expert weights based on the remapping relationship obtained through training by a gating network, thereby realizing automated and refined routing from style semantics to the expert network and improving the accuracy and flexibility of style expression.

[0099] For example, in the task of generating "motivational" copy, the system inputs the style information "motivational" into a gating network. Based on the remapping relationship learned during training—that is, the association pattern between "motivational" and style experts such as "excitement," "vigor," and "determination"—the gating network outputs expert network weights of [0.05, 0.82, 0.03, 0.76, 0.01, 0.68]. This remapping relationship is obtained during model training through supervised learning using sample data labeled "motivational," combined with the corresponding activated expert subsets (such as "excitement," "vigor," and "determination"). Based on these weights, the system employs a top-3 strategy, selecting the three experts with the highest weights: "excitement" (0.82), "vigor," (0.76), and "determination" (0.68) as the target low-rank expert network. Other experts are excluded due to their low weights and do not participate in subsequent calculations. This process achieves a precise mapping from the abstract style of "motivational" to multiple specific style experts, ensuring that subsequent generation can cover the core expressive dimensions of this composite style.

[0100] If the gating mechanism lacks a deep representation capability of style semantics, it is difficult to establish a complex mapping relationship between style description and expert networks, resulting in coarse-grained expert selection and inaccurate style responses, affecting the expressive quality of the generated copy. To address this, the gating network further includes a style embedding layer and a weight generation network. Using the gating network, based on the remapping relationship between style and multiple low-rank expert networks, style information is mapped to the expert network weights of multiple low-rank expert networks. This includes: using the style embedding layer to embed and encode style information into a style representation vector; and using the weight generation network, based on the remapping relationship between style and multiple low-rank expert networks, mapping the style representation vector to the expert network weights of multiple low-rank expert networks.

[0101] The style embedding layer is a feature encoding module in a gating network, used to convert discrete style information into a continuous vector representation. This layer can be implemented using a lookup table structure or a fully connected network, with style labels as input and a fixed-dimensional style representation vector as output.

[0102] The weight generation network is a mapping computation module in the gated network. It receives the style representation vector as input and outputs the activation weights of each low-rank expert network. This network learns a non-linear remapping relationship between style representations and expert weights through training, supporting dynamic and fine-grained expert allocation.

[0103] The style representation vector is a continuous vector representation of style information after being encoded by the style embedding layer. It carries the semantic features of the style and serves as the input signal for the weight generation network.

[0104] In practical applications, the system first inputs style information into the style embedding layer of the gating network. The style embedding layer converts the style information into a fixed-dimensional style representation vector by looking up a pre-trained embedding table or using a learnable transformation function. This vector, as a high-order semantic feature, is fed into the weight generation network. Based on its internally trained remapping relationships, the weight generation network performs a non-linear transformation on the style representation vector, progressively calculating and generating a set of expert network weights. This remapping relationship is learned through supervised signal learning during model training, ensuring that the style representation accurately maps to the activation patterns of relevant experts. The generated weight vector reflects the degree of fit of each low-rank expert network to the current style and is used to subsequently determine the target expert network. For style embedding, one option is to use a learnable embedding matrix for table lookup; another option is to map style descriptors to vectors using a text encoder. For weight generation, one option is to use a multilayer perceptron to implement non-linear mapping; another option is to introduce an attention mechanism to dynamically adjust the activation paths of different experts. The above implementation methods can be flexibly selected according to model complexity and performance requirements.

[0105] In the embodiments of this specification, the synergistic effect of the style embedding layer and the weight generation network enables a phased and high-precision mapping from style information to expert weights, thereby enhancing the gating network's ability to understand and respond to complex style semantics.

[0106] For example, when processing input with the style "motivation," the system feeds the text into the style embedding layer of the entry control network. The embedding layer maps "motivation" into a 128-dimensional style representation vector by looking up a pre-defined embedding table. This vector encodes the semantic features of the style. This vector is then fed into a weight generation network, which consists of two fully connected layers and a ReLU activation function. Its parameters have been trained using extensive style-expert label data to obtain remapping relationships. The weight generation network transforms this vector, outputting six-dimensional expert network weights: [0.05, 0.82, 0.03, 0.76, 0.01, 0.68], corresponding to six pre-defined low-rank expert networks. This process transforms the original style label into quantifiable expert activation signals, providing a precise basis for subsequent selection of target experts such as "excitement," "enthusiasm," and "determination." The entire mapping process is completed collaboratively by the style embedding and weight generation stages, improving the depth and accuracy of style semantic parsing.

[0107] If the weight generation mechanism lacks dynamic awareness of the expert network's operational status, some experts may be frequently activated while others remain idle for extended periods, resulting in uneven model load, low resource utilization efficiency, and impacting generation stability and style coverage. To address this, before mapping the style representation vector to the expert network weights of multiple low-rank expert networks based on the remapping relationship between style and these networks using the weight generation network, the following steps are taken: obtaining the activation frequencies and performance metrics of the multiple low-rank expert networks; and mapping the style representation vector to the expert network weights of these networks using the weight generation network, based on the remapping relationship, including: determining the activation strategy of the low-rank expert networks based on the activation frequencies and performance metrics, and under the control of the activation strategy, matching the remapping relationship between style and multiple low-rank expert networks based on the style representation vector to determine the expert network weights of these networks.

[0108] The activation frequency of a low-rank expert network is the historical number or proportion of times each expert is selected during the network inference process, reflecting their usage popularity in style response, and serving as one of the indicators for evaluating expert load status.

[0109] Performance metrics are quantitative data that measure the quality of each low-rank expert network's output, including style consistency scores, language fluency scores, or task accuracy of the generated text, used to evaluate the expert's expressive ability in a specific style.

[0110] The activation strategy is a control rule formulated based on the expert activation frequency and performance indicators, used to guide the weight generation network in adjusting its preference for expert activation during the mapping process. This strategy can dynamically adjust the weight output of high-load or low-performance experts to achieve load balancing and quality optimization.

[0111] In practical applications, before performing weight mapping, the system first obtains the historical activation frequency and performance metrics of each low-rank expert network. The activation frequency is calculated by the number of times each expert is selected in the statistical inference task, and the performance metrics are collected through offline evaluation or online feedback mechanisms. The system inputs this state information into the weight generation network as the basis for the activation strategy. Based on the activation frequency and performance metrics, the weight generation network determines the current activation strategy: for example, appropriately reducing the weight bias of experts with consistently high loads, and increasing the activation priority of experts with excellent performance but insufficient usage. Under this strategy control, the weight generation network combines style representation vectors to match the remapping relationship between styles and multiple low-rank expert networks, dynamically adjusting the output weights of each expert. For the implementation of the activation strategy, one option is to introduce a penalty term during weight calculation to suppress the output of experts with high activation frequencies; another option is to set a performance threshold, allowing only experts that meet the quality standards to participate in weight allocation. For updating state information, one option is to periodically update the activation frequency and performance metrics in batches; another option is to use a sliding window mechanism for real-time updates to ensure timely policy response. The above implementation methods can be flexibly configured according to the system's stability and flexibility requirements.

[0112] In the embodiments of this specification, by introducing expert activation frequency and performance indicators as the basis for weight generation, dynamic perception and balanced scheduling of expert network usage status are achieved, thereby improving the resource utilization efficiency and generation stability of the model.

[0113] For example, before processing the "motivation" style task, the system statistically analyzes the operating status of each low-rank expert network: the activation frequency of the "excited" expert is 0.45, with a performance score of 4.2 (out of 5); the activation frequency of the "spirited" expert is 0.38, with a performance score of 4.5; and the activation frequency of the "determined" expert is 0.12, with a performance score of 4.6. The system determines that the "excited" expert is overused, while the "spirited" and "determined" experts have excellent performance but are underused. Based on this, the weight generation network generates an activation strategy: applying slight weight suppression to the "excited" expert and granting weight gains to the "spirited" and "determined" experts. Subsequently, the system inputs the style representation vector of "motivation" into the weight generation network, performs remapping calculations under the control of this strategy, and outputs expert network weights of [0.05, 0.75, 0.03, 0.80, 0.01, 0.72]. Compared to the case without strategy control, the weight of the "excited" expert decreases from 0.82 to 0.75, while the weights of the "spirited" and "determined" experts increase from 0.76 and 0.68 to 0.80 and 0.72, respectively. This process maintains the accuracy of style expression while effectively guiding the model towards low-load, high-quality experts, thus achieving a balanced utilization of expert resources.

[0114] If the mapping from style to expert often adopts a one-to-one or fixed grouping mechanism, it is difficult to express the multidimensional semantic characteristics of complex styles, and the low reuse rate of expert resources leads to poor model scalability and an inability to effectively support collaborative expression and knowledge transfer between styles. Therefore, a further remapping relationship is defined as the mapping relationship between style, expert group, and the expert network weights of each low-rank expert network within the expert group, where any low-rank expert network belongs to at least one expert group.

[0115] An expert group is a logical collection of multiple low-rank expert networks, with each group focusing on modeling a specific style semantic space, such as "formal," "emotional," or "rhythmic" style spaces. Experts within the same group learn different expressive sub-patterns within this semantic space through differentiated training.

[0116] In practical applications, the system finds one or more expert groups corresponding to the input style. Each expert group contains multiple low-rank expert networks, which are trained to focus on different expressive dimensions under the semantics of that group's style. The remapping relationship is based on the matching results between style and expert group, activating relevant expert groups and assigning weights within each group according to expert characteristics. Since any low-rank expert network can belong to multiple expert groups, it can be invoked by different but semantically related styles. For example, an expert skilled in expressing "power" can belong to both the "excitement" and "determination" expert groups, and can be activated when the input style is "inspiring" or "vigorous." For expert group partitioning, one option is static grouping based on style semantic clustering results; another option is dynamically constructing temporary expert groups through a learnable gating mechanism. For expert affiliation, one option is to associate experts with multiple style categories simultaneously during training through label constraints; another option is to set multiple affiliation relationships for experts during model initialization and jointly optimize them during training. The above implementation methods can be flexibly selected according to the complexity of the style system and the flexibility requirements of the model.

[0117] In the embodiments of this specification, by introducing an expert group hierarchy and an expert multi-group affiliation mechanism, a fine-grained and highly reusable mapping from style to expert is achieved, supporting multi-expert collaborative expression of complex styles and cross-style knowledge sharing.

[0118] For example, the system pre-defines a "Vibrant Group" containing three experts: E1 (emphasizing explosive power), E2 (emphasizing rhythm), and E3 (emphasizing emotional high); and a "Striving Group" containing E2 (rhythm), E3 (emotional high), and E4 (emphasizing sustainability). When the input style is "Inspiring," the system determines that it is associated with both the "Vibrant" and "Striving" expert groups, thus activating four experts: E1, E2, E3, and E4. The remapping relationship generates weights of 0.78, 0.85, 0.80, and 0.72 for E1, E2, E3, and E4 based on the matching strength between the style and the two groups. E2 and E3, belonging to two expert groups simultaneously, receive higher weights during the fusion process. This mechanism allows the composite style "Inspiring" to invoke multiple relevant experts, and experts E2 and E3 can be reused in various style tasks such as "Vibrant," "Striving," and "Inspiring," improving the model's expressive power and parameter utilization efficiency.

[0119] Step 106: Using at least one target low-rank expert network, perform visual semantic understanding on the visual data description text to obtain at least one style copy.

[0120] The target low-rank expert network consists of one or more low-rank adapter modules selected by a gating network to participate in the current generation task. Each expert network has independent parameters for performing semantic transformations under a specific style. This network is embedded in the feedforward network layer of the pre-trained model and is activated only during forward propagation, while the backbone model parameters remain unchanged.

[0121] The visual data description text is a semantic textual description of the visual content. It has been encoded into a vector sequence in the previous steps and serves as the input content for the low-rank expert networks of each target for stylized semantic understanding.

[0122] The stylistic copy is an intermediate text representation generated under specific style constraints. Each target low-rank expert network independently processes the visual data descriptive text based on its trained style features, outputting semantically enhanced results for the corresponding style. These intermediate results are not yet fused, preserving the expressive characteristics of a single style.

[0123] In practical applications, the system inputs the vector representations of the visual data describing the text into each target low-rank expert network. Each target low-rank expert network internally performs a linear transformation on the input features using a low-rank matrix factorization structure, injecting a semantic shift specific to a particular style. This process is executed layer by layer in each transformer module of the model, ensuring that stylization influences the entire semantic understanding process. Each expert network processes the same input text in parallel, generating intermediate semantic representations in its own style. These intermediate representations are then used to generate preliminary stylized text fragments, i.e., stylized copy. For the processing method, one option is for each target low-rank expert network to independently compute the incremental output in the feedforward network layer and add it to the backbone path; another option is for multiple target low-rank expert networks to share the same input but each maintains an independent low-rank parameter matrix, ensuring decoupling of style features. For parallel computation, one option is to implement multi-expert synchronous forward propagation on a GPU device; another option is to dynamically allocate computational resources based on expert weights, prioritizing high-weight experts. The above implementation methods can be flexibly adjusted according to system resources and performance requirements.

[0124] For example, in the task of generating "inspirational" copy, the system has identified three low-rank expert networks: exhilaration, striving, and determination. The visual data description text, "A young man runs on a city street, with skyscrapers at dusk in the background, accompanied by upbeat music," is simultaneously fed into these three expert networks. The exhilaration expert network strengthens semantic features related to heightened emotions, generating intermediate representations leaning towards expressions like "passion" and "explosiveness." The striving expert network enhances semantic weights for words like "persistence" and "goal," generating expressions leaning towards sustained effort. The determination expert network emphasizes the semantic strength of keywords like "belief" and "never give up." Each of the three expert networks outputs corresponding stylized semantic results, from which the system generates three preliminary stylized copy: such as "He sprints with all his might, every step full of power!" "Running without stopping, firmly moving towards the goal." "At dusk, he writes his perseverance with his footsteps." These intermediate results retain the unique expression of their respective styles, providing diverse inputs for subsequent integration.

[0125] Step 108: Using a style fusion network, generate visual data copy for the target style based on at least one style copy.

[0126] Style fusion networks are neural network modules used to integrate copywriting from multiple styles. Their function is to weightedly merge intermediate outputs from different target low-rank expert networks to generate a unified final copy. This network can be implemented based on linear weighting, attention mechanisms, or feedforward network structures. The input is the semantic representation of each style of copywriting, and the output is the fused target copywriting.

[0127] The target style visual data copy is the final generated complete copy that meets the user's specified style requirements. Its content retains the original visual information and integrates multiple style features to meet diverse expression needs.

[0128] In practical applications, the system inputs at least one style of text generated in the preceding steps into the style fusion network. The fusion network integrates the texts of each style using a weighted approach based on the expert weight distribution provided by the gating network. The weight values ​​reflect the contribution of each expert to the target style, with the output of high-weight experts dominating the fusion result. The fusion process can employ a weighted summation method, adding the semantic vectors of each style of text according to their weight coefficients to obtain the fused semantic representation; alternatively, an attention mechanism can be used to dynamically calculate the importance score of each text and perform soft merging. The fused semantic representation is then fed into the decoder to generate natural language text, forming the final visual data text. For the fusion strategy, one option is to directly use the original weights output by the gating network for linear weighting; another option is to introduce learnable fusion parameters and dynamically adjust the weight distribution based on the context. For output generation, one option is to directly map the fused vectors to a word sequence; another option is to generate the word-by-word sequence through an autoregressive approach to ensure grammatical coherence. These implementation methods can be flexibly configured according to the requirements for generation quality and efficiency.

[0129] For example, in the task of generating "motivational" copy, the system has obtained three style copy: "He sprints with all his might, every step full of power!" "Running without stopping, moving firmly towards the goal." "Under the twilight, he writes perseverance with his footsteps." The style fusion network receives the semantic vectors of these three texts and performs weighted merging based on the weights output by the gating network (e.g., exhilaration 0.82, striving 0.76, and perseverance 0.68). The fusion network uses a weighted summation method to add the three vectors according to their weight ratios, resulting in a comprehensive semantic representation. This representation includes both exhilarating elements such as "sprint" and "power," and also incorporates striving features such as "goal" and "moving forward," while retaining the firm expressions such as "perseverance" and "writing." Subsequently, the system inputs the fused semantic representation into the decoder to generate the final copy: "He sprints with all his might on the streets at dusk, every step demonstrating his firm belief in the goal, writing perseverance with his running!" This copy fully expresses the original visual content while presenting a multi-layered sense of motivation, achieving a natural fusion of multiple styles.

[0130] If fusion methods often use fixed weights or simple splicing to integrate multiple outputs, it is difficult to reflect the differences in contributions of each expert under the target style, leading to style imbalance or expression conflicts in the generated results, affecting the overall consistency and quality of the copy. To address this, a further approach is taken: using a style fusion network, based on at least one style of copy, to generate visual data copy in the target style. This includes: using a style fusion network, based on the expert network weights of multiple low-rank expert networks, to perform weighted fusion of at least one style of copy to generate visual data copy in the target style.

[0131] In practical applications, the system inputs at least one style text generated in the preceding steps, along with its corresponding expert network weights, into the style fusion network. The fusion network weights the semantic representations of each style text based on these expert network weights. Specifically, the vector representations of each style text are weighted and summed according to their corresponding weights to obtain a unified semantic vector after fusion. This vector retains the core information of the original visual content while reflecting the contribution proportions of experts in different styles. The fused semantic vector is then fed into the decoder module to generate the target style visual data text in natural language form. For the fusion method, one option is to directly use the original weights output by the gating network for linear weighting; another option is to introduce a learnable scaling factor or nonlinear transformation to dynamically adjust the weight distribution and optimize the fusion effect. For the decoding process, one option is to use an autoregressive method to generate text word by word to ensure grammatical correctness; another option is to use a non-autoregressive generation strategy to improve output efficiency. These implementation methods can be flexibly configured according to the requirements of generation quality and inference speed.

[0132] In the embodiments of this specification, by introducing expert network weights as the fusion basis, the differentiated integration of multi-style copywriting is achieved, thereby improving the style coordination and expression accuracy of the generated results.

[0133] For example, in the task of generating "motivational" copywriting, the system has obtained three style copywritings: "He sprints with all his might, every step full of power!" (from the "Excitement" expert, weight 0.82), "Running without stopping, moving firmly towards the goal." (from the "Striving" expert, weight 0.76), and "At dusk, he writes perseverance with his footsteps." (from the "Determination" expert, weight 0.68). The style fusion network receives these three copywritings and their weights, and weights their semantic vectors according to the ratios of 0.82, 0.76, and 0.68 to generate a comprehensive semantic representation. This representation is closer to the exciting features such as "sprint" and "power" in the semantic space, while retaining the striving and determination elements such as "goal" and "perseverance." Subsequently, the system inputs the fusion vector into the decoder to generate the final copywriting: "He sprints with all his might on the streets at dusk, every step showing his firm belief in the goal, writing perseverance with running!" This copywriting is consistent with the expert weights in terms of style intensity distribution, with excitement as the dominant theme and striving and determination as supplementary the latter, achieving a natural and collaborative expression of multiple styles.

[0134] If stylization generation models rely on full fine-tuning or static architectures, the training cost is high and it is difficult to support refined modeling of multiple styles, resulting in limited style expressiveness and generation results prone to style mixing or expression degradation. To address this, a visual text generation model is further trained using the following steps: First, sample style information for multiple preset styles, sample visual data description text, and corresponding labeled visual data text for multiple preset styles are obtained. The sample style information and sample visual description text are then input into the text generation model, which includes a gating network, multiple low-rank expert networks, and a style fusion network. Using the gating network, at least one predictive low-rank expert network is determined from the multiple low-rank expert networks based on the sample style information. Using at least one predictive low-rank expert network, visual semantic understanding is performed on the sample visual data description text to obtain at least one predicted style text. Using the style fusion network, predicted visual data text is generated based on at least one predicted style text. The difference loss is calculated based on the predicted visual data text and the labeled visual data text. Based on the difference loss, the parameters of the gating network and multiple low-rank expert networks are iteratively trained until the training stops, thus obtaining the visual text generation model.

[0135] Sample style information is an input signal used to indicate the style category to which the training sample belongs. It can be in the form of style label or semantic description text, and is used as input to the gating network to trigger the activation of the corresponding expert network.

[0136] The sample visual data description text is a semantic textual description of the visual content, which serves as the content input for the model and is used to generate stylized copy that matches it.

[0137] The label visual data copy is a standard output text that matches the sample visual data description text and sample style information, and is used as a supervision signal to calculate the generation error.

[0138] The predicted visual data copy is an intermediate output generated by the model during training. It is generated collaboratively by a gating network, a low-rank expert network, and a style fusion network, and is used to calculate the difference loss by comparing it with the labeled copy.

[0139] Difference loss is a measure of semantic or sequence-level difference between the predicted visual data text and the labeled visual data text. It can be calculated using metrics such as cross-entropy loss, BLEU score, or semantic similarity, and is used to guide model parameter updates.

[0140] In practical applications, the system first acquires multiple training samples with preset styles. Each sample contains sample style information, sample visual data description text, and corresponding label visual data text. The sample style information and sample visual data description text are input into an initial text generation model, which includes a gating network, multiple low-rank expert networks, and a style fusion network, with the backbone parameters kept frozen. The gating network generates expert network weights based on the sample style information and determines at least one predictive low-rank expert network accordingly. Each predictive low-rank expert network processes the sample visual data description text in parallel, generating corresponding predicted style text. The style fusion network performs weighted fusion of multiple predicted style texts according to the expert weights to generate predicted visual data text. The system calculates the difference loss between the predicted visual data text and the label visual data text, and updates the parameters of the gating network and multiple low-rank expert networks based on this loss using a backpropagation algorithm. This process is repeated iteratively until the training stopping condition is met, ultimately obtaining a trained visual text generation model. For loss calculation, one option is to use sequence-level cross-entropy loss to supervise the generation process word by word; another option is to introduce semantic consistency loss, optimizing by comparing the vector similarity between the model-generated results and the tag text. For parameter updates, one option is to update only the trainable parameters of the gating network and the low-rank expert network, keeping the backbone model frozen; another option is to introduce a lightweight learning rate in the later fine-tuning stage to perform limited updates on some backbone layers. The above implementation methods can be flexibly selected according to training efficiency and model performance requirements.

[0141] In the embodiments of this specification, by training the gating network and the low-rank expert network end-to-end, the accurate mapping and collaborative optimization of style information to expert activation are achieved, thereby improving the model's adaptability to multi-style generation tasks and the generation quality.

[0142] For example, when training the "motivational" style model, the system acquires samples: the style information is "motivational," the sample visual data description text is "A young man runs on a city street, with skyscrapers at dusk in the background, and upbeat music playing," and the label visual data text is "He sprints through the dusky streets, each step demonstrating his unwavering belief in his goal, writing perseverance with his running!" The system inputs the style information and description text into the model, and the gating network outputs expert weights, activating three prediction low-rank expert networks: exhilaration, striving, and determination. The three experts each generate intermediate text, which is then weighted and integrated by the style fusion network to output the predicted visual data text. The system calculates the difference loss between the predicted text and the label text, for example, using the cross-entropy loss function to measure word-level differences. Subsequently, backpropagation is used to adjust the weight allocation logic of the gating network and the low-rank parameters of the three low-rank expert networks. This process iterates repeatedly on a training set containing thousands of samples until the model's generated results stably approach the label text, satisfying the training stopping condition, ultimately obtaining a visual text generation model that can be used for inference.

[0143] If the expert selection and fusion weights are designed separately during training, it is difficult to guarantee the consistency between style semantics, expert activation, and result fusion, leading to inconsistent model learning objectives and affecting style control accuracy and generation stability. To address this, a further approach is taken: using a gating network, based on sample style information, at least one target low-rank expert network is determined from multiple low-rank expert networks. This includes: using a gating network to map sample style information to the initial network weights of multiple low-rank expert networks; using a gating network, based on the initial network weights, determining at least one target low-rank expert network from multiple low-rank expert networks; and using a style fusion network, based on at least one predicted style text, generating predicted visual data text, including: determining initial fusion weights corresponding to at least one initial style text based on the initial network weights; and performing a weighted summation of the at least one initial style text according to the initial fusion weights to generate the initial visual data text of the sample style.

[0144] The initial network weights are numerical vectors output by the gating network, representing the activation intensity of each low-rank expert network under the current sample style. These weights are used to determine the target low-rank expert network and serve as the basis for the subsequent fusion process.

[0145] The predicted style copy is an intermediate text representation generated by each target low-rank expert network after independently processing sample visual data description text, reflecting the expressive features of a specific style sub-pattern.

[0146] The initial fusion weight is a coefficient used to weight and merge multiple initial style texts. Its value is determined based on the initial network weights to ensure that the expert activation intensity is consistent with the fusion contribution.

[0147] The initial visual data copy is an intermediate output generated by weighted fusion of multiple initial style copy. It serves as the model's prediction output during training and is used to compare with the labeled copy to calculate the loss.

[0148] In practical applications, the system inputs sample style information into a gating network. The gating network converts this information into a set of initial network weights through an internal mapping function, representing the response strength of each low-rank expert network to that style. Based on these initial network weights, the system selects at least one target low-rank expert network from multiple low-rank expert networks. The selection method can be based on weight thresholds or a top-k strategy. Each target low-rank expert network processes the sample visual data description text in parallel, generating corresponding initial style text. Subsequently, the system uses the initial network weights to determine the initial fusion weights for each initial style text. One option is to directly reuse the initial network weights as fusion coefficients; another option is to normalize or nonlinearly transform the initial network weights before using them as fusion weights. The style fusion network performs a weighted summation of the semantic vectors of multiple initial style texts according to the initial fusion weights, generating the initial visual data text for the sample style. For the weight reuse mechanism, one option is to maintain strong consistency between the gating output and the fusion weights during training; another option is to introduce a learnable adaptation layer to fine-tune the fusion weights based on the initial network weights to optimize the fusion effect. The above implementation method can be flexibly configured according to the requirements of model consistency and expression flexibility.

[0149] In the embodiments described in this specification, the initial network weights output by the gated network are reused as the fusion basis, thereby achieving synergistic optimization of expert selection and result fusion and enhancing the consistency and stability of style control during model training.

[0150] For example, when training the "motivation" sample, the system inputs the sample style information "motivation" into the gating network. The gating network outputs initial network weights of [0.05, 0.82, 0.03, 0.76, 0.01, 0.68], corresponding to six low-rank expert networks. The system uses a top-3 strategy, selecting the three experts with the highest weights (excitement, striving, and determination) as the target low-rank expert networks. The three experts process the descriptive text "A young man is running on a city street...", generating three initial style texts. The system uses the corresponding weights of 0.82, 0.76, and 0.68 from the initial network weights as initial fusion weights, directly used for weighted fusion. The style fusion network weights and sums the semantic vectors of the three initial style texts in a ratio of 0.82:0.76:0.68 to generate the initial visual data text: "He sprints with all his might, running tirelessly, writing perseverance with his footsteps." This process ensures the consistency between the expert activation intensity and the final fusion contribution, enabling the model to stably learn the end-to-end mapping relationship from style to expression during training.

[0151] If stylization training data acquisition methods often rely on a single source or static dataset, the sample coverage is limited and the diversity of style expression is insufficient, resulting in weak generalization ability of the model in complex or mixed style scenarios, and the generated results are prone to problems such as single pattern or style deviation. To address this, further steps are taken to obtain sample style information, sample visual data description text, and corresponding label visual data text for multiple preset styles, including: collecting training data matching each preset style; expanding the training data through data augmentation and diversity sampling strategies to obtain sample style information, sample visual data description text, and corresponding label visual data text.

[0152] The preset style is a predefined category of copywriting expression used to guide the model to learn the ability to generate specific semantic tendencies, such as "formal", "humorous", "literary", and "concise" style types, which serve as the classification basis for data collection and model training.

[0153] In practical applications, the system collects raw training data that semantically matches each preset style from content libraries, annotation platforms, or multimodal data sources. Collection methods may include keyword-based retrieval, style tag filtering, or manual screening. For each collected data point, the corresponding sample visual data description text and label visual data text are extracted, and the corresponding sample style information is annotated. Subsequently, the system applies data augmentation and diversity sampling strategies to the training data for each style to expand the sample size and improve data diversity. Data augmentation methods include synonym substitution, sentence transformation, style transfer rewriting, adding noise, or context expansion to generate new samples that are semantically consistent but expressively diverse. Diversity sampling strategies include representative sampling based on semantic clustering, long-tail style oversampling, hard example mining sampling, or cross-style hybrid sampling to balance data distribution and cover more expression patterns. The augmented and sampled data constitute the final training set, containing rich sample style information, sample visual data description text, and corresponding label visual data text. Regarding data sources, one option is to extract descriptions and text pairs from highly-rated videos in user-generated content; another option is to manually generate stylized annotation data through a crowdsourcing platform. Regarding enhancement strength, one option is to dynamically adjust the enhancement ratio based on the amount of style data, with greater enhancement for styles with less data; another option is to introduce a generative model to assist in synthesizing stylized text, thereby improving data diversity. The above implementation methods can be flexibly configured according to data quality and training requirements.

[0154] In the embodiments described in this specification, a high-quality and diverse training set was constructed by collecting data by style and combining enhancement and sampling strategies, providing sufficient supervision signals for the model to learn refined style expressions.

[0155] For example, when constructing training data for the "inspirational" style, the system first retrieves videos tagged with "inspirational," "sports," and "struggle" from the video platform's content library, extracting automatically generated visual description text and highly-rated user comments as initial matching data. Subsequently, comments matching the "inspirational" expression characteristics are revised into standard tag visual data text through manual annotation, such as revising "He ran so hard" to "He sprinted with all his might on the streets at dusk, each step demonstrating his unwavering belief in his goal." The system enhances the original data: generating variations such as "He ran hard on the city roads under the sunset" through synonym substitution; generating expressions such as "On the streets at dusk, he interpreted perseverance through running" through sentence transformation; and supplementing low-frequency but highly expressive samples such as "In slow motion replay, sweat dripped, but his steps never stopped" through difficult example sampling. Finally, a "inspirational" training subset containing hundreds of samples is obtained, each sample containing the style information "inspirational," visual description text, and corresponding tag text, used for model training.

[0156] If training methods frequently employ serial optimization or fixed expert structures, it becomes difficult to achieve collaborative learning of style control and expressive ability, leading to a disconnect between gating decisions and expert outputs, thus affecting model convergence efficiency and style response accuracy. To address this, a further approach is taken: based on difference loss, the parameters of the gating network and multiple low-rank expert networks are iteratively trained until the training stops, resulting in a visual text generation model. This includes: assigning multiple low-rank expert networks as multiple expert modules trained in parallel based on sample style information; and updating the parameters of the gating network and multiple expert modules in parallel through gradient backpropagation based on difference loss until the training stops, thus obtaining the visual text generation model.

[0157] The expert module is a logical structure that manages the low-rank expert network as an independent computing unit during training, supporting parallel forward computation and gradient updates, thereby improving training efficiency.

[0158] In practical applications, the system organizes multiple low-rank expert networks into multiple parallel-trained expert modules during the training phase. Each expert module can independently receive input, perform forward computation, and receive gradient signals. For each sample data, the gating network generates expert weights based on the sample style information and activates the relevant expert modules. Each activated expert module processes the sample visual data description text in parallel, generating corresponding initial style text, which is then weighted and integrated by a style fusion network to output the predicted visual data text. The system calculates the difference loss between this prediction result and the labeled visual data text, and simultaneously updates the parameters of the gating network and multiple expert modules through the gradient backpropagation algorithm. The parameter update process is repeated in each training batch, allowing the gating network to learn a more accurate style-to-expert mapping relationship, while each expert module optimizes its semantic transformation capabilities under a specific style. For the parallel update mechanism, one option is to implement synchronous gradient computation and parameter updates for multiple expert modules on a GPU device; another option is to dynamically allocate computing resources based on expert weights, performing full gradient backpropagation only for high-weight experts to reduce computational overhead. For parameter freezing strategies, one option is to keep the pre-trained backbone model parameters completely frozen, only updating the trainable parameters of the gating network and low-rank expert network; another option is to introduce an extremely low learning rate during the later fine-tuning stage to perform limited optimization on some backbone layers. These implementations can be flexibly adjusted according to training efficiency and model performance requirements.

[0159] In the embodiments described in this specification, by using a low-rank expert network as a parallel training module and co-optimizing it with a gating network, joint learning of style control and style expression ability is achieved, thereby improving the training efficiency and style response consistency of the model.

[0160] For example, when training "motivational" samples, the system uses three low-rank expert networks—excitement, striving, and determination—as expert modules trained in parallel. A gating network generates weights based on the "motivational" style information and activates these three modules. The three modules process the descriptive text "A young man is running on a city street..." in parallel, generating intermediate text, which is then fused to output the predicted result. The system calculates the difference loss between the predicted text and the labeled text, for example, using the cross-entropy loss function. Subsequently, gradient backpropagation simultaneously updates the mapping parameters from "motivational" to the three experts in the gating network, as well as the low-rank matrix parameters of each of the three expert modules. In subsequent batches, if an expert consistently contributes little or generates poor quality text, its weight will be automatically reduced by the gating network, and the parameter update magnitude will also decrease accordingly. This process is iteratively executed on thousands of samples until the model output stabilizes, the loss converges, and finally, a trained visual text generation model is obtained.

[0161] This copywriting generation method utilizes a gating network to dynamically select and combine multiple expert networks with preset styles based on input style information, enabling the model to flexibly respond to diverse style requirements. Each expert network processes the visual descriptive text in parallel, generating intermediate copy in its corresponding style while preserving the expressive characteristics of different styles. A style fusion network weighted and integrates the multiple outputs, avoiding style rigidity caused by a single expert's dominance and improving the richness and accuracy of the generated copy. The overall architecture achieves refined style control and synergistic optimization of generation quality.

[0162] See Figure 2 , Figure 2 This is a flowchart of a video script generation method provided in one embodiment of this specification, which specifically includes the following steps.

[0163] Step 202: Obtain the style information of the target style and the video description text of the target video, and input the style information and video description text into the visual copywriting generation model. The visual copywriting generation model includes a gating network, a low-rank expert network with multiple preset styles, and a style fusion network.

[0164] Step 204: Using a gating network, based on style information, determine at least one target low-rank expert network from multiple low-rank expert networks.

[0165] Step 206: Using at least one target low-rank expert network, perform visual semantic understanding on the video description text to obtain at least one style video script.

[0166] Step 208: Using a style fusion network, generate a video script of the target style based on at least one style video script.

[0167] It is understood that the video description text is the same as the visual data description text in step 102, and the visual data copy is the same as the video copy in step 108. The execution process of steps 202-208 is the same as that of steps 102-108 mentioned above, and will not be repeated here.

[0168] This video script generation method utilizes a gating network to parse input style information, dynamically activating multiple adapted low-rank expert networks to generate intermediate scripts with distinct style characteristics in parallel, effectively preserving the expressive features of different styles. The style fusion network weights and integrates multiple outputs based on gating weights, avoiding the rigidity caused by a single dominant style. Through multi-expert collaboration and fusion mechanisms, the diversity and accuracy of generated scripts are improved, while also achieving a refined response to complex or mixed style requirements, thus achieving synergistic optimization of style control and generation quality.

[0169] See Figure 3 , Figure 3This is a flowchart of a visual text generation model training method provided in one embodiment of this specification, which specifically includes the following steps.

[0170] Step 302: Obtain sample style information, sample visual data description text, and corresponding label visual data copy for multiple preset styles, and input the sample style information and sample visual description text into the copy generation model. The copy generation model includes a gating network, multiple low-rank expert networks, and a style fusion network.

[0171] Step 304: Using a gating network, based on sample style information, determine at least one predictive low-rank expert network from multiple low-rank expert networks.

[0172] Step 306: Using at least one predictive low-rank expert network, perform visual semantic understanding on the visual data description text of the samples respectively, and obtain at least one predictive style copy.

[0173] Step 308: Using a style fusion network, generate predictive visual data copy based on at least one predictive style copy.

[0174] Step 310: Calculate the difference loss based on the predicted visual data text and the tag visual data text.

[0175] Step 312: Based on the difference loss, iteratively train the parameters of the gating network and multiple low-rank expert networks until the training stops, and obtain the visual text generation model.

[0176] It is understandable that steps 302-312 are the same as the training steps of the aforementioned visual text generation model, and will not be repeated here.

[0177] The training method applied to this visual text generation model achieves efficient parameter fine-tuning by decoupling style learning to an independent low-rank expert network. Multiple style adaptation can be completed by updating only a small number of parameters in the low-rank expert network, significantly reducing training costs. The collaborative training of the gating network and expert network enables the model to automatically learn the optimal mapping relationship between styles and experts, improving the accuracy of style assignment. The multi-expert parallel training mechanism enhances the model's ability to capture different style semantic subspaces, avoids feature mixing, and improves expert utilization through load balancing strategies, ensuring the model's stability and scalability.

[0178] The following describes the front-end interaction process of the text generation method and video text generation method provided in the embodiments of this specification. Please refer to [link / reference]. Figure 4a , Figure 4a This is a schematic diagram of a task material selection interface provided in one embodiment of this specification, such as... Figure 4aAs shown, the task material selection interface can be displayed, which includes multiple media materials, as well as the video generation control "One-Click Film Generation".

[0179] In response to the media material selection operation in the task material selection interface and the triggering of "one-click video creation", multiple task materials for the content generation task in this embodiment can be obtained. Based on these multiple task materials, the specific implementation processes of the above-described text generation method and video text generation method can be executed to automatically generate the corresponding target text content or target video. Figure 4b For example, Figure 4b This is a schematic diagram of a video loading interface provided in one embodiment of this specification. The client can respond to the "One-Click Video Generation" trigger operation of the video generation control in the task material selection interface to obtain the selected media material. The server can generate a video inference process based on the selected media material. The client can determine and display the video inference process 302a on the video loading interface. The inference process 302a includes at least one of the following inference information: at least one highlight segment 302b in the media material, at least one material content description 302g, video theme 302c, video content description 302d, and video content summary 302e. The inference information is determined based on the selected at least one media material. The aforementioned video theme 302c, video content description 302d, and video content summary 302e are generated based on a caption-based and gated multi-expert network.

[0180] by Figure 4b For example, the reasoning process includes a summary of the video content, 302e, which is "An Unforgettable Trip, with everyday narrative text, accompanied by relaxing music, and packaged in a simple, everyday style." Here, the video title is "An Unforgettable Trip"; the video text is "everyday narrative text," which can be understood as a video text type; the background music is "accompanied by relaxing music," which can be understood as a video background music type; the video style is "simple, everyday style packaging," which can be understood as a video style type. The video voiceover can be understood as a video voiceover type, such as "funny voice."

[0181] In addition, the video loading interface also includes a command input field, through which update commands for at least one type of inference information are received. In response to the update command, the inference process updated based on the update command is displayed or dynamically displayed on the video loading interface. If the video inference process displayed on the video loading interface does not meet the video generation requirements, an update command can be input, and the updated inference process can then be displayed on the video loading interface. Continuing... Figure 4bFor example, the video loading interface includes a command input control 316a. Clicking the command input control 316a allows the client to respond to a trigger operation on the command input control 316a by pulling up the keyboard 318a in the video loading interface. Figure 4c As shown, Figure 4c This is a schematic diagram illustrating the update of a video loading interface according to one embodiment of this specification. When the command input control 316a is clicked, the keyboard changes from a hidden state to a raised state. An update command can be entered in the command input area 318b using the keyboard 318a. The entered update command can be displayed in the command input area 318b. The associated position of the command input area 318b may also include an input confirmation control 318c. The client can respond to the trigger operation of the input confirmation control 318c to confirm the entered update command and display the updated reasoning process in the video loading interface, such as... Figure 4d As shown, Figure 4d This is a schematic diagram of an updated video loading interface provided in one embodiment of this specification.

[0182] Continue with Figure 4b For example, the video loading interface also includes a video viewing control 302f. The client can respond to trigger operations on the video viewing control 302f and display, as shown below. Figure 4e The video browsing interface shown is as follows. Figure 4e This is a schematic diagram of a video browsing interface provided in one embodiment of this specification. The video browsing interface includes a video title, "An Unforgettable Trip," and a visual style, namely, a "simple style." For example... Figure 4e As shown, the video browsing interface also includes a video update control "Regenerate". If you are not satisfied with the generated video, or want to try generating other videos, you can click the video update control "Regenerate" in the video browsing interface. The client can respond to the trigger operation of the video update control "Regenerate" and display a new video in the video browsing interface. The new video is different from the previously displayed video.

[0183] The following is in conjunction with the appendix Figure 4f Taking the text generation method provided in this specification as an example of its application in video text generation, the text generation method will be further explained. Among other things, Figure 4f This is a flowchart illustrating the processing steps of a copywriting generation method provided in one embodiment of this specification, specifically including the following steps.

[0184] Step 402: Model architecture design and LoRA multi-expert network construction.

[0185] Specifically, this specification proposes a stylized large language model architecture based on matrix low-rank factorization and a mixture of experts model. By integrating multiple experts and a stylized gating mechanism in the feed-forward network layer of a pre-trained language model and using matrix low-rank factorization for supervised fine-tuning, efficient generation of specified-style copywriting is achieved. The overall architecture design is based on the Transformer structure of the pre-trained large language model, keeping the original model parameters completely frozen, and only introducing the LoRA expert network in the linear transformation layer of each FFN layer. The system adopts a style mixture of experts architecture, where each style corresponds to multiple independent LoRA adapter experts, and the gating network is trained to learn the weight mapping relationship from the style to the multiple experts.

[0186] The LoRA multiple expert network is constructed by deploying a total of M LoRA adapters as an expert library in the linear layer of each FFN layer. Each expert E j is composed of the low-rank matrix factorization A j and B j , where r << d is the rank of the low-rank factorization. For a specified style s, the gating network outputs the expert weight vector G(s) = [g1, g2,..., gm] corresponding to this style. The output calculation formula of the network is: o = W0x + (α / r) ∑ j j = 1 Wj · BjAjx, where W0x represents the output of the original pre-trained model, α is a scaling factor, and Wj is the activation weight of the j-th expert. The stylized expert grouping training strategy divides the total expert library into multiple expert groups, each group focusing on a specific style semantic space. Different experts learn different semantic subspaces and expression patterns of this style through random initialization and different data sampling strategies, forming complementary expert capabilities.

[0187] See Figure 5 shown in Figure 5 is a schematic structural diagram of a visual copywriting generation model provided by an embodiment of this specification. The model is composed of multiple cascaded transformer modules. Each transformer module sequentially includes an attention module, a root mean square normalization layer, and a feed-forward network layer. Among them, the attention module is used to capture the long-range semantic dependency relationships inside the input sequence; the root mean square normalization layer stabilizes the features to improve the training convergence. In the feed-forward network layer, multiple parallel low-rank adaptation experts (LoRA experts) are integrated. Each LoRA expert is composed of low-rank decomposition matrices A and B and is used to model the expression features of a specific style. The gating network generates the weight distribution of each LoRA expert according to the input style information to achieve dynamic selection and weighted activation. The outputs of each expert are fused and added to the backbone path, and finally the stylized features are output. The backbone parameters of the entire pre-trained model are kept frozen, and only efficient fine-tuning is achieved through the LoRA expert network and the gating network, supporting multi-style and low-overhead visual copywriting generation.

[0188] Step 404: Style Multi-Expert Mapping and Gated Network Design.

[0189] Specifically, the style multi-expert mapping mechanism establishes a complex mapping relationship from style to multiple experts, with each style s corresponding to a subset of experts. The system employs a weight distribution Gs. For example, the "Reflections and Insights" style might correspond to experts {1, 3, 7, 12}, while the "Daily Life" style might correspond to experts {2, 5, 9, 15}. Expert subsets for different styles may overlap, and the same expert can serve multiple related styles. The system learns the optimal expert combination and weight allocation for each style through training, achieving multi-expert collaborative representation of complex styles.

[0190] The gating network design employs a style embedding-based weight generation mechanism. It receives style identifiers `s` as input, maps discrete style identifiers to continuous style representation vectors through a style embedding layer, and then generates corresponding expert weight distributions through a multilayer perceptron. The computation process of the gating network is as follows: `style_emb = Embedding(s)`, `expert_weights = Softmax(MLP(style_emb))`, where `Embedding` is the style embedding layer, `MLP` is the weight generation network, and `Softmax` ensures that the sum of the weights is 1. A dynamic expert combination and load balancing mechanism ensures effective utilization and load balancing of different experts. The system monitors the activation frequency and performance of each expert, and adjusts the weight allocation of the gating network through expert importance analysis and load balancing strategies to avoid over-activation or long-term idleness of certain experts.

[0191] Step 406: Joint training strategy and expert weight optimization.

[0192] Specifically, the weight mapping relationship. The training data format is <style identifier, caption, target text>. The system activates the gating network using the specified style to obtain expert weights, then generates text through weighted expert combinations, calculates the generation loss, and backpropagates to optimize the gating network parameters. The training objective is to teach the gating network to select the optimal expert combination and weight allocation for each style.

[0193] Expert weight learning and optimization employs an end-to-end training strategy, jointly optimizing the weight generation of the gating network and the text generation of the experts. During training, for training samples of a given style, the gating network outputs an expert weight distribution. The outputs of multiple experts are weighted and summed to obtain the final result, and the loss function is calculated based on the difference between this result and the target text. Gradients simultaneously update the parameters of the gating network and the LoRA parameters of the relevant experts, achieving collaborative optimization. High-quality training data is collected for each style, and then, through data augmentation and diversity sampling, multiple experts are assigned to that style for parallel training, ensuring that the experts can fully learn the style features.

[0194] Step 408: Reasoning generation process and multi-expert collaboration mechanism.

[0195] Specifically, the inference and generation process receives a video caption and a specified style identifier as input. First, the style identifier is input into a gating network to obtain the expert weight distribution corresponding to that style. The system activates multiple relevant experts based on the weight distribution, with each expert independently processing the input caption to generate intermediate results. Then, the outputs of multiple experts are weighted and fused according to the weights output by the gating network to obtain the final stylized text. This entire process achieves an intelligent transformation from single-style specification to multi-expert collaborative generation, ensuring that complex styles can be fully expressed through the collaborative work of multiple experts, while maintaining inference efficiency and generation quality.

[0196] Applying to steps 402-408 above, through a style multi-expert mapping architecture design, a style-embedded gating network mechanism, the construction of an embedded LoRA expert library in the FFN layer, a multi-expert weighted fusion generation mechanism, an expert load balancing and reuse mechanism, and precise control over specified styles, high-quality and highly controllable multi-style copywriting generation capabilities are achieved while maintaining parameter efficiency. These technological innovations not only solve the problem that complex styles cannot be fully expressed by a single expert, but also significantly improve the richness and accuracy of style expression through intelligent style-to-expert mapping, integration of low-rank adapters into pre-trained models, and soft weight allocation strategies. Simultaneously, they greatly reduce training costs and storage overhead, improve expert utilization efficiency and the consistency of generated results, ultimately providing users with a precise, controllable, and diverse stylized copywriting creation experience, greatly enhancing the flexibility and diversity of content creation.

[0197] See Figure 6 As shown, Figure 6 This is a front-end diagram illustrating a video text generation method provided in one embodiment of this specification. Figure 6 The interface showcases a content sharing application with three channel options: "Following," "Discover," and "Nearby," which users can switch between using the tabs at the top. The "Details" button on the left displays detailed information, while the "Search" button on the right provides a search function. In the center of the screen, users can choose their preferred text style, such as formal, humorous, artistic, or concise, and upload video content via the "Upload Video" button below. The bottom navigation bar includes four buttons: "Home," "Videos," "Messages," and "Me," each corresponding to a different function page. The "Videos" button echoes the "Upload Video" button above, allowing users to quickly access video-related operations.

[0198] Corresponding to the above method embodiments, this specification also provides embodiments of a document generation device. Figure 7A schematic diagram of a document generation apparatus according to one embodiment of this specification is shown. Figure 7 As shown, the device includes:

[0199] The first acquisition module 702 is configured to acquire style information and visual data description text of the target style, and input the style information and visual data description text into the visual copywriting generation model. The visual copywriting generation model includes a gated network, a low-rank expert network with multiple preset styles, and a style fusion network.

[0200] The first determination module 704 is configured to use a gated network to determine at least one target low-rank expert network from multiple low-rank expert networks based on style information.

[0201] The first semantic understanding module 706 is configured to use at least one target low-rank expert network to perform visual semantic understanding on visual data description text, and obtain at least one style copy accordingly.

[0202] The first generation module 708 is configured to use a style fusion network to generate visual data copy of a target style based on at least one style copy.

[0203] Optionally, the first determining module 704 is further configured to use a gating network to map style information to the expert network weights of multiple low-rank expert networks based on the remapping relationship between style and multiple low-rank expert networks, wherein the remapping relationship is obtained by the gating network in advance based on the label data of multiple preset styles; and to use the gating network to determine at least one target low-rank expert network from multiple low-rank expert networks based on the expert network weights.

[0204] Optionally, the gating network includes a style embedding layer and a weight generation network; correspondingly, the first determining module 704 is further configured to use the style embedding layer to embed and encode style information into a style representation vector; and use the weight generation network to map the style representation vector to the expert network weights of multiple low-rank expert networks based on the remapping relationship between style and multiple low-rank expert networks.

[0205] Optionally, the first determining module 704 is further configured to obtain the activation frequency and performance index of multiple low-rank expert networks; use a weighted generation network to determine the activation strategy of the low-rank expert networks based on the activation frequency and performance index; and under the control of the activation strategy, match the remapping relationship between style and multiple low-rank expert networks based on the style representation vector to determine the expert network weights of multiple low-rank expert networks.

[0206] Optionally, the first determining module 704 is further configured to remap the relationship between style, expert group, and the expert network weights of each low-rank expert network in the expert group, wherein any low-rank expert network belongs to at least one expert group.

[0207] Optionally, the first generation module 708 is further configured to utilize a style fusion network to perform weighted fusion of at least one style copy based on the expert network weights of multiple low-rank expert networks, thereby generating visual data copy of the target style.

[0208] Optionally, the copywriting generation device further includes a training module configured to acquire sample style information of multiple preset styles, sample visual data description text, and corresponding tag visual data copywriting of multiple preset styles, and input the sample style information and sample visual description text into the copywriting generation model. The copywriting generation model includes a gating network, multiple low-rank expert networks, and a style fusion network. Using the gating network, based on the sample style information, at least one predictive low-rank expert network is determined from the multiple low-rank expert networks. Using the at least one predictive low-rank expert network, visual semantic understanding is performed on the sample visual data description text to obtain at least one predicted style copywriting. Using the style fusion network, predicted visual data copywriting is generated based on the at least one predicted style copywriting. A difference loss is calculated based on the predicted visual data copywriting and the tag visual data copywriting. Based on the difference loss, the parameters of the gating network and the multiple low-rank expert networks are iteratively trained until the training stops, thus obtaining the visual copywriting generation model.

[0209] Optionally, the training module is further configured to use a gating network to map sample style information to the initial network weights of multiple low-rank expert networks; use the gating network to determine at least one target low-rank expert network from the multiple low-rank expert networks based on the initial network weights; determine the initial fusion weights corresponding to at least one initial style copy based on the initial network weights; and perform weighted summation processing on at least one initial style copy according to the initial fusion weights to generate the initial visual data copy of the sample style.

[0210] Optionally, the training module is further configured to collect training data matching each preset style; and to expand the training data through data augmentation and diversity sampling strategies to obtain sample style information, sample visual data description text, and corresponding label visual data copy.

[0211] Optionally, the training module is further configured to assign multiple low-rank expert networks as multiple expert modules for parallel training based on sample style information; based on difference loss, the parameters of the gated network and the parameters of the multiple expert modules are updated in parallel through gradient backpropagation until the training stops, thus obtaining a visual text generation model.

[0212] Applied to this copywriting generation device, a gating network dynamically selects and combines multiple expert networks with preset styles based on input style information, enabling the model to flexibly respond to diverse style requirements. Each expert network processes the visual descriptive text in parallel, generating intermediate copy in its corresponding style while preserving the expressive characteristics of different styles. A style fusion network weighted and integrates the multiple outputs, avoiding style rigidity caused by a single expert's dominance and improving the richness and accuracy of the generated copy. The overall architecture achieves refined style control and synergistic optimization of generation quality.

[0213] The above is a schematic scheme of a copywriting generation device according to this embodiment. It should be noted that the technical solution of this copywriting generation device and the technical solution of the above-described copywriting generation method belong to the same concept. For details not described in detail in the technical solution of the copywriting generation device, please refer to the description of the technical solution of the above-described copywriting generation method.

[0214] Corresponding to the above method embodiments, this specification also provides embodiments of a video script generation device. Figure 8 A schematic diagram of a video text generation device according to one embodiment of this specification is shown. Figure 8 As shown, the device includes:

[0215] The second acquisition module 802 is configured to acquire style information of the target style and video description text of the target video, and input the style information and video description text into the visual copywriting generation model. The visual copywriting generation model includes a gated network, a low-rank expert network with multiple preset styles, and a style fusion network.

[0216] The second determination module 804 is configured to use a gated network to determine at least one target low-rank expert network from multiple low-rank expert networks based on style information.

[0217] The second semantic understanding module 806 is configured to use at least one target low-rank expert network to perform visual semantic understanding on the video description text and obtain at least one style video script accordingly.

[0218] The second generation module 808 is configured to use a style fusion network to generate a video script of a target style based on at least one style video script.

[0219] Applied to this video script generation device, a gating network parses input style information, dynamically activates multiple adapted low-rank expert networks, and generates intermediate scripts with distinct style characteristics in parallel, effectively preserving the expressive features of different styles. The style fusion network weights and integrates multiple outputs based on gating weights, avoiding the rigidity caused by a single style dominating. Through multi-expert collaboration and fusion mechanisms, the diversity and accuracy of generated scripts are improved, and a fine-grained response to complex or mixed style requirements is achieved, resulting in a synergistic optimization of style control capability and generation quality.

[0220] The above is an illustrative scheme of a video text generation device according to this embodiment. It should be noted that the technical solution of this video text generation device and the technical solution of the video text generation method described above belong to the same concept. For details not described in detail in the technical solution of the video text generation device, please refer to the description of the technical solution of the video text generation method described above.

[0221] Corresponding to the above method embodiments, this specification also provides embodiments of a visual text generation model training device. Figure 9 A schematic diagram of a visual text generation model training device according to one embodiment of this specification is shown. Figure 9 As shown, the device includes:

[0222] The third acquisition module 902 is configured to acquire sample style information of multiple preset styles, sample visual data description text, and corresponding label visual data copy of multiple preset styles, and input the sample style information and sample visual description text into the copy generation model. The copy generation model includes a gating network, multiple low-rank expert networks, and a style fusion network.

[0223] The third determination module 904 is configured to use a gating network to determine at least one predictive low-rank expert network from multiple low-rank expert networks based on sample style information.

[0224] The third semantic understanding module 906 is configured to use at least one predictive low-rank expert network to perform visual semantic understanding on the sample visual data description text, and obtain at least one predictive style copy accordingly.

[0225] The third generation module 908 is configured to use a style fusion network to generate predictive visual data copy based on at least one predictive style copy.

[0226] The calculation module 910 is configured to calculate the difference loss based on the predicted visual data copy and the label visual data copy;

[0227] Training module 912 is configured to iteratively train the parameters of the gating network and multiple low-rank expert networks based on difference loss until the training stops, thereby obtaining a visual text generation model.

[0228] The training device applied to this visual text generation model achieves efficient parameter fine-tuning by decoupling style learning to an independent low-rank expert network. Multiple style adaptation can be completed by updating only a small number of parameters in the low-rank expert network, significantly reducing training costs. The collaborative training of the gating network and expert network enables the model to automatically learn the optimal mapping relationship between styles and experts, improving the accuracy of style assignment. The multi-expert parallel training mechanism enhances the model's ability to capture different style semantic subspaces, avoids feature mixing, and improves expert utilization through load balancing strategies, ensuring the model's stability and scalability.

[0229] The above is a schematic scheme of a visual text generation model training device according to this embodiment. It should be noted that the technical solution of this visual text generation model training device and the technical solution of the above-described visual text generation model training method belong to the same concept. For details not described in detail in the technical solution of the visual text generation model training device, please refer to the description of the technical solution of the above-described visual text generation model training method.

[0230] Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0231] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0232] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0233] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1000 can also be a mobile or stationary server.

[0234] The processor 1020 is used to execute the following computer program / instruction, which, when executed by the processor, implements the steps of the above-mentioned text generation method, video text generation method, or visual text generation model training method.

[0235] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computing device embodiments are relatively simple in description because they are fundamentally similar to the text generation method, video text generation method, or visual text generation model training method embodiments. Relevant details can be found in the descriptions of the text generation method, video text generation method, or visual text generation model training method embodiments.

[0236] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described text generation method, video text generation method, or visual text generation model training method.

[0237] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computer-readable storage medium embodiments are described simply because they are substantially similar to the text generation method, video text generation method, or visual text generation model training method embodiments. Relevant details can be found in the descriptions of the text generation method, video text generation method, or visual text generation model training method embodiments.

[0238] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described text generation method, video text generation method, or visual text generation model training method.

[0239] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the above-mentioned text generation method, video text generation method, or visual text generation model training method. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solutions of the above-mentioned text generation method, video text generation method, or visual text generation model training method.

[0240] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0241] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0242] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0243] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0244] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for generating copy, characterized in that, include: The style information and visual data description text of the target style are obtained, and the style information and visual data description text are input into the visual copywriting generation model. The visual copywriting generation model includes a gating network, a low-rank expert network with multiple preset styles, and a style fusion network. Using the gating network, at least one target low-rank expert network is determined from multiple low-rank expert networks based on the style information; Using the at least one target low-rank expert network, visual semantic understanding is performed on the visual data description text to obtain at least one style copy. Using the style fusion network, visual data copy of the target style is generated based on the at least one style copy.

2. The method according to claim 1, characterized in that, The step of using the gating network to determine at least one target low-rank expert network from multiple low-rank expert networks based on the style information includes: Using the gated network, based on the remapping relationship between style and multiple low-rank expert networks, the style information is mapped to the expert network weights of multiple low-rank expert networks, wherein the remapping relationship is obtained by the gated network in advance based on the label data of the multiple preset styles; Using the gating network, at least one target low-rank expert network is determined from the plurality of low-rank expert networks based on the weights of the expert network.

3. The method according to claim 2, characterized in that, The gated network includes a style embedding layer and a weight generation network; the step of using the gated network to map the style information to the expert network weights of multiple low-rank expert networks based on the remapping relationship between style and multiple low-rank expert networks includes: The style information is embedded and encoded into a style representation vector using the style embedding layer. Using the weight generation network, the style representation vector is mapped to the expert network weights of multiple low-rank expert networks based on the remapping relationship between style and multiple low-rank expert networks.

4. The method according to claim 3, characterized in that, Before mapping the style representation vector to the expert network weights of the multiple low-rank expert networks based on the remapping relationship between style and multiple low-rank expert networks using the weight generation network, the method further includes: Obtain the activation frequency and performance metrics of multiple low-rank expert networks; The step of using the weight generation network to map the style representation vector to the expert network weights of multiple low-rank expert networks based on the remapping relationship between style and multiple low-rank expert networks includes: Using the weight generation network, based on the activation frequency and the performance index, the activation strategy of the low-rank expert network is determined. Under the control of the activation strategy, based on the style representation vector, the remapping relationship between the style and multiple low-rank expert networks is matched, and the expert network weights of the multiple low-rank expert networks are determined.

5. The method according to claim 2, characterized in that, The remapping relationship is a mapping relationship between style, expert group, and expert network weights of each low-rank expert network in the expert group, where any low-rank expert network belongs to at least one expert group.

6. The method according to any one of claims 2-5, characterized in that, The step of generating visual data copy for the target style based on the at least one style copy using the style fusion network includes: Using the style fusion network, based on the expert network weights of the multiple low-rank expert networks, the at least one style copy is weighted and fused to generate the visual data copy of the target style.

7. The method according to claim 1, characterized in that, The visual text generation model is trained using the following steps: The method acquires sample style information, sample visual data description text, and corresponding label visual data text for multiple preset styles, and inputs the sample style information and sample visual description text into a text generation model. The text generation model includes a gating network, multiple low-rank expert networks, and a style fusion network. Using the gating network, at least one predictive low-rank expert network is determined from multiple low-rank expert networks based on the sample style information; Using the at least one predictive low-rank expert network, visual semantic understanding is performed on the sample visual data description text to obtain at least one predictive style copy. Using the style fusion network, the predicted visual data text is generated based on the at least one predicted style text; Based on the predicted visual data text and the labeled visual data text, calculate the difference loss; Based on the difference loss, the parameters of the gated network and the multiple low-rank expert networks are iteratively trained until the training stops, thus obtaining a visual text generation model.

8. The method according to claim 7, characterized in that, The step of using the gating network to determine at least one target low-rank expert network from multiple low-rank expert networks based on the sample style information includes: Using the gating network, the sample style information is mapped to the initial network weights of multiple low-rank expert networks; Using the gated network, and based on the initial network weights, at least one target low-rank expert network is determined from the plurality of low-rank expert networks: The step of generating the predicted visual data text based on the at least one predicted style text using the style fusion network includes: The initial fusion weights corresponding to the at least one initial style copy are determined based on the initial network weights; The initial style copy is weighted and summed according to the initial fusion weight to generate the initial visual data copy of the sample style.

9. The method according to claim 7, characterized in that, The process of acquiring sample style information of multiple preset styles, sample visual data description text, and corresponding label visual data text of the multiple preset styles includes: For each preset style, training data matching each preset style is collected; By employing data augmentation and diversity sampling strategies, the training data is expanded to obtain sample style information, sample visual data description text, and corresponding label visual data text.

10. The method according to claim 7, characterized in that, The step of iteratively training the parameters of the gated network and the multiple low-rank expert networks based on the difference loss until the training stops, to obtain a visual text generation model, includes: Based on the sample style information, the multiple low-rank expert networks are assigned to multiple expert modules trained in parallel; Based on the difference loss, the parameters of the gated network and the parameters of the multiple expert modules are updated in parallel through gradient backpropagation until the training stops, thus obtaining the visual text generation model.

11. A method for generating video scripts, characterized in that, include: The style information of the target style and the video description text of the target video are obtained, and the style information and the video description text are input into the visual copywriting generation model. The visual copywriting generation model includes a gating network, a low-rank expert network with multiple preset styles, and a style fusion network. Using the gating network, at least one target low-rank expert network is determined from multiple low-rank expert networks based on the style information; Using the at least one target low-rank expert network, visual semantic understanding is performed on the video description text to obtain at least one style video script; Using the style fusion network, a video script of the target style is generated based on the at least one style video script.

12. A method for training a visual text generation model, characterized in that, include: The method acquires sample style information, sample visual data description text, and corresponding label visual data text for multiple preset styles, and inputs the sample style information and sample visual description text into a text generation model. The text generation model includes a gating network, multiple low-rank expert networks, and a style fusion network. Using the gating network, at least one predictive low-rank expert network is determined from multiple low-rank expert networks based on the sample style information; Using the at least one predictive low-rank expert network, visual semantic understanding is performed on the sample visual data description text to obtain at least one predictive style copy. Using the style fusion network, the predicted visual data text is generated based on the at least one predicted style text; Based on the predicted visual data text and the labeled visual data text, calculate the difference loss; Based on the difference loss, the parameters of the gated network and the multiple low-rank expert networks are iteratively trained until the training stops, thus obtaining a visual text generation model.

13. A copywriting generation device, characterized in that, include: The first acquisition module is configured to acquire style information and visual data description text of the target style, and input the style information and visual data description text into the visual copywriting generation model, wherein the visual copywriting generation model includes a gated network, a low-rank expert network with multiple preset styles, and a style fusion network. The first determining module is configured to use the gating network to determine at least one target low-rank expert network from a plurality of low-rank expert networks based on the style information. The first semantic understanding module is configured to use the at least one target low-rank expert network to perform visual semantic understanding on the visual data description text respectively, and obtain at least one style copy accordingly. The first generation module is configured to use the style fusion network to generate visual data copy of the target style based on the at least one style copy.

14. A video script generation device, characterized in that, include: The second acquisition module is configured to acquire style information of the target style and video description text of the target video, and input the style information and video description text into the visual copywriting generation model, wherein the visual copywriting generation model includes a gated network, a low-rank expert network with multiple preset styles and a style fusion network; The second determining module is configured to use the gating network to determine at least one target low-rank expert network from a plurality of low-rank expert networks based on the style information. The second semantic understanding module is configured to use the at least one target low-rank expert network to perform visual semantic understanding on the video description text respectively, and obtain at least one style video script accordingly. The second generation module is configured to use the style fusion network to generate a video script of the target style based on the at least one style video script.

15. A training device for a visual text generation model, characterized in that, include: The third acquisition module is configured to acquire sample style information of multiple preset styles, sample visual data description text, and corresponding label visual data text of the multiple preset styles, and input the sample style information and sample visual description text into the text generation model, wherein the text generation model includes a gated network, multiple low-rank expert networks and a style fusion network. The third determining module is configured to use the gated network to determine at least one predictive low-rank expert network from multiple low-rank expert networks based on the sample style information. The third semantic understanding module is configured to use the at least one predictive low-rank expert network to perform visual semantic understanding on the sample visual data description text, and obtain at least one predictive style copy accordingly. The third generation module is configured to use the style fusion network to generate the predicted visual data copy based on the at least one predicted style copy. The calculation module is configured to calculate the difference loss based on the predicted visual data text and the label visual data text; The training module is configured to iteratively train the parameters of the gated network and the plurality of low-rank expert networks based on the difference loss until the training stops, thereby obtaining a visual text generation model.

16. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, they implement the steps of the text generation method according to any one of claims 1-10, the video text generation method according to claim 11, or the visual text generation model training method according to claim 12.

17. A computer-readable storage medium, characterized in that, It stores a computer program / instruction that, when executed by a processor, implements the steps of the text generation method according to any one of claims 1-10, the video text generation method according to claim 11, or the visual text generation model training method according to claim 12.

18. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the steps of the text generation method according to any one of claims 1-10, the video text generation method according to claim 11, or the visual text generation model training method according to claim 12.

Images