An artificial intelligence-based image generation method, device, and medium

Abstract

This application discloses an image generation method, device, and medium based on artificial intelligence, applied to digital human-driven product sales scenarios. The method includes: receiving multimodal digital human-driven product sales data; constructing a temporal knowledge graph; mining temporal dependencies and logically modeling them using a temporal graph attention network and XGBoost algorithm to generate a temporal constraint parameter set; performing vector transformation on the data, and generating cross-modal fusion vectors through temperature parameter adaptive adjustment and a dual-branch attention fusion network; constructing four-dimensional constraints for product marketing based on the temporal constraint parameter set and the cross-modal fusion vectors, and optimizing the output constraint parameters through reinforcement learning; injecting the temporal constraint parameter set and the cross-modal fusion vectors into a diffusion model, and collaboratively generating digital human actions and product displays through a spatiotemporal dual-branch modulation network, outputting an initial image; evaluating and iteratively optimizing the initial image to generate the final digital human-driven product sales image. This application achieves temporal coordination and accurate generation of digital human actions and product displays.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence image generation technology, and in particular to an image generation method, device and medium based on artificial intelligence. Background Technology

[0002] In the current e-commerce and live-streaming marketing industry, digital human-driven sales have become a mainstream marketing format. The market has placed stringent demands on the efficiency of generating relevant marketing materials, their adaptability to different scenarios, and their visual consistency. There is an urgent need to quickly generate images that deeply integrate "digital human-product-marketing scenario," while also meeting core requirements such as natural dynamic movements, clear product details, and a style that matches the brand's tone.

[0003] Existing image generation technologies mostly rely on architectures such as diffusion models, generative adversarial networks, and Multi-ControlNet to achieve basic generation capabilities from text to images and from reference images to target images. However, they lack adaptability in digital human e-commerce scenarios, can only handle static entity associations, and cannot capture the temporal logic of "action-product interaction." Moreover, relying on static structural constraints makes it difficult to restore the collaborative relationship between digital human actions and products. On the other hand, existing technologies lack e-commerce marketing-specific feature design, resulting in weak conversion power of generated materials and common problems such as unstable cross-modal alignment accuracy, poor multi-constraint adaptability, and lack of spatiotemporal continuity in the generation of dynamic elements.

[0004] In summary, existing technologies lack a time-series cognitive framework and cross-modal dynamic alignment mechanism specific to digital human-driven sales scenarios, making it impossible to simultaneously meet core requirements such as dynamic action collaboration and product detail fidelity, and thus difficult to adapt to the high-precision, high-conversion material generation requirements of e-commerce marketing. Summary of the Invention

[0005] To address the technical problems existing in the background art, embodiments of this application provide an image generation method, device, and medium based on artificial intelligence. The method includes: receiving multimodal digital human product sales data; the multimodal digital human product sales data includes marketing text descriptions, product reference images, digital human image images, and scene type parameters; performing element parsing and state association on the multimodal digital human product sales data to construct a temporal knowledge graph; the temporal knowledge graph includes four types of nodes: digital human, product, selling point, and time sequence; performing temporal dependency mining on the temporal knowledge graph using a temporal graph attention network and the XGBoost algorithm, and logically modeling the mined temporal knowledge graph to generate a temporal constraint parameter set; performing vector transformation on the multimodal digital human product sales data, and using a preset temperature parameter adaptive adjustment mechanism and a dual-branch attention fusion network. The transformed vectors are fused to generate a cross-modal fusion vector. Based on the temporal constraint parameter set and the cross-modal fusion vector, a four-dimensional constraint for product marketing is constructed, and the weights of the four-dimensional constraint are optimized through a preset reinforcement learning algorithm to output the generation constraint parameters. The four-dimensional constraint for product marketing includes four dimensions: product detail fidelity, natural digital human movements, unified brand style, and clear marketing information. Based on the generation constraint parameters, the temporal constraint parameter set and the cross-modal fusion vector are injected into the generation process of the diffusion model. Through a preset spatiotemporal dual-branch modulation network, the collaborative generation of digital human movements and product display is realized, and an initial digital human product promotion image is output. The initial digital human product promotion image is evaluated for marketing suitability, and the parameters involved in the initial digital human product promotion image generation process are adjusted according to the evaluation results to iteratively generate the final digital human product promotion image.

[0006] In one example, the multimodal input data is parsed and associated with states to construct a temporal knowledge graph. Specifically, this includes: parsing the marketing text description using an improved BERT model to obtain text elements; the improved BERT model includes 128 neurons and a 768-dimensional hidden layer for identifying sales actions and product associations; and using Mask... The R-CNN instance segmentation model analyzes the product reference image and the digital human image image to extract product detail features and digital human image features, obtaining image features. Using a pre-set confidence threshold, it marks the core selling point areas and key action parts of the image features. The digital human subject, product entity, and selling point phrases identified in the text elements are mapped to digital human nodes, product nodes, and selling point nodes in the knowledge graph, respectively. Product detail features and core selling point areas in the image features are associated with the corresponding product nodes, and digital human image features and key action parts are associated with the corresponding digital human nodes. Based on the temporal relationship of actions in the text elements, temporal nodes are constructed; these temporal nodes correspond to the time step sequence of the digital human's sales-driving actions. Through an industry rule base and a temporal long short-term memory network, association edges are established between digital human nodes, product nodes, selling point nodes, and temporal nodes to generate a temporal knowledge graph including four types of nodes: digital human, product, selling point, and temporal, and their associated relationships.

[0007] In one example, a temporal dependency mining process is performed on the temporal knowledge graph using a temporal graph attention network and the XGBoost algorithm. Logical modeling is then performed on the mined temporal knowledge graph to generate a set of temporal constraint parameters. Specifically, this includes: inputting the temporal knowledge graph into the temporal graph attention network; mining cross-timestep dependencies between digital human nodes, product nodes, and selling point nodes at temporal nodes using the graph attention layer; identifying implicit temporal patterns in the node sequences of the temporal knowledge graph using the XGBoost algorithm; and establishing a mapping table between sales actions and temporal constraint parameters using a temporal logistic regression model based on the cross-timestep dependencies and implicit temporal patterns, thereby transforming dynamic relationships into quantifiable temporal constraints and generating a set of temporal constraint parameters.

[0008] In one example, the vector transformation of the multimodal digital human-driven sales data specifically includes: logically enhancing the marketing text using a three-dimensional Prompt Enhancement strategy to generate a temporally enhanced marketing text vector; extracting surface features from the product reference image and the digital human image using convolutional kernels; extracting mid-level features from the product reference image and the digital human image using max pooling; extracting deep features from the product reference image and the digital human image using fully connected layer mapping; and concatenating the surface features, mid-level features, and deep features to generate a multidimensional image feature vector.

[0009] In one example, a pre-defined temperature parameter adaptive adjustment mechanism and a dual-branch attention fusion network are used to fuse the transformed vectors to generate a cross-modal fusion vector. Specifically, this involves: inputting the temporal enhanced marketing text vector and the multi-dimensional image feature vector into the dual-branch attention fusion network, and using a cross-attention mechanism to achieve feature interaction; the dual-branch attention fusion network includes a text branch and an image branch, both of which extract features using a Transformer encoder; the cross-attention mechanism pre-defined element priority weights, setting the weights of product detail features and digital human action features higher than the scene feature weights; and according to the pre-defined temperature parameter adaptive adjustment mechanism, the marketing text... Complexity, product reference image clarity, and brand style requirements are input into an MLP network, which outputs a temperature parameter. The marketing text complexity is evaluated by counting the number of product parameters, action description words, and selling point phrases in the marketing text description, normalizing these three statistical values, and then weighting and summing them to obtain a complexity score. The product reference image clarity is evaluated by calculating the peak signal-to-noise ratio of the product reference image and mapping it to a clarity score. The brand style requirements are quantified based on the number of style categories and the level of detail in the style descriptions preset in the scene type parameters. Based on the temperature parameter and the element priority weights, the output features of the text branch and the image branch are aligned and fused through contrastive learning to generate a cross-modal fusion vector.

[0010] In one example, based on the temporal constraint parameter set and the cross-modal fusion vector, a four-dimensional constraint for product marketing is constructed, and the weights of the four-dimensional constraint are optimized through a preset reinforcement learning algorithm to output the generated constraint parameters. Specifically, this includes: extracting digital human action constraints, product display constraints, selling point association constraints, and scene adaptation constraints from the temporal constraint parameter set; extracting product detail features, digital human action features, brand style features, and marketing information features from the cross-modal fusion vector; associating the digital human action constraints with the digital human action features to construct a natural constraint for digital human action; and associating the product display constraints with the product detail features to construct a product detail constraint. True constraints; associating the scenario adaptation constraints with the brand style features to construct a unified brand style constraint; associating the selling point association constraints with the marketing information features to construct a clear marketing information constraint; combining the digital human action naturalness constraint, product detail fidelity constraint, unified brand style constraint, and clear marketing information constraint into a four-dimensional product marketing constraint; using the four-dimensional product marketing constraint as the state space of reinforcement learning, and the weight allocation scheme of each constraint dimension as the action space, iteratively optimizing the weight allocation of each constraint dimension through the PPO algorithm until the preset convergence condition is met, and outputting the generated constraint parameters; the reward function of the PPO algorithm is set as... ; A represents product fidelity, B represents naturalness of movement, C represents style consistency, and D represents marketing clarity. These correspond to quantitative indicators of the constraints on product detail fidelity, digital human movement naturalness, brand style consistency, and marketing information clarity, respectively. E represents the product selling point prominence coefficient, which indicates the detail texture and color contrast of the core selling point area. F represents the element conflict degree, which indicates the degree of element overlap in each image.

[0011] In one example, based on the generation constraint parameters, the temporal constraint parameter set and the cross-modal fusion vector are injected into the generation process of the diffusion model. Through a preset spatiotemporal dual-branch modulation network, the collaborative generation of digital human actions and product display is achieved, outputting an initial digital human product-selling image. Specifically, this includes: inserting a digital human product-selling feature temporal injection module into the middle layer of the diffusion model to inject the temporal constraint parameter set and the cross-modal fusion vector into the generation process of the diffusion model through feature concatenation and gating fusion; the diffusion model is a denoising-based generative diffusion model; the injected temporal constraint parameter set and cross-modal fusion vector are input into a preset spatiotemporal dual-branch modulation network; the spatiotemporal dual-branch modulation network includes time modulation... The system consists of a temporal modulation branch and a spatial modulation branch. The temporal modulation branch captures the cross-time step correlation of the digital human's sales actions through a temporal convolutional network, generating temporal action features. The spatial modulation branch ensures the positional coordination and fit between the digital human and the product in the image space through a spatial convolutional network, generating spatial coordination features. The temporal action features and the spatial coordination features are fused and input into a product selling point enhancement generator, which outputs enhanced features. The product selling point enhancement generator enhances the detail texture and color contrast of the core selling point area of ​​the product based on the product selling point prominence coefficient in the generation constraint parameters. The enhanced features are input into the reverse denoising process of the diffusion model, and the initial digital human sales image is generated through iterative denoising.

[0012] In one example, a marketing suitability assessment is performed on the initial digital human product promotion image, and the parameters involved in the initial digital human product promotion image generation process are adjusted based on the assessment results to iteratively generate the final digital human product promotion image. Specifically, this includes: constructing a multi-dimensional assessment index system to evaluate the initial digital human product promotion image and obtain assessment results; the multi-dimensional assessment index system includes four categories of assessment items: product detail fidelity detection, digital human motion naturalness detection, brand style matching detection, and marketing information clarity detection, with each item having a preset weight; based on the assessment results, the parameters involved in the digital human product promotion image generation process are adjusted using a gradient descent algorithm, and the digital human product promotion image generation process is iteratively optimized again based on the adjusted parameters until a final digital human product promotion image meeting the preset marketing material usage requirements is generated; the parameters include the dependency weights of the temporal graph attention network, temperature parameters, the weight allocation strategy of the four-dimensional constraint, and the product selling point highlighting coefficient.

[0013] On the other hand, embodiments of this application provide an artificial intelligence-based image generation device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-mentioned artificial intelligence-based image generation methods.

[0014] On the other hand, embodiments of this application provide an artificial intelligence-based image generation non-volatile computer storage medium storing computer-executable instructions that can execute any of the above-mentioned artificial intelligence-based image generation methods.

[0015] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: This solution constructs a temporal knowledge graph comprising four types of nodes: "digital human, product, selling point, and time sequence." Combining a temporal graph attention network with the XGBoost algorithm to uncover explicit and implicit temporal dependencies, it increases the dynamic synergy coefficient between digital human actions and product display to 0.94, achieving 98% product detail fidelity. This effectively solves problems such as stiff movements and misaligned hands and objects found in existing technologies. By introducing features like marketing text complexity and image clarity to dynamically calculate temperature parameters, and combining a dual-branch attention fusion network with a product feature weight enhancement mechanism, the accuracy of product parameter restoration is improved from 65% to 92%, and brand style matching from 60 to 88 points. A four-dimensional constraint system is constructed, and the PPO algorithm is used to dynamically optimize weights, increasing the marketing material qualification rate from 45% to 82%, significantly enhancing conversion potential. A spatiotemporal dual-branch modulation network and a product selling point enhancement generator are designed, improving the naturalness of actions by 35%. Finally, through a closed-loop feedback optimization mechanism, operational satisfaction reaches 97%, and the generation time for a single piece of material is less than 10 seconds, significantly reducing the cost of marketing material production. Attached Figure Description

[0016] To more clearly illustrate the technical solution of this application, some embodiments of this application will be described in detail below with reference to the accompanying drawings, in which: Figure 1 A flowchart illustrating an artificial intelligence-based image generation method provided in this application embodiment; Figure 2 A schematic diagram of module connections for an artificial intelligence-based image generation method provided in an embodiment of this application; Figure 3 A schematic diagram of the internal structure of a dynamic scene cognition module in an artificial intelligence-based image generation method provided in this application embodiment; Figure 4This application provides a schematic diagram of a cross-modal alignment module fusion network structure for an artificial intelligence-based image generation method, as illustrated in an embodiment of the present application. Figure 5 A schematic diagram of a dual-branch coupling architecture of a dynamic feature modulation unit in an artificial intelligence-based image generation method provided in this application embodiment; Figure 6 This is a schematic diagram of the structure of an artificial intelligence-based image generation device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Some embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0019] Figure 1 This is a flowchart illustrating an artificial intelligence-based image generation method provided in an embodiment of this application. This method can be applied to different business domains. Certain input parameters or intermediate results in this process can be manually adjusted to help improve accuracy.

[0020] The analysis method involved in the embodiments of this application can be implemented by a terminal device or a server, and this application does not impose any special limitations on it. For ease of understanding and description, the following embodiments are all described in detail using a server as an example.

[0021] Based on this Figure 1 The process may include the following steps: S101: Receive multimodal digital human sales data; the multimodal digital human sales data includes marketing text descriptions, product reference images, digital human image images, and scene type parameters.

[0022] In some embodiments of this application, the system first receives multimodal digital human sales data input or uploaded by operators through the front-end interactive interface or API interface of the e-commerce marketing system. The multimodal digital human sales data includes marketing text descriptions, product reference images, digital human avatar images, and scene type parameters.

[0023] Marketing text is described as natural language text, including product parameters (such as brand, specifications, color, and function), digital human requirements (such as clothing style, expression, and action description), marketing selling points (such as "long-lasting non-stick," "automatic obstacle avoidance," and "120-minute battery life"), and scene atmosphere requirements (such as "traditional Chinese style dressing table" and "modern minimalist living room"). The text length is usually between 50 and 200 characters. Product reference images are high-definition images that clearly show the overall appearance of the product, the position of the logo, and the core selling point areas (such as lipstick lipstick and home appliance function buttons). The image resolution is no less than 1080P. Digital human image is a front or side view of the digital human, including visual information such as the digital human's clothing, posture, and facial features. Scene type parameters are discrete tags used to identify the purpose of the generated image, including but not limited to "e-commerce detail page image," "live broadcast preview poster," and "short video cover." Each scene type corresponds to a preset information density, layout preference, and style tendency.

[0024] S102: Perform element parsing and state association on the multimodal digital human sales data to construct a time-series knowledge graph; the time-series knowledge graph includes four types of nodes: digital human, product, selling point, and time sequence.

[0025] In some embodiments of this application, element parsing and state association are performed on the received multimodal data to construct a knowledge graph rich in temporal logic.

[0026] First, for marketing text descriptions, an improved BERT model is used for element parsing. This model adds a new branch with 128 neurons and a hidden layer dimension of 768 to the original architecture, specifically for capturing the association logic of "digital human action - product operation - selling point transmission" in the text, such as "holding lipstick → application demonstration → pointing to the selling point of long-lasting non-stickiness". The training corpus of the improved BERT model contains more than 2 million digital human sales scenario texts, covering more than 30 e-commerce product categories such as beauty, clothing, home appliances, and food. Among them, more than 800,000 texts are labeled using the "action type + product category + selling point tag" ternary annotation method. The ratio of training set, validation set and test set is 8:1:1. At the same time, a product keyword weight enhancement mechanism is introduced to increase the attention weight of product name, parameters and selling point words by 4 times. The optimizer is AdamW, with a learning rate of 5e-5, weight decay of 1e-6, 5000 iterations and a convergence threshold of 0.0001.

[0027] Secondly, the Mask R-CNN instance segmentation model is used to analyze the product reference image and the digital human image to extract product details (logo position, size ratio, core selling point area) and digital human image features (clothing style, facial features, and basic posture). The "key areas for sales promotion", such as the product selling point area and key parts of the digital human's hands, are additionally labeled with a confidence threshold of 0.8 to ensure the accuracy of core element extraction.

[0028] Furthermore, the digital human subject, product entity, and selling point phrases extracted from the text are mapped to digital human nodes, product nodes, and selling point nodes in the knowledge graph, respectively; the product detail features and core selling point areas in the image features are associated with the corresponding product nodes, and the digital human image features and key parts of the actions are associated with the corresponding digital human nodes.

[0029] Simultaneously, temporal nodes are constructed based on the temporal relationships of actions in text elements (such as the time step sequence t1→t2→t3). Each temporal node corresponds to a time step, used to characterize the evolution of dynamic actions. Finally, based on the industry rule base and temporal long short-term memory network, association rules between four types of nodes are automatically generated: the rule base covers the temporal logic of 1200+ digital human live-streaming sales scenarios, such as "beauty live-streaming → holding lipstick (t1) → applying lipstick (t2) → displaying the texture of the lipstick (t3) → pointing to the 'long-lasting and non-sticky' selling point (t4)". Implicit associations are automatically mined and supplemented through a temporal LSTM network, ultimately forming a temporal knowledge graph containing four types of nodes and their associated edges, including digital human, product, selling point, and temporal nodes.

[0030] S103: Using a temporal graph attention network and the XGBoost algorithm, temporal dependency mining is performed on the temporal knowledge graph, and logical modeling is performed on the mined temporal knowledge graph to generate a set of temporal constraint parameters.

[0031] In some embodiments of this application, based on the constructed temporal knowledge graph, explicit and implicit temporal dependencies across time steps are further mined, and they are transformed into quantifiable generative constraints through logical modeling.

[0032] First, the temporal knowledge graph is input into the temporal graph attention network, which uses 4 attention heads, 256 hidden layer dimensions, a learning rate of 3e-4, and 3000 iterations. The graph attention layer aggregates neighbor node information and updates the representation of each node, thereby mining the cross-time step dependencies of digital human nodes, product nodes, and selling point nodes at different temporal nodes. For example, "the digital human's hand holding angle is 45° at t1 and transitions to 60° at t2" and "the product maintains a rotation trajectory from t1 to t2".

[0033] Secondly, the updated node representations are input into the XGBoost algorithm model, which has a tree depth of 6, 1000 iterations, and a learning rate of 0.01. Through feature importance ranking and decision tree ensemble learning, implicit temporal patterns are identified from the node sequence, such as implicit associations like "displaying the phone → holding it at a 30° angle (t1) → rotating the phone body (t2) → turning on the screen to display the function (t3)". This improves the rule update efficiency by 60%.

[0034] Furthermore, the discovered dependencies and implicit temporal patterns are input into a temporal logistic regression model with a regularization coefficient of 0.01. Based on the input data, a mapping table of "sales-driving actions - temporal constraint parameters" is established, transforming dynamic relationships into quantifiable generation constraints and generating a set of temporal constraint parameters. This parameter set contains more than 30 sub-parameters in four categories: digital human action constraints (such as limb angles and action trajectories), product display constraints (detail clarity, proportion accuracy, and logo visibility ≥99%), selling point association constraints (selling point identification positions and digital human pointing actions), and scene adaptation constraints (layout, color, and information density). A new exclusive parameter, "product selling point prominence coefficient," is also added to provide precise temporal guidance for the subsequent generation process.

[0035] S104: Perform vector transformation on the multimodal digital human product sales data to generate temporally enhanced marketing text vectors and multidimensional image feature vectors. Then, through a preset temperature parameter adaptive adjustment mechanism and a dual-branch attention fusion network, fuse the transformed temporally enhanced marketing text vectors and multidimensional image feature vectors to generate cross-modal fusion vectors.

[0036] In some embodiments of this application, the multimodal data is first vectorized. For the marketing text description, a three-dimensional Prompt Enhancement strategy is used to enhance the logic: the original text is expanded into a more detailed description from three dimensions: "core needs", "action logic" and "selling point sequence". For example, "Chinese style beauty live streaming: digital human wearing Hanfu shows lipstick, highlighting long-lasting and non-sticky" is expanded to "Chinese style beauty live streaming → digital human wearing Hanfu holding lipstick (t1) → applying lipstick (t2) → showing the lipstick (t3) → pointing to the selling point of 'long-lasting and non-sticky' (t4) → background is an ancient style dressing table with a reddish-brown tone". Then, a 768-dimensional time-series enhanced marketing text vector is generated through the BERT model.

[0037] For product reference images and digital human avatar images, a multi-level feature extraction network is adopted: surface features are extracted through 3×3 convolutional kernels, including 128-dimensional product color features and 64-dimensional digital human clothing features; mid-level features are extracted through max pooling, including 96-dimensional scene layout features and 80-dimensional product structure features; deep features are extracted through fully connected layer mapping, including 128-dimensional brand style features and 64-dimensional sales-driving action characteristic features; finally, the surface, mid-level, and deep features are concatenated to generate a 544-dimensional multi-dimensional image feature vector specific to the marketing scenario.

[0038] Next, cross-modal fusion is performed through a dual-branch attention fusion network: both the text branch and the image branch use a 6-layer Transformer encoder, and feature interaction is achieved through a cross-attention mechanism; the core element priority weight is introduced in the cross-attention, and the weights of product detail features and digital human action features are set to 1.8 times the weights of scene features. At the same time, a marketing scene dynamic temperature parameter adaptive algorithm is introduced: the marketing text complexity (counting the number of product parameters, the number of action description words, and the number of selling point phrases, and then normalizing and weighting the sum), the clarity of the product reference image (calculating the PSNR value and mapping it to the clarity score), and the brand style requirements (quantified according to the number of style categories and the level of detail in the scene type parameter) are input into the MLP network, and the contrast learning temperature parameter τ is dynamically calculated. The value range is verified to be 0.05 to 0.12 (when τ < 0.05, the semantic matching degree drops to 78%; when τ > 0.12, the accuracy of product detail restoration drops to 82%; only when τ ∈ [0.05-0.12] is the semantic matching degree ≥ 88% and the product restoration accuracy ≥ 92%).

[0039] Based on the temperature parameter and priority weight, the output features of the text branch and the image branch are aligned and fused through contrastive learning to generate a 1024-dimensional cross-modal fusion vector. This vector contains a deep fusion representation of text semantics, product details, digital human actions, brand style and marketing information.

[0040] S105: Based on the time-series constraint parameter set and the cross-modal fusion vector, construct a four-dimensional constraint for product marketing, and optimize the weights of the four-dimensional constraint through a preset reinforcement learning algorithm to output the generated constraint parameters; the four-dimensional constraint for product marketing includes four dimensions: product detail fidelity, natural digital human movements, unified brand style, and clear marketing information.

[0041] In some embodiments of this application, firstly, digital human action constraints, product display constraints, selling point association constraints, and scene adaptation constraints are extracted from the temporal constraint parameter set, and product detail features, digital human action features, brand style features, and marketing information features are extracted from the cross-modal fusion vector. By associating the above constraints with features, a quantifiable four-dimensional constraint system is constructed: The digital human action constraint is associated with digital human action features to construct a natural digital human action constraint, the quantification index of which is the dynamic synergy coefficient, calculated as "action trajectory continuity × 0.6 + action-selling point correlation × 0.4", with a threshold set experimentally verified to be ≥0.92; the product display constraint is associated with product detail features to construct a product detail fidelity constraint, the quantification index of which is product feature similarity (calculated by cosine similarity between generated images and reference images, threshold ≥0.9); the scene adaptation constraint is associated with brand style features to construct a brand style uniformity constraint, the quantification index of which is style feature distance (distance ≤0.18); and the selling point correlation constraint is associated with marketing information features to construct a marketing information clarity constraint, the quantification index of which is marketing information visibility (evaluated by image information recognition accuracy, threshold ≥0.85). These four constraints are combined to form the four-dimensional constraints for product marketing.

[0042] Then, the PPO algorithm, which is dedicated to digital human-driven product sales, is used to optimize the weights of each constraint dimension: the four-dimensional constraints are used as the state space, the weight allocation scheme of each dimension is used as the action space, and the reward function is designed as "product fidelity × 0.35 + action naturalness × 0.3 + style consistency × 0.2 + marketing clarity × 0.15 + product selling point prominence coefficient × 0.08 - element conflict degree × 0.1", where the product selling point prominence coefficient ranges from [0.8, 1.2], and the element conflict degree measures the degree of overlap of elements in the image.

[0043] In addition, the initial weights are dynamically adjusted based on the scenario type parameters: the product detail page images increase the weight for product authenticity to 0.4, the posters increase the weight for style consistency to 0.3, and the short video covers increase the weight for marketing clarity to 0.25. Through iterative optimization using the PPO algorithm, until the preset convergence conditions are met, the optimized generation constraint parameters are output. This parameter set contains the final weights and specific numerical requirements for each constraint dimension, providing precise control instructions for subsequent generation.

[0044] S106: Based on the generation constraint parameters, the temporal constraint parameter set and cross-modal fusion vector are injected into the generation process of the diffusion model. Through a preset spatiotemporal dual-branch modulation network, the collaborative generation of digital human actions and product display is realized, and the initial digital human product image is output.

[0045] In some embodiments of this application, image generation is based on the Flux.1 diffusion model, and a digital human product-selling feature temporal injection module is inserted into its 3rd, 5th, and 7th intermediate layers. The temporal constraint parameter set and cross-modal fusion vector are injected into the generation process through feature concatenation and gating fusion. The injection intensity is adaptively adjusted according to the temporal constraint parameters, and the dynamic feature decay rate is set to 0.8 to 0.95, focusing on preserving product details and digital human action features. The activation function adopts the Swish+ temporal activation term to enhance the continuity of actions.

[0046] The injected features are input into a pre-defined spatiotemporal dual-branch modulation network: the temporal modulation branch uses 1D convolution to capture the cross-time step correlation of the digital human's sales actions, such as gesture movement trajectory and changes in product status; the spatial modulation branch uses 2D convolution to ensure the synergy between the digital human and the product in image space, such as the fit between the hand and the product and their spatial positional relationship. The generated action temporal features and spatial synergy features are fused and input into a product selling point enhancement generator. This generator enhances the detail texture and color contrast of the core selling point area of ​​the product based on the product selling point prominence coefficient in the generation constraint parameters. At the same time, the marketing information adaptation module adjusts the information density according to the scene type parameters and outputs dynamic features that meet marketing needs.

[0047] Next, the enhanced features are input into the inverse denoising process of the diffusion model, and the initial digital human product promotion image is generated through iterative denoising. After generation, the image resolution is increased to 4K through an improved ESRGAN super-resolution enhancement module, and the details such as the product logo and core components are optimized by combining the U-Net detail restoration network to ensure accurate proportions and no distortion. Finally, the image tone is adjusted based on the brand reference color through the brand tone calibration module to ensure a consistent style and output the optimized initial digital human product promotion image.

[0048] S107: Perform a marketing suitability assessment on the initial digital human product promotion image, and adjust the parameters involved in the initial digital human product promotion image generation process according to the assessment results, so as to iteratively generate the final digital human product promotion image.

[0049] In some embodiments of this application, a multi-dimensional evaluation index system specifically for digital human-driven product sales is constructed to automatically evaluate the initial image. This system includes four main categories of evaluation items: product detail fidelity detection (logo integrity, proportional accuracy, clarity of selling point areas, weight 0.35), digital human motion naturalness detection (dynamic coordination coefficient, motion rationality, weight 0.3), brand style matching detection (tone, layout, texture, weight 0.2), and marketing information clarity detection (visibility of selling point markings, information density suitability, weight 0.15). The weighted average of each evaluation result yields a comprehensive score.

[0050] Based on the evaluation results, the parameters involved in the generation process are dynamically adjusted using the gradient descent algorithm: the dependency weights of the temporal GAT network are adjusted (adjustment step size 0.03), the dynamic temperature parameter τ of the cross-modal alignment module is adjusted (adjustment step size 0.005), the weight allocation strategy of the multi-constraint balancing module is adjusted (adjustment range ±0.05), and the product selling point enhancement coefficient of the refined generation module is adjusted (adjustment step size 0.02). Simultaneously, it supports operators to input manual feedback requirements through a visual marking tool, such as "darker lipstick color" or "digital human gesture closer to selling point markers," converting feedback into specific constraint parameters.

[0051] By integrating automated evaluation and adjustments with manual feedback, the generation process from S102 to S106 is iteratively optimized and repeated until a final digital human-based product image that meets the needs of marketing materials is generated. Experimental results demonstrate that this closed-loop feedback mechanism achieves a 97% operational satisfaction rate, is compatible with over 30 e-commerce categories including beauty, apparel, and home appliances, and generates a single image in ≤10 seconds, significantly reducing the cost of marketing material production.

[0052] It should be noted that, although the embodiments in this application are based on... Figure 1 Steps S101 to S107 will be described sequentially, but this does not mean that steps S101 and S107 must be performed in a strict order. The reason this embodiment follows this order is... Figure 1 The order in which steps S101 to S107 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, in the embodiments of this application, the order of steps S101 to S107 can be appropriately adjusted according to actual needs.

[0053] pass Figure 1This solution constructs a temporal knowledge graph comprising four types of nodes: "digital human, product, selling point, and time sequence." Combining a temporal graph attention network and the XGBoost algorithm to mine explicit and implicit temporal dependencies, it improves the dynamic synergy coefficient between digital human actions and product display to 0.94, achieving 98% product detail fidelity, effectively solving problems such as stiff movements and misaligned objects in existing technologies. By introducing features such as marketing text complexity and image clarity to dynamically calculate temperature parameters, and combining a dual-branch attention fusion network and a product feature weight enhancement mechanism, the accuracy of product parameter restoration is improved from 65% to 92%, and the brand style matching score is improved from 60 to 88. A four-dimensional constraint system is constructed, and the PPO algorithm is used to dynamically optimize weights, increasing the marketing material qualification rate from 45% to 82%, significantly enhancing conversion potential. A spatiotemporal dual-branch modulation network and a product selling point enhancement generator are designed, improving the naturalness of actions by 35%. Finally, through a closed-loop feedback optimization mechanism, operational satisfaction reaches 97%, and the generation time for a single material is less than 10 seconds, significantly reducing the cost of marketing material production.

[0054] Figure 2 This diagram illustrates the module connections of an artificial intelligence-based image generation method provided in an embodiment of this application. Figure 2 The document displays the connection relationships of the five core modules, the data flow direction, and the transmission path of time sequence parameters, and marks the exclusive data interface for e-commerce marketing.

[0055] Figure 3 This is a schematic diagram of the internal structure of a dynamic scene cognition module in an AI-based image generation method provided in this application embodiment. It shows the input, output, and execution flow of the annotation element parsing unit, the state association unit (digital human product-selling time-series knowledge graph + time-series GAT network), and the logic modeling unit (time-series logistic regression model).

[0056] Figure 4 This is a schematic diagram of the cross-modal alignment module fusion network structure of an AI-based image generation method provided in this application embodiment. It illustrates the network hierarchy and feature flow for extracting implicit logic from annotated text (three-dimensional Prompt Enhancement), extracting deep features from the reference image (multi-level feature network), and fusion of two-branch attention (Transformer encoder + dynamic temperature parameter adjustment).

[0057] Figure 5 This is a schematic diagram of the dual-branch coupling architecture of a dynamic feature modulation unit in an artificial intelligence-based image generation method provided in this application embodiment. The connection relationships and processing flow of the temporal modulation branch (1D convolution), spatial modulation branch (2D convolution), and product selling point enhancement generator are annotated.

[0058] Figure 6A schematic diagram of an artificial intelligence-based image generation device provided in this application embodiment includes: At least one processor; and, A memory that is communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, which, when executed by at least one processor, enable the at least one processor to perform any of the above-mentioned artificial intelligence-based image generation methods.

[0059] Some embodiments of this application provide an artificial intelligence-based image generation non-volatile computer storage medium storing computer-executable instructions that can execute any of the above-mentioned artificial intelligence-based image generation methods.

[0060] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0061] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0062] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0064] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0065] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0066] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0067] Memory may include non-persistent storage in computer-readable media, random access memory (RAM), and non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0068] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0069] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0070] The above are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the technical principles of this application should fall within the protection scope of this application.

Claims

1. An image generation method based on artificial intelligence, characterized in that, The method includes: Receive multimodal digital human sales data; the multimodal digital human sales data includes marketing text descriptions, product reference images, digital human image images, and scene type parameters; The multimodal digital human product sales data is parsed and associated with states to construct a time-series knowledge graph; the time-series knowledge graph includes four types of nodes: digital human, product, selling point, and time sequence. Temporal dependency mining is performed on the temporal knowledge graph using a temporal graph attention network and the XGBoost algorithm. Logical modeling is then performed on the mined temporal knowledge graph to generate a set of temporal constraint parameters. The multimodal digital human product sales data is vectorized to generate temporally enhanced marketing text vectors and multidimensional image feature vectors. The transformed temporally enhanced marketing text vectors and multidimensional image feature vectors are then fused through a preset temperature parameter adaptive adjustment mechanism and a dual-branch attention fusion network to generate a cross-modal fusion vector. Based on the temporal constraint parameter set and the cross-modal fusion vector, a four-dimensional constraint for product marketing is constructed, and the weights of the four-dimensional constraint are optimized through a preset reinforcement learning algorithm to output the generated constraint parameters; the four-dimensional constraint for product marketing includes four dimensions: product detail fidelity, natural digital human movements, unified brand style, and clear marketing information. Based on the generation constraint parameters, the temporal constraint parameter set and cross-modal fusion vector are injected into the generation process of the diffusion model. Through a preset spatiotemporal dual-branch modulation network, the collaborative generation of digital human actions and product display is realized, and the initial digital human product promotion image is output. The initial digital human product promotion image is evaluated for marketing suitability, and the parameters involved in the initial digital human product promotion image generation process are adjusted based on the evaluation results to iteratively generate the final digital human product promotion image.

2. The method according to claim 1, characterized in that, The step of parsing elements and associating states in the multimodal digital human sales data to construct a temporal knowledge graph specifically includes: The marketing text description is parsed using an improved BERT model to obtain text elements; the improved BERT model includes 128 neurons and a product association recognition branch with a hidden layer dimension of 768. The product reference image and the digital human image image are analyzed by the Mask R-CNN instance segmentation model to extract product detail features and digital human image features, obtain image features, and mark the core selling point area and key action parts of the image features by using a pre-set confidence threshold. The digital human subject, product entity, and selling point phrase identified in the text elements are respectively mapped to digital human nodes, product nodes, and selling point nodes in the knowledge graph; The product detail features and core selling point areas in the image features are associated with the corresponding product nodes, and the digital human image features and key movement parts are associated with the corresponding digital human nodes. Based on the temporal relationship of actions in text elements, time sequence nodes are constructed; the time sequence nodes correspond to the time step sequence of the digital human's sales actions. By using an industry rule base and a temporal long short-term memory network, we establish connection edges between digital human nodes, product nodes, selling point nodes, and temporal nodes to generate a temporal knowledge graph that includes four types of nodes: digital human, product, selling point, and temporal, and their relationships.

3. The method according to claim 1, characterized in that, The process involves using a temporal graph attention network and the XGBoost algorithm to mine temporal dependencies in the temporal knowledge graph, and then logically modeling the mined temporal knowledge graph to generate a set of temporal constraint parameters. Specifically, this includes: The temporal knowledge graph is input into the temporal graph attention network, and the cross-time step dependencies of digital human nodes, product nodes, and selling point nodes on the temporal nodes are mined through the graph attention layer. The XGBoost algorithm is used to identify hidden temporal patterns in the node sequences of a temporal knowledge graph. Based on the cross-time step dependencies and implicit temporal patterns, a mapping table between sales actions and temporal constraint parameters is established through a temporal logistic regression model to transform dynamic relationships into quantifiable temporal constraints and generate a set of temporal constraint parameters.

4. The method according to claim 1, characterized in that, The step of vectorizing the multimodal digital human sales data to generate time-enhanced marketing text vectors and multidimensional image feature vectors specifically includes: The marketing text is logically enhanced using a three-dimensional Prompt Enhancement strategy, generating a time-series enhanced marketing text vector. Surface features of product reference images and digital human character images are extracted using convolutional kernels; Mid-level features of product reference images and digital human avatar images are extracted using max pooling. Deep features of the product reference image and digital human image image are extracted through fully connected layer mapping; The surface features, mid-layer features, and deep features are concatenated to generate a multi-dimensional image feature vector.

5. The method according to claim 1, characterized in that, By employing a pre-defined temperature parameter adaptive adjustment mechanism and a dual-branch attention fusion network, the converted temporal enhanced marketing text vector and multi-dimensional image feature vector are fused to generate a cross-modal fusion vector, specifically including: The temporal enhanced marketing text vector and the multidimensional image feature vector are respectively input into a dual-branch attention fusion network, and feature interaction is achieved through a cross-attention mechanism. The dual-branch attention fusion network includes a text branch and an image branch, and both branches extract features through a Transformer encoder. The cross-attention mechanism presets element priority weights, setting the weights of product detail features and digital human action features to be higher than the weights of scene features. Based on a preset temperature parameter adaptive adjustment mechanism, the marketing text complexity, product reference image clarity, and brand style requirements are input into the MLP network, and the temperature parameter is output. The evaluation rule for the marketing text complexity is to count the number of product parameters, the number of action description words, and the number of selling point phrases in the marketing text description, and then normalize the three statistical values ​​and sum them by weight to obtain the complexity score. The evaluation rule for the product reference image clarity is to calculate the peak signal-to-noise ratio of the product reference image and map it to a clarity score. The brand style requirements are quantified based on the number of preset style categories and the level of detail in the style description in the scene type parameter. Based on the temperature parameter and the element priority weight, the output features of the text branch and the image branch are aligned and fused through contrastive learning to generate a cross-modal fusion vector.

6. The method according to claim 1, characterized in that, The step of constructing a four-dimensional constraint for product marketing based on the temporal constraint parameter set and the cross-modal fusion vector, and optimizing the four-dimensional constraint weights through a preset reinforcement learning algorithm to output generated constraint parameters, specifically includes: Extract the digital human action constraints, product display constraints, selling point association constraints, and scene adaptation constraints from the time-series constraint parameter set; Extract product detail features, digital human action features, brand style features, and marketing information features from the cross-modal fusion vector; By associating the digital human motion constraints with the digital human motion features, a natural constraint for digital human motion is constructed. By associating the product display constraints with the product detail features, a product detail fidelity constraint is constructed. By associating the scenario adaptation constraints with the brand style characteristics, a unified brand style constraint is constructed. By associating the selling point association constraints with the marketing information features, a clear marketing information constraint is constructed. The constraints of natural digital human movement, authenticity of product details, consistency of brand style, and clarity of marketing information are combined into four-dimensional constraints for product marketing. The four-dimensional constraints of product marketing are used as the state space of reinforcement learning, and the weight allocation scheme of each constraint dimension is used as the action space. The weight allocation of each constraint dimension is iteratively optimized through the PPO algorithm until the preset convergence condition is met, and the generated constraint parameters are output. The reward function of the PPO algorithm is set as follows: ； A represents product fidelity, B represents naturalness of movement, C represents style consistency, and D represents marketing clarity. These correspond to quantitative indicators of the constraints on product detail fidelity, digital human movement naturalness, brand style consistency, and marketing information clarity, respectively. E represents the product selling point prominence coefficient, which indicates the detail texture and color contrast of the core selling point area. F represents the element conflict degree, which indicates the degree of element overlap in each image.

7. The method according to claim 1, characterized in that, The generation process of injecting the temporal constraint parameter set and cross-modal fusion vector into the diffusion model according to the generation constraint parameters, and realizing the collaborative generation of digital human actions and product display through a preset spatiotemporal dual-branch modulation network, outputs an initial digital human product promotion image, specifically including: A digital human product-selling feature temporal injection module is inserted into the middle layer of the diffusion model to inject the temporal constraint parameter set and the cross-modal fusion vector into the generation process of the diffusion model through feature concatenation and gating fusion; the diffusion model is a generative diffusion model based on denoising. The injected temporal constraint parameter set and cross-modal fusion vector are input into a preset spatiotemporal dual-branch modulation network; the spatiotemporal dual-branch modulation network includes a temporal modulation branch and a spatial modulation branch; the temporal modulation branch captures the cross-time step correlation of the digital human's sales actions through a temporal convolutional network to generate action temporal features; the spatial modulation branch ensures the positional coordination and fit between the digital human and the goods in the image space through a spatial convolutional network to generate spatial coordination features; The action timing features and spatial coordination features are fused together and input into the product selling point enhancement generator, which outputs enhanced features. The product selling point enhancement generator enhances the details, textures and color contrast of the core selling point area of ​​the product based on the product selling point prominence coefficient in the generation constraint parameters. The enhanced features are input into the inverse denoising process of the diffusion model, and the initial digital human live-streaming sales image is generated through iterative denoising.

8. The method according to claim 1, characterized in that, The step of conducting a marketing suitability assessment on the initial digital human product promotion image and adjusting the parameters involved in the initial digital human product promotion image generation process based on the assessment results to iteratively generate the final digital human product promotion image specifically includes: A multi-dimensional evaluation index system is constructed to evaluate the initial digital human product promotion image and obtain the evaluation results. The multi-dimensional evaluation index system includes four major categories of evaluation items: product detail fidelity detection, digital human movement naturalness detection, brand style matching detection, and marketing information clarity detection. The evaluation weights of each item are preset weights. Based on the evaluation results, the parameters involved in the digital human product promotion image generation process are adjusted using the gradient descent algorithm. Then, based on the adjusted parameters, the digital human product promotion image generation process is re-executed for iterative optimization until the final digital human product promotion image that meets the preset marketing material requirements is generated. The parameters include the dependency weights of the temporal graph attention network, temperature parameters, the weight allocation strategy of the four-dimensional constraint, and the product selling point highlighting coefficient.

9. An image generation device based on artificial intelligence, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform an artificial intelligence-based image generation method as described in any one of claims 1-8.

10. An image generation storage medium based on artificial intelligence, storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of executing an artificial intelligence-based image generation method as described in any one of claims 1-8.

Images