An artificial intelligence visual asset data generation method, device and equipment

Abstract

This application provides a method, apparatus, and device for generating artificial intelligence visual asset data, relating to the field of artificial intelligence generation technology. A method for generating artificial intelligence visual asset data includes: acquiring multimodal scene creation requirement data to construct a three-dimensional reference scene; configuring a standardized constraint instruction set based on the three-dimensional reference scene; obtaining model generation results in the three-dimensional reference scene according to the standardized constraint instruction set through a preset artificial intelligence generation model; and associating the three-dimensional reference scene, the standardized constraint instruction set, and the model generation results to form reusable artificial intelligence visual asset data. According to embodiments of this application, through standardized scene construction, dynamic generation constraints, and iterative asset management, accurate, consistent, and batch generation of artificial intelligence visual asset data is achieved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence generation technology, and more specifically, to a method, apparatus, and device for generating artificial intelligence visual asset data. Background Technology

[0002] With the development of generative artificial intelligence technology, text-to-image and text-to-video tools have achieved significant breakthroughs in the quality of single content generation. However, in industrialized and serialized professional content production scenarios, existing technologies still have the following structural pain points that are difficult to solve.

[0003] 1. Lack of asset continuity: In the random generation mode based on text prompts, visual elements such as character image, scene style, lighting and perspective cannot be kept consistent in multiple generation processes. It is like "drawing blind boxes". It is impossible to form an accumulated and reusable visual asset library. Each creation must start from scratch, resulting in high repetition costs.

[0004] 2. Insufficient control over narrative and creation: Creators cannot precisely control core elements such as camera position, character movement, and composition like traditional film and television directors. Visual narratives rely heavily on the randomness generated by artificial intelligence, which severely restricts the overall planning of content and the continuous narrative expression.

[0005] 3. Production process cannot be closed: Existing tools are only suitable for inspiration or concept demonstration, and lack the ability to support the entire process from world-building and storyboard design to batch content generation. They cannot be integrated into the existing workflow of professional production, resulting in a disconnect between the AI ​​generation stage and subsequent production stages.

[0006] 4. High costs of creative collaboration and decision-making: Complex visual concepts require repeated descriptions in text, resulting in low communication efficiency. Creators without an art background struggle to accurately express their visual intentions. Furthermore, creators spend a significant amount of time rewriting prompts and testing generation parameters, preventing them from focusing on the creative idea itself and narrative decisions.

[0007] To address the aforementioned issues, some current technical solutions attempt to improve content consistency by optimizing prompts and fixing generation parameters. However, these only achieve localized optimization and cannot fundamentally solve core problems such as spatial relationships, lighting logic, and asset reuse. Other solutions, which simply combine 3D scene tools with AI generation tools, lack standardized scene description protocols and dynamic constraint control capabilities. They cannot achieve precise matching between generated content and preset 3D scenes, and have not yet formed a controllable and reusable industrialized production process. Summary of the Invention

[0008] According to one aspect of this application, a method for generating artificial intelligence visual asset data is provided, comprising: acquiring multimodal scene creation requirement data to construct a three-dimensional benchmark scene; configuring a standardized constraint instruction set based on the three-dimensional benchmark scene; obtaining model generation results in the three-dimensional benchmark scene through a preset artificial intelligence generation model according to the standardized constraint instruction set; and associating the three-dimensional benchmark scene, the standardized constraint instruction set, and the model generation results to form reusable artificial intelligence visual asset data.

[0009] According to some embodiments, acquiring multimodal scene creation requirement data to construct a three-dimensional benchmark scene includes: extracting multimodal features from the scene creation requirement data; fusing the multimodal features through a preset feature fusion network to obtain a fused feature tensor; generating a structured scene dataset based on the fused feature tensor; transforming the structured scene dataset according to preset scene description rules to generate three-dimensional scene description data; and generating a three-dimensional benchmark scene through a platform visualization interface based on the three-dimensional scene description data.

[0010] According to some embodiments, the scene creation requirement data includes natural language descriptions, two-dimensional sketches, and / or three-dimensional models; wherein, extracting multimodal features from the scene creation requirement data includes: generating semantic feature vectors corresponding to the natural language descriptions through a preset visual semantic understanding model; generating visual feature matrices corresponding to the two-dimensional sketches through a preset computer vision algorithm; and / or parsing the three-dimensional models to obtain geometric feature datasets corresponding to the three-dimensional models.

[0011] According to some embodiments, multimodal features are fused through a preset feature fusion network to obtain a fused feature tensor, including: fusing semantic feature vectors, visual feature matrices and / or geometric feature datasets through a feature fusion network according to preset weight ratios to obtain fused multimodal features; and encoding the fused multimodal features to generate a fused feature tensor.

[0012] According to some embodiments, generating a structured scene dataset based on a fusion feature tensor includes: reconstructing the fusion feature tensor from a spatial dimension to generate spatial coordinate base data; reconstructing the fusion feature tensor from an object dimension to generate object topology data; reconstructing the fusion feature tensor from an attribute dimension to generate attribute parameter data; and integrating the spatial coordinate base data, object topology data, and attribute parameter data to obtain a structured scene dataset.

[0013] According to some embodiments, a structured scene dataset is transformed according to preset scene description rules to generate three-dimensional scene description data, including: converting the structured scene dataset to a format according to scene description rules; performing data verification on the converted structured scene dataset; and compressing the verified structured scene dataset to obtain three-dimensional scene description data.

[0014] According to some embodiments, a standardized constraint instruction set is configured based on a three-dimensional reference scene, including: obtaining preset camera position parameters, character element parameters, and composition style parameters; and converting the camera position parameters, character element parameters, and composition style parameters based on the three-dimensional reference scene to generate a standardized constraint instruction set.

[0015] According to some embodiments, based on a standardized constraint instruction set, a model generation result in a 3D benchmark scene is obtained through a preset artificial intelligence generation model, including: performing hierarchical parsing of the standardized constraint instruction set according to a preset priority; obtaining comprehensive input data for the artificial intelligence generation model based on the hierarchically parsed standardized constraint instruction set; obtaining the model generation result through the artificial intelligence generation model based on the comprehensive input data; and performing matching degree verification on the model generation result based on the standardized constraint instruction set.

[0016] According to some embodiments, the standardized constraint instruction set after hierarchical parsing includes spatial constraint instructions, composition constraint instructions, and / or style constraint instructions; wherein, based on the standardized constraint instruction set after hierarchical parsing, the comprehensive input data of the artificial intelligence generation model is obtained, including: transforming the spatial constraint instructions, composition constraint instructions, and / or style constraint instructions to obtain hierarchical constraint parameters that the artificial intelligence generation model can recognize, the hierarchical constraint parameters including spatial attention mask, image composition guidance vector, and / or style feature embedding vector; and concatenating the 3D scene description data corresponding to the 3D reference scene with the hierarchical constraint parameters to obtain comprehensive input data.

[0017] According to some embodiments, based on a standardized constraint instruction set, the matching degree of the model generation result is verified, including: extracting feature parameters of the model generation result through preset image recognition rules; determining whether the matching degree between the feature parameters and the standardized constraint instruction set reaches a preset threshold; if the matching degree reaches the preset threshold, determining that the model generation result passes the matching degree verification; and outputting the model generation result that passes the matching degree verification to the platform visualization interface.

[0018] According to some embodiments, the matching degree verification of the model generation result based on the standardized constraint instruction set further includes: if the matching degree does not reach a preset threshold, obtaining the deviation data between the feature parameters and the standardized constraint instruction set; and optimizing the parameters of the standardized constraint instruction set based on the deviation data until the model generation result passes the matching degree verification.

[0019] According to some embodiments, the three-dimensional benchmark scene, standardized constraint instruction set and model generation result are associated to form reusable artificial intelligence visual asset data, including: associating the three-dimensional benchmark scene, standardized constraint instruction set and model generation result; setting a unique identifier for the associated three-dimensional benchmark scene, standardized constraint instruction set and model generation result to form artificial intelligence visual asset data; and storing the artificial intelligence visual asset data on both ends.

[0020] According to one aspect of this application, an apparatus for generating artificial intelligence visual asset data is provided, comprising: a scene construction module for acquiring multimodal scene creation requirement data to construct a three-dimensional benchmark scene; an instruction configuration module for configuring a standardized constraint instruction set based on the three-dimensional benchmark scene; a content generation module for obtaining model generation results in the three-dimensional benchmark scene according to the standardized constraint instruction set and a preset artificial intelligence generation model; and an asset management module for associating the three-dimensional benchmark scene, the standardized constraint instruction set, and the model generation results to form reusable artificial intelligence visual asset data.

[0021] According to one aspect of this application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to perform the method as described above.

[0022] According to one aspect of this application, a computer-readable storage medium is provided that stores a computer program or instructions thereon, which, when executed by a processor, implement the method described above.

[0023] According to the embodiments of this application, a three-dimensional benchmark scene is used as a unified base, and all generation behaviors are carried out under scene constraints. This changes the traditional "random blind box" mode of artificial intelligence generation, solves the problem of consistency of generated content, puts narrative planning in advance, fully grasps the control of visual narrative, realizes precise control of the entire creation process, reduces the cost of creative decision-making and collaboration, and builds a complete industrialized content production process to realize the sustainable iteration and large-scale reuse of visual assets.

[0024] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application.

[0026] Figure 1 A flowchart illustrating a method for generating artificial intelligence visual asset data according to an example embodiment of this application is shown.

[0027] Figure 2 A schematic diagram of an apparatus for generating artificial intelligence visual asset data according to an example embodiment of this application is shown.

[0028] Figure 3 A schematic diagram of an electronic device according to an example embodiment of this application is shown. Detailed Implementation

[0029] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0030] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0031] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of these specific details, or other methods, components, materials, apparatus, or operations may be employed. In these cases, well-known structures, methods, apparatuses, implementations, materials, or operations will not be shown or described in detail.

[0032] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0033] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0034] This application provides a method, apparatus, and device for generating artificial intelligence visual asset data, which can achieve accurate, consistent, and batch generation of artificial intelligence images / videos.

[0035] The following will describe in detail, with reference to the accompanying drawings, a method, apparatus, and device for generating artificial intelligence visual asset data according to embodiments of this application.

[0036] Figure 1 A flowchart illustrating a method for generating artificial intelligence visual asset data according to an example embodiment of this application is shown.

[0037] like Figure 1 As shown, in step S100, multimodal scene creation requirement data is obtained to construct a three-dimensional reference scene.

[0038] Multimodal scene creation requirements data include any one or more combinations of natural language descriptions, 2D sketches, and 3D models.

[0039] Accordingly, the generation device generates semantic feature vectors corresponding to natural language descriptions through a preset visual semantic understanding model, generates visual feature matrices corresponding to two-dimensional sketches through a preset computer vision algorithm, and / or analyzes the three-dimensional model to obtain a geometric feature dataset corresponding to the three-dimensional model. The semantic feature vectors, visual feature matrices, and / or geometric feature datasets constitute the multimodal features of the scene creation requirement data.

[0040] According to some embodiments, for natural language descriptions, the generation device performs word segmentation, entity recognition and semantic mapping through a pre-trained visual semantic understanding model, and extracts core semantic features such as objects, spatial relationships, lighting and shadow tones and physical attributes in the scene to generate semantic feature vectors.

[0041] According to some embodiments, for a two-dimensional sketch, the generating device uses computer vision algorithms such as edge detection, contour extraction, and region segmentation to identify visual features such as object outlines, relative positions, and size ratios in the sketch in order to generate a visual feature matrix.

[0042] According to some embodiments, for a 3D model, the generation device analyzes the geometric features of the model, such as vertex coordinates, surface structure, material parameters, and spatial transformation matrix, to generate a geometric feature dataset.

[0043] Then, according to the preset weight ratio, the generation device fuses the semantic feature vector, visual feature matrix and / or geometric feature dataset through a preset feature fusion network to obtain the fused multimodal features.

[0044] According to some embodiments, the generation device maps semantic feature vectors, visual feature matrices and / or geometric feature datasets to the same feature space through a feature fusion network, and assigns weights to different modal features through an attention mechanism according to a preset weight ratio, strengthening core features such as object name, spatial location and lighting parameters, and weakening other irrelevant and redundant features.

[0045] Simultaneously, the generation device encodes the fused multimodal features to generate a fused feature tensor. This fused feature tensor contains information on all elements of the scene.

[0046] The generation device performs structured decomposition and reconstruction of the fused feature tensor according to spatial, object, and attribute dimensions to obtain a structured scene dataset. This structured scene dataset includes basic spatial coordinate data, object topology data, and attribute parameter data.

[0047] According to some embodiments, in the spatial dimension, the generation device establishes a three-dimensional Cartesian coordinate system to define the overall spatial extent and coordinate axis orientation of the scene, and generates basic spatial coordinate data. In the object dimension, the generation device assigns a unique identifier to each identified scene object to record the relative positions, connections, and hierarchical relationships between objects, and generates object topology data. In the attribute dimension, the generation device extracts and standardizes the material, color, and reflectivity of each object, as well as the overall lighting type, light source position, light intensity, perspective parameters, etc., of the scene to generate attribute parameter data.

[0048] The generation device integrates spatial coordinate data, object topology data, and attribute parameter data to obtain a structured scene dataset.

[0049] According to some embodiments, a structured scene dataset includes the scene's spatial coordinates, object relationships, lighting and shadow tones, and physical properties.

[0050] Furthermore, the generation device performs format conversion on the structured scene dataset according to preset scene description rules.

[0051] According to some embodiments, the scene description rules can be customized according to scene requirements and are compatible with mainstream AI generation models for text-to-image and text-to-video. Only the core scene parameters required by the AI ​​generation model are retained, while redundant 3D modeling detail parameters are removed.

[0052] According to some embodiments, during the format conversion process, the generation device maps the structured scene dataset into a key-value pair format, vector format, or matrix format specified by the scene description rules, so as to ensure that the artificial intelligence generation model can directly parse and read it.

[0053] The generation device performs data verification on the structured scene dataset that has undergone format conversion.

[0054] According to some embodiments, data verification of the structured scene dataset includes verification of legality, completeness, and consistency. Specifically, the generation device verifies the rationality of spatial coordinates in the structured scene dataset to avoid overlapping object positions and contradictory perspective relationships. The generation device also verifies the matching of object attributes and lighting parameters in the structured scene dataset to ensure that the physical logic of material reflectivity and light source parameters is consistent. Furthermore, the generation device automatically corrects problems found during the verification of the structured scene dataset, and completes missing key parameters according to industry-standard specifications.

[0055] The generation device compresses the structured scene dataset that has undergone data verification to obtain three-dimensional scene description data.

[0056] According to some embodiments, the generation device performs lightweight compression on the structured scene dataset that has been verified by data, reducing the computational consumption for data transmission and parsing, and finally generates standardized, computable, and AI-compatible 3D scene description data for use in the generation of 3D reference scenes.

[0057] Based on the 3D scene description data, the generation device generates a 3D reference scene through the platform's visualization interface.

[0058] According to some embodiments, the generation device generates an interactively editable 3D reference scene in the platform's visual interface based on 3D scene description data. Platform users can adjust elements such as objects, lighting, and perspective in the 3D reference scene in real time until a reference scene that meets their creative needs is determined, serving as a unified base for all subsequent generation activities.

[0059] In step S200, a standardized constraint instruction set is configured based on the three-dimensional benchmark scene.

[0060] The generation device acquires preset camera position parameters, character element parameters, and composition style parameters.

[0061] According to some embodiments, camera position parameters can be set according to the narrative needs of platform users, including parameters such as shot type (e.g., close-up, medium shot, long shot), angle, and focal length. Character element parameters include parameters such as the character's appearance, position, and actions. Composition style parameters include parameters such as aspect ratio, visual focus, and color tone.

[0062] Based on a 3D baseline scene, the generation device transforms camera position parameters, character element parameters, and composition style parameters to generate a standardized set of constraint instructions.

[0063] According to some embodiments, the generation device transforms camera position parameters, character element parameters, and composition style parameters into spatial constraints, compositional constraints, and style constraints for the AI-generated model, thereby generating a standardized constraint instruction set. This standardized constraint instruction set is associated with the 3D scene description data corresponding to the 3D baseline scene to ensure that subsequent generation actions strictly adhere to the scene base and the creative intent of the platform user.

[0064] In step S300, the model generation result in the three-dimensional reference scene is obtained by using a preset artificial intelligence generation model based on the standardized constraint instruction set.

[0065] The generation device performs hierarchical parsing of the standardized constraint instruction set according to a preset priority.

[0066] According to some embodiments, the generation device performs hierarchical parsing of the standardized constraint instruction set according to spatial constraints, compositional constraints, and / or style constraints to obtain spatial constraint instructions, compositional constraint instructions, and / or style constraint instructions.

[0067] According to some embodiments, the generation device assigns a corresponding model control dimension and parsing priority to each layer of constraint instructions after hierarchical parsing. Specifically, spatial constraint instructions correspond to the spatial generation dimension of the AI-generated model, including instructions such as the spatial coordinates of the camera position, the spatial position and movement trajectory of the character, and the spatial perspective relationship of objects; these have the highest parsing priority. Composition constraint instructions correspond to the image composition dimension of the AI-generated model, including instructions such as shot size, aspect ratio, and visual focus position; these have the next highest parsing priority. Style constraint instructions correspond to the visual style dimension of the AI-generated model, including instructions such as color tone, light and shadow intensity, and image texture; these have the lowest parsing priority.

[0068] According to some embodiments, spatial constraint instructions, composition constraint instructions, and / or style constraint instructions all include binding identifiers for 3D scene description data corresponding to the 3D reference scene, to ensure a one-to-one correspondence between constraint instructions and scene data.

[0069] The generation device transforms spatial constraint instructions, compositional constraint instructions, and / or style constraint instructions to obtain hierarchical constraint parameters that can be recognized by the artificial intelligence generation model.

[0070] According to some embodiments, the AI-generated model can be a model based on mainstream architectures such as Transformer and diffusion models. The generation device converts spatial constraint instructions into spatial attention masks that the model can recognize, composition constraint instructions into image composition guidance vectors that the model can recognize, and style constraint instructions into style feature embedding vectors that the model can recognize.

[0071] The generation device concatenates the 3D scene description data corresponding to the 3D reference scene with the hierarchical constraint parameters to obtain the comprehensive input data for the artificial intelligence generation model.

[0072] According to some embodiments, the artificial intelligence generation model is equipped with a custom adaptation module, which can ensure that the hierarchical constraint parameters match the feature dimensions and computational logic of the model, and achieve seamless access to instructions.

[0073] Based on the comprehensive input data, the generation device obtains the model generation result through artificial intelligence generation model.

[0074] According to some embodiments, the generation device monitors and constrains the model generation results in real time across spatial, compositional, and / or stylistic dimensions by integrating hierarchical constraint parameters from the input data. Specifically, in the spatial dimension, the generation device uses spatial attention masks to restrict the spatial feature generation of the AI-generated model, ensuring that the object positions, perspective relationships, and camera angles of the generated content are completely consistent with the 3D baseline scene and camera position parameters, avoiding spatial distortion. In the compositional dimension, the generation device guides the image feature generation of the AI-generated model through image composition guidance vectors, ensuring that the framing, aspect ratio, and visual focus meet preset requirements. In the stylistic dimension, the generation device constrains the visual style generation of the AI-generated model through style feature embedding vectors, ensuring that color, lighting, texture, and overall tone remain consistent with the 3D baseline scene.

[0075] The generation device extracts feature parameters of the model generation result through preset image recognition rules.

[0076] According to some embodiments, the feature parameters of the model generation result include spatial perspective parameters, object position parameters, camera position parameters, aspect ratio, visual focus coordinates, color histogram, lighting parameters, and style features.

[0077] The generation device determines whether the matching degree between the feature parameters of the generated model and the standardized constraint instruction set reaches a preset threshold. The preset threshold includes the threshold corresponding to each constraint instruction in the standardized constraint instruction set.

[0078] According to some embodiments, the generation device extracts spatial perspective parameters, object position parameters, and camera position parameters from the model-generated result, and compares them numerically with spatial constraint instructions in a standardized constraint instruction set to determine whether the threshold corresponding to the spatial constraint instruction has been met. The generation device extracts the aspect ratio and visual focus coordinates from the model-generated result, and calculates the matching degree with composition constraint instructions in the standardized constraint instruction set to determine whether the threshold corresponding to the composition constraint instruction has been met. The generation device extracts the color histogram, lighting parameters, and style features from the model-generated result, and performs similarity analysis with style constraint instructions in the standardized constraint instruction set to determine whether the threshold corresponding to the style constraint instruction has been met.

[0079] If the matching degree between the feature parameters of the model generation result and each constraint instruction in the standardized constraint instruction set reaches the corresponding preset threshold, the generation device determines that the model generation result has passed the matching degree verification.

[0080] Then, the generation device outputs the model generation results, which have passed the matching degree verification, to the platform's visualization interface.

[0081] If the matching degree between the feature parameters of the model generation result and each constraint instruction in the standardized constraint instruction set does not reach the corresponding preset threshold, the generation device acquires the deviation data between the feature parameters and the standardized constraint instruction set, and uses this data to optimize the parameters of the standardized constraint instruction set until the model generation result passes the matching degree verification.

[0082] According to some embodiments, the generation device can generate a constraint instruction correction scheme based on the deviation data between the feature parameters and the standardized constraint instruction set, so as to adjust the parameter weights of the corresponding constraint instructions in the standardized constraint instruction set, and provide an optimized standardized constraint instruction set for the next model generation result, until the model generation result passes the matching degree verification.

[0083] In step S400, the three-dimensional benchmark scene, the standardized constraint instruction set, and the model generation results are correlated to form reusable artificial intelligence visual asset data.

[0084] The generation device correlates the 3D baseline scene, the standardized constraint instruction set, and the model generation results.

[0085] According to some embodiments, the generation device obtains the correspondence between the three-dimensional reference scene, the standardized constraint instruction set and the model generation result, and uses this correspondence to associate and bind the three-dimensional reference scene, the standardized constraint instruction set and the model generation result.

[0086] The generation device assigns unique identifiers to the associated 3D baseline scene, standardized constraint instruction set, and model generation results to form artificial intelligence visual asset data.

[0087] According to some embodiments, the generation device uniformly stores the identified and associated 3D benchmark scene, standardized constraint instruction set and model generation results, and assigns a corresponding unique identifier to the uniformly stored data to form artificial intelligence visual asset data.

[0088] The generating device stores AI visual asset data on both ends.

[0089] According to some embodiments, AI visual asset data can be stored in both the cloud and local storage, and can be directly retrieved in subsequent content creation to achieve data reuse.

[0090] For example, platform users can directly access the 3D baseline scene in the stored AI visual asset data, modify local elements (such as adjusting character actions, adding scene objects, etc.) to generate new content, and realize the traceability of scene status and the iterability of versions.

[0091] According to the embodiments of this application, a three-dimensional reference scene is used as a unified basis, and all generation behaviors are carried out under scene constraints, which solves the problem of consistency of generated content, realizes precise control of the entire creation process, and enables the sustainable iteration and large-scale reuse of visual assets.

[0092] The technical solution of this application will be described in detail below with reference to specific embodiments.

[0093] Example 1: Production of short drama series.

[0094] Suppose a short drama production team (hereinafter referred to as the first user) needs to produce a 10-episode continuous narrative short drama. The core requirement is to ensure the visual consistency of the main character's image and core scenes (such as the main character's study) throughout the entire drama, while improving the efficiency of storyboard design and content generation. The specific implementation can be carried out in the following steps.

[0095] 1. Construction of a 3D baseline scene.

[0096] The generation device obtains the natural language description uploaded by the first user—"a 20-square-meter retro study room with a solid wood desk by the window, a bookshelf on the left, warm yellow overhead lighting, and soft light and shadow"—as well as a simple sketch of the desk and bookshelf through the platform's visual interface, using this as the scene creation requirement data. The generation device converts the above scene creation requirement data into three-dimensional scene description data and generates an interactive study room scene (i.e., a three-dimensional baseline scene) to be displayed to the first user.

[0097] The first user fine-tuned the arrangement of books on the bookshelves and the layout of items on the desk in the study scene through the platform's visual interface, thus finalizing the study scene. Simultaneously, the first user imported the protagonist's 2D character design sketch through the platform's visual interface, converted it into a 3D character model, added it to the study scene, and determined the character's basic appearance parameters.

[0098] 2. Generate constraint configuration.

[0099] The first user configured preset parameters for three key shots in episode 1 within the study scene, including: Shot 1, medium shot, frontal view, the protagonist sitting at the desk reading a book; Shot 2, close-up, side view, the protagonist picking up a water glass from the desk; Shot 3, wide shot, rear view, the protagonist standing up and walking towards the bookshelf.

[0100] The generation device converts the above parameters into a set of constraint instructions, which includes spatial constraint instructions, composition constraint instructions, and style constraint instructions, and binds them to the study scene.

[0101] 3. Controllable content generation.

[0102] The generation device synchronizes the description data and constraint instruction set corresponding to the study scene to the preset Wensheng video model (i.e., the artificial intelligence generation model), generating a video clip with three shots. The first user discovered through the platform's visualization interface that the protagonist's hand movements in shot 2 did not match the position of the water cup on the desk. Without changing the study scene, the constraint parameters of the character's hand movements were fine-tuned, triggering the model's secondary generation to quickly obtain the optimized video clip.

[0103] 4. Asset iteration and reuse.

[0104] The generation device stores the study scene, the main character's appearance parameters, and the constraint instruction set for episode 1. When producing the subsequent 9 episodes of the short drama, the first user can directly access the study scene and only adjust the character's movements and camera positions according to the plot requirements (such as adding overhead shots and adjusting character positions), without having to rebuild the scene. At the same time, the first user connects the generated shots of each episode into a storyboard, optimizing the narrative rhythm in real time. Ultimately, the content production efficiency of the 10 episodes of the short drama is 80% higher than that of traditional AI generation methods, and the visual consistency of the main character's appearance and the study scene is maintained throughout, with no rework costs.

[0105] Example 2: Production of cutscenes for standalone games.

[0106] An independent game developer (hereinafter referred to as the second user) needs to produce cutscenes for multiple games. The core requirement is to ensure the consistency of scene style (such as a magical forest) and character image within the game's world view, while quickly completing the testing of multiple versions of storyboards. The specific implementation can be carried out in the following steps.

[0107] 1. Construction of a 3D baseline scene.

[0108] The second user imports a simplified 3D model of the magical forest through the platform's visual interface, and simultaneously inputs a natural language description: "There are glowing ancient trees in the forest, moss on the ground, cool blue ambient light, and light spots filtering through the leaves." The generation device generates an editable magical forest scene (i.e., a 3D baseline scene) by optimizing the model's lighting and physical properties. The second user then adjusts the glowing intensity of the ancient trees and the distribution range of the moss to determine the final magical forest scene as the base.

[0109] 2. Generate constraint configuration.

[0110] The second user sets two different sets of parameters in the magical forest scene: Set A, low camera position, following the protagonist's movement to emphasize the vastness of the forest; Set B, level camera position, at eye level with the protagonist to emphasize the details of the forest. The generation device generates corresponding standardized constraint instruction sets for each of the two different sets of parameters.

[0111] 3. Controllable content generation.

[0112] The generation device generates corresponding cutscene clips based on two sets of constraint instruction sets with different parameters. The second user compares the effects of the two versions of the cutscene clips through the platform's visual interface, quickly determining that Suite A is the final storyboard solution. Simultaneously, addressing the insufficient bokeh effect in some shots within Suite A, the second user fine-tunes the lighting constraint parameters, achieving real-time generation optimization.

[0113] 4. Asset iteration and reuse.

[0114] The second user stores the enchanted forest scene and the main character's appearance parameters. When creating other cutscenes later, the second user only needs to add new characters (such as game villains) to the enchanted forest scene and adjust the camera position to quickly generate new animation content, realizing the reuse of scene assets and significantly reducing the production cost of game art.

[0115] Figure 2 A schematic diagram of an apparatus for generating artificial intelligence visual asset data according to an example embodiment of this application is shown.

[0116] like Figure 2 As shown, the generation device 100 includes a scene construction module 110, an instruction configuration module 120, a content generation module 130, and an asset management module 140.

[0117] The scene construction module 110 acquires scene creation requirement data and extracts multimodal features from the scene creation requirement data, including semantic feature vectors, visual feature matrices and / or geometric feature datasets.

[0118] According to the preset weight ratio, the scene construction module 110 fuses the semantic feature vector, visual feature matrix and / or geometric feature dataset through a preset feature fusion network to obtain the fused multimodal features.

[0119] The scene construction module 110 encodes the fused multimodal features to generate a fused feature tensor.

[0120] The scene construction module 110 performs structured decomposition and reconstruction of the fused feature tensor according to the spatial dimension, object dimension and attribute dimension to obtain a structured scene dataset.

[0121] The scene construction module 110 performs format conversion on the structured scene dataset according to preset scene description rules.

[0122] The scene construction module 110 performs data validation on the structured scene dataset after format conversion.

[0123] The scene construction module 110 compresses the structured scene dataset that has undergone data verification to obtain three-dimensional scene description data.

[0124] Based on the 3D scene description data, the scene construction module 110 generates a 3D benchmark scene through the platform's visualization interface.

[0125] The instruction configuration module 120 obtains preset camera position parameters, character element parameters, and composition style parameters.

[0126] Based on the 3D baseline scene, the instruction configuration module 120 converts the camera position parameters, character element parameters, and composition style parameters to generate a standardized constraint instruction set.

[0127] The content generation module 130 performs hierarchical parsing of the standardized constraint instruction set according to a preset priority to obtain spatial constraint instructions, composition constraint instructions, and / or style constraint instructions.

[0128] The content generation module 130 transforms spatial constraint instructions, composition constraint instructions, and / or style constraint instructions to obtain hierarchical constraint parameters that can be recognized by the artificial intelligence generation model.

[0129] The content generation module 130 concatenates the 3D scene description data corresponding to the 3D reference scene with the hierarchical constraint parameters to obtain the comprehensive input data for the artificial intelligence generation model.

[0130] Based on the comprehensive input data, the content generation module 130 obtains the model generation results through an artificial intelligence generation model.

[0131] The content generation module 130 extracts feature parameters of the model-generated results through preset image recognition rules.

[0132] The content generation module 130 determines whether the matching degree between the feature parameters of the model generation result and the standardized constraint instruction set reaches a preset threshold.

[0133] If the matching degree between the feature parameters of the model generation result and each constraint instruction in the standardized constraint instruction set reaches the corresponding preset threshold, the content generation module 130 determines that the model generation result passes the matching degree verification.

[0134] The content generation module 130 outputs the model generation results that have passed the matching degree verification to the platform's visualization interface.

[0135] If the matching degree between the feature parameters of the model generation result and each constraint instruction in the standardized constraint instruction set does not reach the corresponding preset threshold, the content generation module 130 obtains the deviation data between the feature parameters and the standardized constraint instruction set, and uses it to optimize the parameters of the standardized constraint instruction set until the model generation result passes the matching degree verification.

[0136] The asset management module 140 associates the 3D baseline scene, the standardized constraint instruction set, and the model generation results.

[0137] The asset management module 140 sets unique identifiers for the associated 3D benchmark scenes, standardized constraint instruction sets, and model generation results to form artificial intelligence visual asset data.

[0138] The asset management module 140 stores AI-powered visual asset data on both ends.

[0139] Figure 3 A schematic diagram of an electronic device according to an example embodiment of this application is shown.

[0140] like Figure 3 As shown, the electronic device 600 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0141] like Figure 3 As shown, the electronic device 600 is manifested in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc. The storage unit stores program code, which can be executed by the processing unit 610, causing the processing unit 610 to perform the methods described in this specification according to the various exemplary embodiments of this application. For example, the processing unit 610 can perform, for example... Figure 1 The method shown.

[0142] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include read-only memory (ROM) 6203.

[0143] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0144] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0145] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0146] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. The technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0147] Software products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0148] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0149] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0150] The aforementioned computer-readable medium carries one or more programs, which, when executed by a device, cause the computer-readable medium to perform the aforementioned functions.

[0151] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0152] The embodiments of this application have been described in detail above. These descriptions are solely for the purpose of helping to understand the method and core ideas of this application. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of this application, its specific implementation methods, and its application scope, are all within the scope of protection of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for generating artificial intelligence visual asset data, characterized in that, include: Acquire multimodal scene creation requirement data to construct a 3D benchmark scene; Based on the aforementioned three-dimensional benchmark scene, configure a standardized constraint instruction set; Based on the standardized constraint instruction set, the model generation result in the three-dimensional benchmark scene is obtained through a preset artificial intelligence generation model; The three-dimensional baseline scene, the standardized constraint instruction set, and the model generation results are correlated to form reusable artificial intelligence visual asset data.

2. The method according to claim 1, characterized in that, Acquire multimodal scene creation requirement data to construct a 3D baseline scene, including: Extract multimodal features from the creation requirement data of the aforementioned scenarios; The multimodal features are fused using a pre-defined feature fusion network to obtain a fused feature tensor; A structured scene dataset is generated based on the fused feature tensor; The structured scene dataset is transformed according to preset scene description rules to generate three-dimensional scene description data; Based on the 3D scene description data, the 3D benchmark scene is generated through the platform's visualization interface.

3. The method according to claim 2, characterized in that, The scene creation requirement data includes natural language descriptions, two-dimensional sketches, and / or three-dimensional models; The extraction of multimodal features from the scene creation requirement data includes: The semantic feature vector corresponding to the natural language description is generated by using a preset visual semantic understanding model; A visual feature matrix corresponding to the two-dimensional sketch is generated using a preset computer vision algorithm; and / or The 3D model is parsed to obtain the geometric feature dataset corresponding to the 3D model.

4. The method according to claim 3, characterized in that, The multimodal features are fused using a pre-defined feature fusion network to obtain a fused feature tensor, including: According to a preset weight ratio, the semantic feature vector, the visual feature matrix, and / or the geometric feature dataset are fused through the feature fusion network to obtain the fused multimodal features; The fused multimodal features are encoded to generate the fused feature tensor.

5. The method according to claim 1, characterized in that, Based on the aforementioned three-dimensional benchmark scene, a standardized constraint instruction set is configured, including: Obtain preset camera position parameters, character element parameters, and composition style parameters; Based on the three-dimensional reference scene, the camera position parameters, character element parameters, and composition style parameters are transformed to generate the standardized constraint instruction set.

6. The method according to claim 5, characterized in that, Based on the standardized constraint instruction set, the model generation result in the three-dimensional reference scene is obtained through a preset artificial intelligence generation model, including: The standardized constraint instruction set is parsed hierarchically according to a preset priority. Based on the standardized constraint instruction set that has undergone hierarchical parsing, the comprehensive input data of the artificial intelligence generation model is obtained; Based on the comprehensive input data, the model generation result is obtained through the artificial intelligence generation model; Based on the standardized constraint instruction set, the matching degree of the model generation result is verified.

7. The method according to claim 6, characterized in that, Based on the standardized constraint instruction set, the matching degree of the model generation result is verified, including: Feature parameters of the model's generated results are extracted using preset image recognition rules; Determine whether the matching degree between the feature parameters and the standardized constraint instruction set reaches a preset threshold; If the matching degree reaches the preset threshold, the model generation result is determined to have passed the matching degree verification. The model generation results that pass the matching degree verification are output to the platform's visualization interface.

8. A device for generating artificial intelligence visual asset data, characterized in that, include: The scene building module is used to acquire multimodal scene creation requirement data in order to build a 3D benchmark scene; The instruction configuration module is used to configure a standardized constraint instruction set based on the three-dimensional reference scene; The content generation module is used to obtain the model generation result in the three-dimensional benchmark scene according to the standardized constraint instruction set and through a preset artificial intelligence generation model; The asset management module is used to associate the three-dimensional benchmark scene, the standardized constraint instruction set, and the model generation results to form reusable artificial intelligence visual asset data.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method as described in any one of claims 1-7.

Images