Method and system for quickly generating movie and television animation scene in combination with AI algorithm

Abstract

The invention relates to the technical field of generative artificial intelligence, and particularly provides a film and television animation scene rapid generation method and system combined with an AI algorithm, and the method comprises the steps: firstly receiving scene description text input, carrying out the semantic analysis, obtaining a scene element set and element association features, and then calling a pre-trained scene resource matching model, matching a scene resource combination in a multi-modal resource library according to an analysis result, then performing spatio-temporal layout optimization on the resource combination, generating a layout adjustment parameter and a dynamic adjustment parameter, constructing a three-dimensional space topological structure according to the layout adjustment parameter and the dynamic adjustment parameter, and rendering to generate a target animation scene frame sequence, and finally, synchronously calibrating the target animation scene frame sequence and a preset plot time axis, and outputting a complete animation scene flow, thereby improving the efficiency and accuracy of movie and television animation scene generation by means of an AI algorithm, and realizing rapid and intelligent scene generation highly conforming to the plot.

Description

Technical Field

[0001] The present invention relates to the field of generative artificial intelligence technology, and more specifically, to a method and system for quickly generating film and television animation scenes in combination with AI algorithms. Background Art

[0002] In today's film and television animation production field, scene generation is a key and complex link. Traditional film and television animation scene generation methods have many limitations.

[0003] In the early days, the production of film and television animation scenes relied primarily on manual drawing. Animators relied on their own drawing skills and imagination to create scenes frame by frame. This method was not only extremely inefficient and time-consuming, but also limited its ability to express complex scenes, making it difficult to achieve dynamic changes and precise control of details. With the advancement of computer technology, 3D modeling software has gradually been applied to the production of film and television animation scenes. Producers used professional 3D modeling tools to create a 3D model of the scene, then applied materials, set up lighting, and rendered the scene. However, this method still suffered from operational complexity and a high technical barrier to entry. Producers required professional knowledge in 3D modeling and animation production, resulting in a high learning curve. Furthermore, acquiring and integrating scene resources often required manual searching and selection from various libraries, a tedious process that was prone to resource mismatches.

[0004] Traditional methods for generating dynamic scenes typically pre-define scene change rules and parameters, lacking flexibility and intelligence. This makes efficient response and generation difficult when scenes need to be quickly adjusted to suit varying plot requirements. Furthermore, matching the generated scenes to the plot timeline often relies on manual adjustments, which can easily lead to timing inconsistencies, impacting the overall animation quality and viewing experience. Summary of the Invention

[0005] In view of the above-mentioned problems, in combination with the first aspect of the present invention, the present invention provides a method for quickly generating film and television animation scenes in combination with an AI algorithm, the method comprising: Receive a scene description text input, perform semantic parsing on the scene description text, and obtain a scene element set and element association features; Calling a pre-trained scene resource matching model to match corresponding scene resource combinations in a preset multimodal resource library according to the scene element set and the element association features; Performing spatiotemporal layout optimization processing on the scene resource combination to generate scene layout parameters and dynamic adjustment parameters; Constructing a three-dimensional space topology structure according to the scene layout parameters and the dynamic adjustment parameters, and rendering and generating a target animation scene frame sequence in the three-dimensional space topology structure; The target animation scene frame sequence is synchronized and calibrated with a preset plot timeline to output a complete animation scene stream.

[0006] On the other hand, the present invention also provides a rapid generation system for film and television animation scenes combined with AI algorithms, including a processor and a machine-readable storage medium, the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0007] Based on the above aspects, the present invention performs semantic parsing after receiving scene description text input, can accurately extract scene element sets and element association features, greatly improving the pertinence and accuracy of subsequent scene resource matching, calls a pre-trained scene resource matching model to perform matching in a multimodal resource library, fully leveraging the advantages of AI algorithms, can quickly and accurately find scene resource combinations that match the parsed results, avoiding the tedious and inefficient manual resource search in traditional methods, and performs spatiotemporal layout optimization processing on the matched scene resource combinations to generate reasonable scene layout parameters and dynamic adjustment parameters, so that the constructed scene is not only reasonably laid out in space, but also dynamically optimized in the time dimension. A three-dimensional spatial topological structure is constructed based on the scene layout parameters and dynamic adjustment parameters, and a target animation scene frame sequence is rendered to generate the target animation scene frame sequence, which can efficiently generate animation scene frames that meet the requirements. Finally, the target animation scene frame sequence is synchronized and calibrated with the preset plot timeline, and a complete animation scene stream is output, ensuring a high degree of temporal alignment between the animation scene and the plot, effectively shortening the generation cycle of film and television animation scenes, reducing production costs, and improving the quality of the animation scene and its matching degree with the plot. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a schematic diagram of the execution flow of a method for rapidly generating film and television animation scenes in combination with an AI algorithm provided in an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram of exemplary hardware and software components of a system for rapidly generating film and television animation scenes in combination with AI algorithms, provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0010] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1This is a flow chart of a method for quickly generating film and television animation scenes using an AI algorithm provided by an embodiment of the present invention. The method for quickly generating film and television animation scenes using an AI algorithm is introduced in detail below.

[0011] Step S110: receiving a scene description text input, performing semantic parsing on the scene description text, and obtaining a scene element set and element association features.

[0012] In this embodiment, in a film and television animation generation scenario, for example, the received scene description text is: "On a street in a bustling city, sunlight shines through the gaps between tall buildings onto the ground. A black sedan slowly drives up and stops in front of a roadside cafe. A stylishly dressed woman gets out of the car and walks into the cafe. Inside, several customers are sitting at tables chatting, and waiters are bustling between tables. Several artworks hang on the wall, and green plants are placed in the corner." The goal of semantic parsing the scene description text is to break down and analyze the information in the text to extract scene elements and their associated features. Scene elements are the basic objects that make up the scene, while element association features describe the relationships between these elements, such as positional relationships and temporal relationships.

[0013] Step S111: Sentence processing is performed on the scene description text to generate multiple semantic units.

[0014] In this embodiment, the scene description text can be sentence-processed according to its semantic and grammatical structure. For example, the scene description text can be divided into the following semantic units: "On a street in a bustling city, sunlight shines on the ground through the gaps between tall buildings," which describes the overall environment and lighting conditions of the street; "A black car slowly drives up and stops in front of a roadside cafe," which clarifies the movement and final position of the car; "A stylishly dressed woman gets out of the car and walks into the cafe," which explains the woman's movements and movement trajectory; "Inside the cafe, several customers are sitting at the table talking, and the waiters are busy shuttling between the tables," which describes the activities of people inside the cafe; "Several art paintings are hung on the wall, and green plants are placed in the corner," which supplements the decorative elements in the cafe. Each semantic unit contains specific semantic information, which will provide a basis for the subsequent generation of unit semantic vectors.

[0015] Step S112: calling a pre-trained semantic encoder to perform context encoding on each semantic unit to generate a unit semantic vector.

[0016] After obtaining multiple semantic units, a pre-trained semantic encoder is used to encode the context of each semantic unit. A pre-trained semantic encoder is a model trained on a large amount of text data, which can capture the semantic information and contextual relationships in the text.

[0017] Step S1121: performing word segmentation processing on the semantic unit to generate a vocabulary sequence.

[0018] For each semantic unit, we first perform word segmentation. For example, in the sentence "On a street in a bustling city, sunlight shines through the gaps between tall buildings onto the ground," the resulting word sequence is "in," "bustling," "city," "of," "a," "street," "on," "sunlight," "through," "tall buildings," "of," "gap," "sprinkled," "on," and "ground." The purpose of word segmentation is to break down semantic units into individual words for easier processing.

[0019] Step S1122: adding special markers at the beginning and end of the vocabulary sequence to generate an extended vocabulary sequence.

[0020] To help the semantic encoder better identify word sequence boundaries and context, special markers are added to the beginning and end of the generated word sequence. Assuming the special markers are "[CLS]" and "[SEP]," the above word sequence becomes "[CLS] On a bustling city street, sunlight filters through the gaps between tall buildings onto the ground [SEP]."

[0021] Step S1123: Input the expanded vocabulary sequence into the pre-trained semantic encoder, and extract the contextual dependencies between words through a multi-layer attention mechanism.

[0022] The expanded word sequence is fed into a pre-trained semantic encoder. Semantic encoders typically employ a multi-layer attention mechanism, which allows the model to focus on different relationships between words. At each layer, the model calculates the attention weight of each word toward other words based on factors such as the distance between the words and their semantic relevance. For example, the word "sunshine" may have a strong attention relationship with words like "high-rise building," "gap," and "sprinkle," as they semantically share a common description of a lighting scene. By calculating these multi-layer attention mechanisms, the model can gradually extract complex contextual dependencies between words.

[0023] Step S1124: extracting the position features corresponding to the special marker in the last hidden state layer of the semantic encoder, and performing weighted fusion on the features output from different layers to generate the unit semantic vector.

[0024] In the last hidden state layer of the semantic encoder, the positional feature corresponding to the special marker "[CLS]" is extracted. This positional feature can be considered a comprehensive representation of the entire semantic unit. At the same time, to fully utilize the information output by different layers, the features output by different layers are weighted and fused. Assume that the semantic encoder has n layers, and the features output by each layer are F1, F2, ..., Fn, respectively. The corresponding weights are w1, w2, ..., wn, respectively, and w1+w2+...+wn=1. The final unit semantic vector is generated using the weighted fusion formula: unit semantic vector = w1*F1+w2*F2+...+wn*Fn. This unit semantic vector contains rich semantic information about the semantic unit.

[0025] Step S1125: performing dimensionality reduction processing on the unit semantic vectors, and concatenating the unit semantic vectors after dimensionality reduction in the order of the semantic units to generate global semantic context features.

[0026] To reduce the dimensionality of the data and facilitate subsequent processing, the generated unit semantic vectors are subjected to dimensionality reduction. Dimensionality reduction can be performed using methods such as principal component analysis (PCA). Assuming the reduced unit semantic vectors are V1, V2, V3, V4, and V5 (corresponding to the five semantic units mentioned above), these reduced unit semantic vectors are concatenated in the order of the semantic units, resulting in a global semantic context feature = [V1, V2, V3, V4, V5]. This global semantic context feature integrates information from all semantic units, providing more comprehensive semantic information for subsequent clustering analysis and element extraction.

[0027] Step S113: performing cluster analysis on the unit semantic vector to determine a core semantic cluster and subsidiary semantic clusters.

[0028] Clustering analysis algorithms are used to cluster unit semantic vectors. The purpose of cluster analysis is to group semantically similar unit semantic vectors into the same category. The K-means clustering algorithm can be used, assuming that the number of cluster categories is set to k=2. During the clustering process, the K-means clustering algorithm can judge the similarity between unit semantic vectors based on the distance between them (such as the Euclidean distance), and group vectors with closer distances into the same cluster. After clustering, core semantic clusters and subsidiary semantic clusters are determined. The core semantic cluster contains the most critical and core semantic information in the scene, while the subsidiary semantic cluster contains relatively minor semantic information. For example, in this scene, semantic information about streets, cafes, and people's activities may belong to the core semantic cluster, while semantic information about decorative elements such as art paintings on the wall and plants in the corner may belong to the subsidiary semantic cluster.

[0029] Step S114: extracting main scene elements from the core semantic cluster and extracting auxiliary scene elements from the subsidiary semantic cluster, and merging them to form the scene element set.

[0030] Extract the primary scene elements from the core semantic cluster. Primary scene elements are the main objects that make up the scene and play a key role in the scene's core content. In this scene, the primary scene elements include the street, the black car, the stylishly dressed woman, the cafe, the customers, and the waiter. Extract the auxiliary scene elements from the auxiliary semantic cluster. Auxiliary scene elements are elements that supplement and modify the primary scene, such as the art on the wall and the green plants in the corner. Merge the primary and auxiliary scene elements to form the scene element set = {street, black car, stylishly dressed woman, cafe, customers, waiter, art, green plants}.

[0031] Step S115: Construct the spatial dependency relationship and timing trigger conditions between the main scene element and the auxiliary scene element to generate the element association features; wherein the spatial dependency relationship includes the relative position constraints and physical interaction rules between the elements, and the timing trigger conditions include the time threshold of the element state change and the event response logic.

[0032] Construct spatial dependencies between primary and secondary scene elements. For example, art paintings hang on the walls of a cafe, and green plants are placed in the corners of the cafe. This describes the relative positional constraints between them and the cafe. You can also define physical interaction rules, such as the communication and service interactions between customers and waiters within the cafe.

[0033] Construct temporal trigger conditions. The woman's actions of getting off her car and entering the cafe follow a specific time sequence, and customers' conversations within the cafe also last for a certain period of time. You can set time thresholds and event response logic, such as requiring the woman to enter the cafe within a certain timeframe after getting off her car, or requiring the waiter to serve the customer within a specified timeframe after placing an order. Based on these spatial dependencies and temporal trigger conditions, element association features are generated. Element association features comprehensively describe the relationships between scene elements, providing an important basis for subsequent scene resource matching.

[0034] Step S120: calling a pre-trained scene resource matching model, and matching a corresponding scene resource combination in a preset multimodal resource library according to the scene element set and the element association features.

[0035] After obtaining the scene element set and element-related features, it is necessary to call the pre-trained scene resource matching model to search for matching scene resource combinations in the preset multimodal resource library. The multimodal resource library contains various types of resources, such as 3D model resources, action clip resources, and lighting effects resources.

[0036] Step S121: constructing a resource index structure of the multimodal resource library, wherein the resource index structure includes a static model resource layer, a dynamic behavior resource layer, and an environmental special effect resource layer.

[0037] A resource index structure for a multimodal resource library is constructed, dividing resources into different layers. The static model resource layer includes various static 3D models, such as street models, car models, cafe models, and character models. The dynamic behavior resource layer includes character action clips, such as a woman getting out of a car and entering a cafe, customers chatting, and waiters serving. The environmental effects resource layer includes lighting effects, such as sunlight and cafe lighting. This hierarchical resource index structure facilitates resource matching.

[0038] Step S122: Call the first matching network of the pre-trained scene resource matching model to match the three-dimensional model resources that match the geometric shapes of the main scene elements and the auxiliary scene elements from the static model resource layer, and extract the topological features and material features of the three-dimensional model resources.

[0039] The first matching network of the pre-trained scene resource matching model is called. This first matching network is specifically used to match appropriate 3D model resources from the static model resource layer. For each element in the scene element set, a search is performed in the static model resource layer based on its geometric form. For example, for the "street" element, a 3D street model that matches its geometric form is searched; for the "black car" element, a 3D model of a black car is searched. After matching the appropriate 3D model resource, its topological features and material features are extracted. Topological features describe the structure and shape information of the 3D model, and material features describe the material properties of the 3D model surface, such as color, glossiness, etc.

[0040] Step S123: Call the second matching network of the pre-trained scene resource matching model to match action clip resources that are consistent with the physical interaction rules in the element association features from the dynamic behavior resource layer, and extract the motion trajectory features and key frame features of the action clip resources.

[0041] The second matching network of the scene resource matching model is invoked to match action clip resources from the dynamic behavior resource layer that match the physical interaction rules in the element-association features. For example, based on the physical interaction rules of a woman getting out of a car and walking into a cafe, the dynamic behavior resource layer is searched for corresponding action clip resources. After matching a suitable action clip resource, its motion trajectory features and keyframe features are extracted. Motion trajectory features describe the motion path of an object during the action, while keyframe features describe the key moments in the action.

[0042] Step S124: calling the third matching network of the pre-trained scene resource matching model, matching the lighting special effect resources that meet the timing trigger conditions from the environmental special effect resource layer, and extracting the color distribution characteristics and intensity change characteristics of the lighting special effect resources.

[0043] The third matching network of the scene resource matching model is invoked to match lighting effects resources from the environmental effects resource layer that match the timing trigger conditions. For example, based on the timing trigger condition of sunlight shining through the gaps between tall buildings onto the ground, the environmental effects resource layer is searched for corresponding lighting effects resources. After matching a suitable lighting effects resource, its color distribution characteristics and intensity variation characteristics are extracted. The color distribution characteristics describe the color distribution of the light, and the intensity variation characteristics describe how the light intensity changes over time.

[0044] Step S125: Combining and optimizing the three-dimensional model resources, the action clip resources, and the lighting special effect resources according to the spatial dependency relationship in the element association features to generate an initial scene resource combination.

[0045] The matched 3D model assets, action clip assets, and lighting effects assets are combined and optimized based on the spatial dependencies within the element association features. For example, the street model, car model, and cafe model are positioned relative to each other within the scene; the action clip asset of a woman getting out of her car and entering the cafe is associated with her 3D model; and the sunlight lighting effects asset is applied to the entire scene. This combined optimization generates the initial scene asset combination.

[0046] Step S126: performing resource conflict detection on the initial scene resource combination. When a resource attribute conflict is detected, adjusting resource parameters based on the timing trigger conditions in the element association features to generate an optimized scene resource combination.

[0047] Perform resource conflict detection on the initial scene resource combination. Resource conflicts may include geometric conflicts and compatibility conflicts.

[0048] Step S1261: Detecting the spatial overlap area between the collision volume of the three-dimensional model resource and the motion trajectory of the action clip resource.

[0049] Detects the spatial overlap between the collision volume of a 3D model asset and the motion trajectory of an action clip asset. For example, detect whether the motion trajectory of a woman getting out of a car and walking into a cafe overlaps with the collision volume of the car or cafe.

[0050] Step S1262: converting the collision volume of the three-dimensional model resource and the motion trajectory of the action clip resource into parameters in a relative coordinate system, and calculating the area ratio of the spatial overlapping region.

[0051] Convert the collision volume of the 3D model asset and the motion trajectory of the action clip asset into parameters in a relative coordinate system for unified calculations. Determine whether a geometric conflict exists by calculating the area ratio of the spatial overlap. Assuming the area of ​​the collision volume is A and the area of ​​the overlap between the motion trajectory and the collision volume is B, then the area ratio of the spatial overlap = B / A.

[0052] Step S1263: When the area ratio exceeds a preset threshold, it is determined that a geometric conflict exists.

[0053] When the calculated area ratio exceeds a preset threshold, a geometric conflict is determined to exist. The preset threshold can be set according to actual conditions, for example, 0.1. If the area ratio is greater than 0.1, a geometric conflict is considered to exist.

[0054] Step S1264: adjusting the motion trajectory of the action clip resource according to the priority or scaling the collision volume of the three-dimensional model resource based on a relative ratio.

[0055] When a geometric conflict is detected, adjustments are made based on the set priority. You can prioritize adjusting the motion trajectory of the action clip asset to avoid the collision volume. If adjusting the motion trajectory fails to resolve the conflict, the collision volume of the 3D model asset is scaled based on the relative scale.

[0056] Step S1265: Detecting the compatibility between the intensity change of the lighting special effect resource and the light reflection property of the material feature.

[0057] Check the compatibility between the intensity variations of lighting effects assets and the light reflection properties of material features. For example, check whether the sunlight shining on the surfaces of cars and cafes will cause overexposure or color distortion.

[0058] Step S1266: When overexposure or color distortion is detected, the intensity curve of the lighting effect resource is adjusted in segments based on the time threshold in the timing trigger condition.

[0059] When overexposure or color distortion is detected, the intensity curve of the lighting effects asset is adjusted in stages based on the time thresholds in the timing trigger conditions. For example, during periods of peak sunlight, the lighting intensity is appropriately reduced to avoid overexposure. These adjustments generate an optimized scene asset combination.

[0060] Step S130: performing spatiotemporal layout optimization processing on the scene resource combination to generate scene layout parameters and dynamic adjustment parameters.

[0061] After obtaining the optimized scene resource combination, it is necessary to perform spatiotemporal layout optimization to generate scene layout parameters and dynamically adjust parameters. The purpose of spatiotemporal layout optimization is to make the layout of elements in the scene more reasonable in space and time, in line with the logic and visual effects of the scene.

[0062] Step S131: Analyze the space occupancy attributes and motion attributes of each resource in the scene resource combination to generate an initial space distribution matrix.

[0063] Analyze the spatial occupancy and motion properties of each resource in the scene asset composition. For 3D model resources, spatial occupancy properties include the model's size and position; for action clip resources, motion properties include speed and direction. Based on these properties, generate an initial spatial distribution matrix. The initial spatial distribution matrix describes the spatial distribution of each resource in the scene at the initial moment.

[0064] Step S132: constructing spatial constraints according to the spatial dependencies in the element association features, and iteratively optimizing the initial spatial distribution matrix based on the spatial constraints.

[0065] Spatial constraints are constructed based on the spatial dependencies within the element-association features. For example, the relative position of streets and cafes, or the parking positions of cars and cafes, are considered. Based on these spatial constraints, the initial spatial distribution matrix is ​​iteratively optimized. During each iteration, the positions of each resource in the matrix are checked to see if they meet the spatial constraints. If not, adjustments are made.

[0066] Step S133: During each iteration, priority weights are generated according to the spatial dependencies in the element association features, and the visual focus weight matrix is ​​dynamically adjusted based on the priority weights. The matching score between the current spatial distribution matrix and the adjusted visual focus weight matrix is ​​calculated, and the position offset and rotation angle of the resource are adjusted in priority order according to the matching score.

[0067] During each iteration, priority weights are generated based on the spatial dependencies within the element-association features. For example, as the primary character in the scene, the woman's position is given a higher priority. The visual focus weight matrix is ​​dynamically adjusted based on the priority weights. The visual focus weight matrix describes the importance of each location in the scene. A matching score is calculated between the current spatial distribution matrix and the adjusted visual focus weight matrix. The matching score is calculated by calculating the similarity between the two matrices. Based on the matching score, the position offset and rotation angle of the asset are adjusted in order of priority to improve the matching.

[0068] Step S134: When the maximum number of iterations is reached or the matching score meets the convergence threshold, the final spatial distribution matrix is ​​locked and the scene layout parameters are extracted.

[0069] When the maximum number of iterations is reached or the matching score meets the convergence threshold, the final spatial distribution matrix is ​​locked. The maximum number of iterations can be set based on actual conditions, for example, to 100. The convergence threshold can also be set based on actual conditions, for example, to 0.9. The scene layout parameters are extracted from the final spatial distribution matrix. The scene layout parameters include information such as the final position and rotation angle of each resource.

[0070] Step S135: According to the timing trigger conditions in the element association characteristics, key time nodes are marked in the final spatial distribution matrix, and the dynamic adjustment parameters are generated; wherein the dynamic adjustment parameters include resource display and hide timing, motion speed curve and special effect intensity gradient function.

[0071] Based on the timing trigger conditions in the element-association features, key time nodes are annotated in the final spatial distribution matrix. For example, the time when the woman gets off the car or walks into the cafe. Based on these key time nodes, dynamic adjustment parameters are generated. These dynamic adjustment parameters include resource visibility and concealment timing, such as whether the car hides after the woman gets off the car; motion speed curves, such as the change in the woman's walking speed; and special effect intensity gradient functions, such as the function of how sunlight intensity changes over time.

[0072] Step S1351: parse the event response logic in the timing trigger condition, and extract the event trigger condition and the corresponding response action.

[0073] For the sequential trigger conditions in the previously constructed scenario, analyze the event response logic in detail. For example, for the event of a woman entering a cafe, the event trigger condition might be her arrival at the cafe entrance, and the corresponding response action would be the cafe door automatically opening and the waiter coming to greet her. From all the sequential trigger conditions, extract these event trigger conditions and response actions one by one. For example, if the event trigger is a customer placing an order, the corresponding response action is for the waiter to record the order and deliver it to the kitchen. These event trigger conditions and response actions constitute the basic logic of the state changes of elements in the scenario.

[0074] Step S1352: Map the event triggering condition to a corresponding time interval in the plot timeline.

[0075] The plot timeline represents the temporal progression of an entire film or animation scene. Extracted event triggers are mapped to corresponding time intervals on the plot timeline. For example, the event trigger of "a woman getting out of a car" should occur shortly after the car comes to a complete stop, as described in the scene description. Therefore, this event trigger is mapped to a specific interval on the plot timeline after the car comes to a complete stop. Similarly, the event trigger of "a customer placing an order" is mapped to a corresponding time interval on the plot timeline based on the chronological order of the customer's actions in the scene, such as entering the cafe, taking a seat, and browsing the menu.

[0076] Step S1353: according to the rendering frame rate of the target animation scene frame sequence, a plurality of sampling time points are set at equal intervals within the corresponding time interval, and the resource state parameters corresponding to each sampling time point are calculated.

[0077] The target animation scene frame sequence has a specific rendering frame rate, which determines the number of frames displayed per second. Based on the rendering frame rate, multiple sampling time points are set at equal intervals within the previously mapped time interval. For example, if the rendering frame rate is 24 frames per second, 120 sampling time points will be set for a 5-second event time interval. For each sampling time point, the corresponding resource state parameters are calculated. For example, when a woman enters a cafe, resource state parameters such as her position and posture will vary at different sampling time points. At the initial sampling time point, the woman may have just reached the cafe entrance, with initial values ​​for parameters such as her position coordinates and body orientation. As time progresses to subsequent sampling time points, as she gradually enters the cafe, her position coordinates will continuously change, and her posture may also change, such as from standing and pushing the door to stepping in. For the cafe door, at the sampling time point when the woman arrives at the door, the door is closed, but at subsequent sampling time points, it may be open, and the angle of opening is also a resource state parameter.

[0078] Step S1354: generating a parameter adjustment curve according to the type of the response action, wherein the parameter adjustment curve includes a linear change curve, an exponential decay curve, and a step transition curve.

[0079] Generate a corresponding parameter adjustment curve based on the type of response action. Different response actions require different types of curves to describe parameter changes. For example, for a response action like the automatic opening of a cafe door, a linear change curve might be used to describe the change in the door opening angle over time. Assuming the door opens evenly over a certain period of time, from closed (angle = 0) to fully open (angle = maximum), the change in angle over time is linear. For changes in the intensity of special effects, such as a response action where sunlight intensity gradually decreases over time, an exponential decay curve might be used to describe the change in intensity over time. For sudden changes in state, such as a response action where a waiter suddenly starts recording a customer's order after hearing it, a step transition curve might be used to describe the change in state parameters, i.e., the instantaneous transition from one state to another.

[0080] Step S1355: Associating the parameter adjustment curve with the sampling time point to generate a time-driven parameter control instruction.

[0081] Associate the generated parameter adjustment curve with the previously set sampling time points. For each sampling time point, determine the corresponding resource state parameter value based on the parameter adjustment curve. For example, at a certain sampling time point, the current opening angle of the cafe door is determined based on the linear variation curve; the current sunlight intensity is determined based on the exponential decay curve. These corresponding relationships are combined to generate time-driven parameter control instructions. This time-driven parameter control instruction specifies the state parameters of each resource at each sampling time point.

[0082] Step S1356: embedding the time-driven parameter control instruction into the final spatial distribution matrix to form an enhanced spatial distribution matrix with a timing mark.

[0083] The generated time-driven parameter control instructions are embedded into the final spatial distribution matrix. The final spatial distribution matrix originally only contains the spatial layout information of the scene resources. After embedding the time-driven parameter control instructions, an enhanced spatial distribution matrix with time stamps is formed. This enhanced spatial distribution matrix not only describes the spatial distribution of the scene resources but also includes dynamic information about how the resources change over time. For example, the enhanced spatial distribution matrix records not only the woman's initial position in space but also her position changes and posture changes at different time points.

[0084] Step S140: constructing a three-dimensional space topology structure according to the scene layout parameters and the dynamic adjustment parameters, and rendering and generating a target animation scene frame sequence in the three-dimensional space topology structure.

[0085] Based on the scene layout parameters and dynamic adjustment parameters obtained above, a three-dimensional space topology structure is constructed, and rendering is performed in the three-dimensional space topology structure to generate a target animation scene frame sequence.

[0086] Step S141: converting the scene layout parameters into a vertex data set and a material map mapping relationship in a three-dimensional coordinate system.

[0087] The scene layout parameters include information such as the position and rotation angle of each resource in the scene. These parameters are converted into a vertex data set and a material texture mapping relationship in a three-dimensional coordinate system. For each three-dimensional model resource, its vertex coordinates in the three-dimensional coordinate system are determined according to its position and posture in the scene layout parameters. These vertex coordinates constitute the vertex data set. At the same time, the material texture mapping relationship is determined for each model, that is, how the material texture on the model surface is mapped to the vertices of the model is determined. For example, for the wall model of a cafe, its position and shape in three-dimensional space are determined according to the scene layout parameters, and the vertex data set of the wall is obtained. Then, the material texture of the wall (such as brick texture) is mapped to these vertices according to the set mapping rules to form a material texture mapping relationship.

[0088] Step S142: generating a time axis control instruction according to the dynamic adjustment parameters, wherein the time axis control instruction includes a vertex displacement instruction, a material switching instruction, and a light intensity adjustment instruction.

[0089] Timeline control instructions are generated based on dynamic adjustment parameters. Dynamic adjustment parameters include information such as resource display and hiding timing, motion speed curves, and special effect intensity gradient functions. Different types of timeline control instructions are generated based on this information. Vertex displacement instructions control the change in the position of the model's vertices over time based on the motion speed curve. For example, for the model of a woman, vertex displacement instructions are generated based on her motion speed curve, causing the woman's position in three-dimensional space to change accordingly at different time points. Material switching instructions control the switching of model materials based on resource display and hiding timing and scene logic. For example, when the display effect of a certain area of ​​the cafe needs to change at a specific time, the material switching instruction is used to switch the material of that area to the corresponding material. Light intensity adjustment instructions control the change in light intensity over time based on the special effect intensity gradient function. For example, light intensity adjustment instructions are generated based on the function of how sunlight intensity changes over time, so that the lighting effect in the scene matches the actual situation.

[0090] Step S143: creating a virtual camera perspective in the three-dimensional space topological structure, and setting perspective transformation parameters according to a preset lens motion trajectory.

[0091] Create a virtual camera perspective within the 3D spatial topology. The virtual camera determines the viewer's perspective within the animated scene. Set perspective transformation parameters based on a preset lens motion trajectory. Lens motion trajectories can be fixed, moving, or rotating. For example, set a lens motion trajectory that follows a woman into a cafe. At different points in time, the virtual camera's position, orientation, and other perspective transformation parameters will be adjusted accordingly. When the woman first exits the car, the virtual camera might observe her from a distance and at a certain angle. As she walks toward the cafe, the virtual camera gradually follows and adjusts its perspective to maintain focus on her.

[0092] Step S144: Initialize a scene rendering context based on the vertex data set and the material map mapping relationship.

[0093] Initialize the scene rendering context based on the vertex data set and material texture mappings. The scene rendering context contains all the information needed to render the scene, such as the model's geometry, material textures, and lighting conditions. Loading the vertex data set and material texture mappings into the rendering context prepares it for subsequent rendering operations. For example, the vertex data set and material texture mappings for the cafe's walls are passed to the rendering engine. Based on this information, the rendering engine knows how to draw the cafe's walls.

[0094] Step S145: in the scene rendering context, updating vertex positions, material properties and lighting effects frame by frame according to the timeline control instructions, and adjusting the observation matrix of the virtual camera according to the perspective transformation parameters.

[0095] In the scene rendering context, vertex positions, material properties, and lighting effects are updated frame by frame according to the timeline control instructions. For each frame of rendering, the positions of the model vertices are updated according to the vertex displacement instructions, the model's material properties are updated according to the material switching instructions, and the lighting effects are updated according to the light intensity adjustment instructions. At the same time, the virtual camera's observation matrix is ​​adjusted according to the perspective transformation parameters. The observation matrix determines how the virtual camera observes the scene, and by adjusting the observation matrix, the virtual camera's perspective is changed. For example, in a certain frame, the woman's model's vertex position is moved to a new position according to the timeline control instructions, the material of a certain area of ​​the cafe is switched to a new material, and the lighting intensity is adjusted to simulate different time periods. At the same time, the virtual camera's observation matrix is ​​adjusted according to the perspective transformation parameters so that the scene perspective seen by the audience conforms to the preset lens motion trajectory.

[0096] Step S146: performing post-processing on the rendering result of each frame to generate the target animation scene frame sequence, wherein the post-processing includes anti-aliasing processing, dynamic blur overlay and color correction.

[0097] Post-process the rendered results of each frame to improve the visual quality of the animation scene. Post-processing operations include anti-aliasing, motion blur overlay, and color correction.

[0098] Step S1461: Calculate the pixel displacement vector according to the movement speed and direction of the virtual camera, generate a motion blur effect based on the pixel displacement vector, and after completing the motion blur processing, use multi-sampling anti-aliasing technology to smooth the edge contour of the rendered frame.

[0099] Pixel displacement vectors are calculated based on the speed and direction of the virtual camera's movement. The speed and direction of the virtual camera's movement determine the speed and direction of objects in the scene. By calculating pixel displacement vectors, the blur effect of moving objects can be simulated. Motion blur is generated based on pixel displacement vectors, making fast-moving objects appear blurred and trailing in the frame, enhancing the realism of the animation. After the motion blur is completed, multi-sampling anti-aliasing technology is used to smooth the edge contours of the rendered frame. Multi-sampling anti-aliasing technology reduces edge jaggedness by sampling multiple points around each pixel and averaging the colors of these points, resulting in smoother edges.

[0100] Step S1462: extracting the brightness histogram of the current frame, and adjusting the color saturation and contrast according to a preset tone mapping curve.

[0101] Extract the brightness histogram of the current frame. The brightness histogram describes the distribution of pixels of varying brightness values ​​within the image. Adjust color saturation and contrast based on a preset tone mapping curve. The tone mapping curve defines how the brightness and color values ​​of the original image are mapped to new brightness and color values. Adjusting color saturation can enhance or reduce the vividness of the image's colors; adjusting contrast can brighten highlights and darken shadows, enhancing the image's depth. For example, if a scene needs to create a bright and cheerful atmosphere, increase color saturation and contrast using a tone mapping curve.

[0102] Step S1463: Detect detail loss in highlight areas and shadow areas, and restore texture information using a local contrast enhancement algorithm.

[0103] Detect detail loss in highlight and shadow areas of the current frame. Under certain lighting conditions, highlight areas may be too bright, resulting in loss of detail, while shadow areas may be too dark, also losing some texture information. A local contrast enhancement algorithm is used to restore texture information in these areas. The local contrast enhancement algorithm analyzes different local areas of the image and adjusts based on the brightness and contrast of each area, making details in highlight and shadow areas clearer. For example, in the shadow areas of a cafe, the local contrast enhancement algorithm can make the textures of tables, chairs, and floors more distinct.

[0104] Step S1464: performing a difference comparison between the processed rendering frame and the previous frame to eliminate inter-frame flicker artifacts.

[0105] Compare the difference between the processed rendered frame and the previous frame. Because animations are composed of a series of consecutive frames, if the changes between adjacent frames are too drastic, inter-frame flickering artifacts may occur. By comparing the differences between adjacent frames, the parts with large differences are smoothed to eliminate inter-frame flickering artifacts. For example, when the virtual camera moves quickly, the image changes between adjacent frames may be relatively large. By comparing and processing the differences, the image transitions can be smoother.

[0106] Step S1465: Arrange the continuously processed rendering frames in chronological order to generate a visually coherent target animation scene frame sequence.

[0107] The rendered frames, which have undergone a series of post-processing, are arranged in chronological order. Because each frame is precisely controlled and processed during the rendering and post-processing process, a visually coherent target animation scene frame sequence is generated after chronological arrangement. Each frame in this target animation scene frame sequence has high visual quality and is temporally continuous, presenting a smooth and realistic animation scene to the audience.

[0108] Step S150: performing synchronization calibration processing on the target animation scene frame sequence and the preset plot timeline, and outputting a complete animation scene stream.

[0109] After obtaining the target animation scene frame sequence, it needs to be synchronized and calibrated with the preset plot timeline to ensure that the time logic of the animation scene is consistent with the plot, and finally output the complete animation scene stream.

[0110] For example, step S151: parsing key event markers and scene switching instructions in the plot timeline.

[0111] Analyze the key event markers and scene transition instructions in the pre-set plot timeline. Key event markers are significant moments in the plot, such as when a woman gets off a car or when a customer places an order. Scene transition instructions indicate when to switch scenes, such as from a street scene to a scene inside a cafe. By analyzing this information, you can understand the plot's temporal logic and scene change rules.

[0112] Step S152: Locate the start frame and the end frame corresponding to the key event marker in the target animation scene frame sequence.

[0113] Locate the start and end frames corresponding to key event markers in the target animation scene's frame sequence. Based on the event trigger conditions previously mapped to the plot timeline, find the corresponding frames in the animation scene's frame sequence. For example, for the key event marker of a woman getting off a bus, find the start frame where she begins getting off and the end frame where she completes the action. This positioning method determines the time range of each event in the animation scene.

[0114] Step S153: Calculate the time offset between the starting frame and the key event mark point. If the time offset exceeds a preset threshold, adjust the playback rate of the target animation scene frame sequence through a dynamic frame interpolation algorithm, and use an audio time stretching algorithm to maintain audio and video synchronization.

[0115] Calculate the time offset between the start frame and the key event marker. If the generation speed of the animation scene frame sequence does not completely match the plot timeline, there may be a time offset between the start frame and the key event marker. If the time offset exceeds the preset threshold, it means that the time difference between the two is large and needs to be adjusted. Adjust the playback rate of the target animation scene frame sequence through the dynamic frame interpolation algorithm. The dynamic frame interpolation algorithm can insert additional frames between frames or delete some frames to change the playback speed of the animation. At the same time, in order to maintain synchronization between sound and picture, the audio is processed using the audio time stretching algorithm. The audio time stretching algorithm can adjust the playback duration of the audio without changing the audio pitch so that it is consistent with the playback rate of the adjusted animation scene frame sequence.

[0116] Step S154: when the scene switching instruction is detected, the preloaded frame sequence of the next scene is extracted, and a gradual transition effect is inserted between the end frame of the current scene and the start frame of the next scene.

[0117] When a scene switch instruction is detected, the preloaded frame sequence of the next scene is extracted. To ensure the smoothness of the scene switch, the frame sequence of the next scene is usually loaded in advance. A gradient transition effect is inserted between the end frame of the current scene and the start frame of the next scene. Gradient transition effects can be fade-in, fade-out, blurred transition, etc., making the switching between scenes more natural. For example, when switching from a street scene to a scene inside a cafe, a fade-in, fade-out effect is inserted between the last frame of the street scene and the first frame of the cafe interior scene, making the audience feel that the scene transition is smoother.

[0118] Step S155: performing audio synchronization processing on the adjusted target animation scene frame sequence to align the time of the scene action with the background sound effect.

[0119] Perform audio synchronization on the adjusted target animation scene frame sequence. In addition to the audio time-stretching process previously performed to adjust the playback rate, further steps are required to ensure that the scene action and background sound effects are aligned. For example, the woman's action of closing the door should precisely match the sound effect of the door closing, and the customer's action of placing an order should be consistent with the sound effect of the order exchange. By fine-tuning the audio playback time, the scene action and background sound effects are synchronized.

[0120] Step S156: Encapsulating the target frame sequence after the audio synchronization processing into an animation file of a specified format to generate the complete animation scene stream.

[0121] The target frame sequence, after audio synchronization, is encapsulated into an animation file in a specified format. The specified format can be common animation file formats such as MP4 and AVI. During the encapsulation process, the animation frame sequence and the synchronized audio data are combined to generate a complete animation scene stream. This complete animation scene stream can be used for playback, storage, or further post-production, providing complete material for the final presentation of film and television animation.

[0122] For example, the above method may further include a method for training the scene resource matching model, including: Step S210: Construct a training data set, which includes a set of sample scene elements, sample element associated features and a corresponding annotated scene resource combination, wherein the annotated scene resource combination includes a triple of a static model resource identifier, a dynamic behavior resource identifier and an environmental special effect resource identifier.

[0123] Construct a training dataset for the scene resource matching model. Collect a large number of sample scene description texts, perform semantic parsing on these texts similar to the previous steps, and obtain a set of sample scene elements and sample element association features. At the same time, annotate the corresponding scene resource combination for each sample. The annotated scene resource combination is a triple containing a static model resource identifier, a dynamic behavior resource identifier, and an environmental special effect resource identifier. For example, for a forest adventure scene described in a sample scene description text, the annotated scene resource combination may be a static model resource identifier of "forest tree model, adventurer model, monster model", a dynamic behavior resource identifier of "adventurer attack action, monster defense action", and an environmental special effect resource identifier of "forest lighting special effect, magic light special effect".

[0124] Step S220: extracting a topological feature vector and a material feature vector from the three-dimensional model resource corresponding to the static model resource identifier, and normalizing the topological feature vector and the material feature vector to generate a static feature matrix.

[0125] For the 3D model resource corresponding to the static model resource identifier in the annotated scene resource assembly, extract the topological feature vector and material feature vector. The topological feature vector describes the model's structure and shape information, such as vertex connectivity and face distribution; the material feature vector describes the model's surface material properties, such as color and gloss. The extracted topological feature vectors and material feature vectors are normalized to generate a static feature matrix. The purpose of normalization is to map different feature vectors to the same scale range for subsequent processing. For the topological feature vector, calculate the minimum and maximum values ​​of each dimension and perform a linear transformation on each dimension to keep it within a specific range, such as 0 to 1. Similarly, perform a similar normalization operation on the material feature vector. Assuming the topological feature vector has m dimensions and the material feature vector has n dimensions, the normalized topological feature vector is denoted as T_norm and the material feature vector is denoted as M_norm. The normalized topological feature vectors T_norm and M_norm are concatenated in sequence to generate the static feature matrix S, i.e., S = [T_norm, M_norm]. The static feature matrix contains the topology and material information of the 3D model resource, and the scale of each feature dimension is unified, which facilitates subsequent model processing.

[0126] Step S230: extracting a motion trajectory tensor and a key frame sequence from the action clip resource corresponding to the dynamic behavior resource identifier, decomposing the motion trajectory tensor into a displacement component and a rotation component, performing zero-mean normalization processing on each component, and then splicing them into a dynamic feature matrix.

[0127] For the action clip resource corresponding to the dynamic behavior resource identifier in the annotated scene resource set, we first extract the motion trajectory tensor and keyframe sequence. The motion trajectory tensor describes the trajectory of the object's position change in three-dimensional space during the action. It is a tensor containing multiple time steps and multiple dimensions. The keyframe sequence records the state information of key moments in the action.

[0128] Next, the motion trajectory tensor is decomposed into displacement and rotation components. The displacement component describes the object's translational motion in space, while the rotation component describes its rotational motion. For the displacement component, the mean and standard deviation are calculated for each dimension. The mean is subtracted from the value in each dimension and then divided by the standard deviation to perform zero-mean normalization, resulting in the standardized displacement component D_std. The rotation component is also subjected to zero-mean normalization, resulting in the standardized rotation component R_std.

[0129] Assume that a keyframe sequence has p keyframes, each with q feature dimensions, and denote the keyframe sequence as K. The normalized displacement component D_std, rotation component R_std, and keyframe sequence K are sequentially concatenated to generate a dynamic feature matrix D, i.e., D = [D_std, R_std, K]. This dynamic feature matrix integrates the motion trajectory and keyframe information of the action clip resource, and each feature is normalized to ensure dimensional consistency.

[0130] Step S240: extracting the color intensity timing curve and attenuation coefficient of the lighting special effect resource corresponding to the environmental special effect resource identifier, aligning the time axis using piecewise linear interpolation, and converting them into special effect feature vectors of equal length.

[0131] For the lighting effects resource corresponding to the environmental effects resource identifier in the labeled scene resource combination, extract the color intensity timing curve and attenuation coefficient. The color intensity timing curve describes how the color intensity of the light changes over time, while the attenuation coefficient indicates the degree to which the light intensity decays with distance or time.

[0132] Because the timelines of different lighting effects assets may be inconsistent, piecewise linear interpolation is used to align them for ease of subsequent processing. Specifically, a unified time range and interval are determined, and piecewise linear interpolation is performed on the unified timeline for each lighting effects asset's color intensity timing curve, ensuring that different curves have corresponding intensity values ​​at the same time point.

[0133] Assuming there are r time points on a unified timeline, the interpolated color intensity timing curves and attenuation coefficients are arranged in sequence to form a vector, which is the equal-length special effect feature vector E. In this way, different lighting special effect resources are converted into special effect feature vectors with the same dimension, which facilitates the input of subsequent models.

[0134] Step S250: Construct a three-branch network architecture of the scene resource matching model, including: a first matching network composed of a multi-layer convolutional neural network, the number of input channels matches the dimension of the static feature matrix, and the output layer uses a Softmax activation function to generate a static resource matching probability distribution; a second matching network composed of a bidirectional long short-term memory network, the input sequence length is aligned with the time step of the dynamic feature matrix, and the output layer generates a dynamic resource matching probability distribution through a temporal attention mechanism; a third matching network composed of a fully connected layer, the input dimension is consistent with the special effect feature vector, and the output layer uses a Sigmoid activation function to generate a special effect resource matching probability distribution.

[0135] A three-branch network architecture is constructed for the scene resource matching model, where each branch network is responsible for processing different types of features.

[0136] The first matching network consists of a multi-layer convolutional neural network. Convolutional neural networks have powerful feature extraction capabilities and are suitable for processing static images or matrix data. The number of input channels matches the dimensions of the static feature matrix, ensuring that the input data can be correctly processed by the network. In the convolutional layer, different convolution kernels are used to extract local features from the static feature matrix. After multiple layers of convolution and pooling operations, the features are gradually abstracted and compressed. The output layer uses the Softmax activation function to convert the network output into a static resource matching probability distribution. The Softmax function maps each output value to a range between 0 and 1, and the sum of all output values ​​is 1, which represents the probability of each static resource being matched.

[0137] The second matching network consists of a bidirectional long short-term memory (Bi-LSTM) network. Bi-LSTMs can process sequential data and capture dependencies within a sequence, making them suitable for processing the characteristics of dynamic behavioral resources. The input sequence length is aligned with the time step length of the dynamic feature matrix, ensuring that the network can fully process the temporal sequence information of the action clip resources. In the Bi-LSTM layer, a gating mechanism controls the transmission and forgetting of information, thereby learning long-term dependencies within the sequence. The output layer uses a temporal attention mechanism to generate a probability distribution for dynamic resource matching. This temporal attention mechanism allows the network to focus on important time steps in the sequence, thereby improving matching accuracy.

[0138] The third matching network consists of a fully connected layer. This layer performs a linear transformation and nonlinear activation on the input special effect feature vector. Its input dimensions match those of the special effect feature vector, ensuring dimensionality matching. The output layer uses a Sigmoid activation function, mapping the output values ​​to a range between 0 and 1 to generate a probability distribution for special effect resource matching. The Sigmoid function converts the network output into a probability value, indicating the likelihood of each special effect resource being matched.

[0139] Step S260: Encode the main scene elements and auxiliary scene elements in the sample scene element set into one-hot vectors respectively, and perform tensor splicing with the spatial dependency matrix and temporal trigger condition matrix in the sample element association features to generate a joint query vector.

[0140] The primary and auxiliary scene elements in the sample scene element set are each encoded as a one-hot vector. A one-hot vector is a method for converting discrete variables into vector representations. Each element corresponds to a dimension in the vector, and only the dimension corresponding to that element is 1, while the remaining dimensions are 0. For example, if there are s different elements in the sample scene element set, each primary and auxiliary scene element can be represented by a one-hot vector of length s.

[0141] The spatial dependency matrix in the sample element association features describes the spatial positional relationships between scene elements, while the temporal trigger condition matrix describes the temporal relationships between element state changes. The encoded primary and auxiliary scene element one-hot vectors are tensor-concatenated with the spatial dependency matrix and the temporal trigger condition matrix. Tensor concatenation involves connecting different tensors along a certain dimension to form a larger tensor. This concatenation operation generates a joint query vector Q, which combines scene element information and element association feature information and serves as the query basis for subsequent matching.

[0142] Step S270: inputting the joint query vector into the three-branch network architecture, matching it with the static feature matrix, the dynamic feature matrix and the special effect feature matrix respectively, and generating an initial matching score.

[0143] The generated joint query vector is input into the three-branch network architecture of the scene resource matching model. In the first matching network, the joint query vector is matched with the static feature matrix. The network processes the joint query vector and the static feature matrix using learned parameters, calculates the degree of match between each static resource and the query vector, and generates an initial match score for the static resource.

[0144] In the second matching network, the joint query vector is matched with the dynamic feature matrix. Based on the time series information of the joint query vector and the dynamic feature matrix, the bidirectional long short-term memory network calculates the matching degree between each dynamic behavior resource and the query vector, generating an initial matching score for the dynamic resource.

[0145] In the third matching network, the joint query vector is matched with the special effects feature matrix. Based on the feature information of the joint query vector and the special effects feature matrix, the fully connected layer calculates the degree of match between each environmental special effects resource and the query vector, generating an initial matching score for the special effects resource.

[0146] These three initial matching scores reflect the matching status of static resources, dynamic resources and special effect resources with the joint query vector respectively.

[0147] Step S280: aggregating the initial matching scores using a weighted fusion layer to generate a comprehensive matching score, wherein the weight coefficients are dynamically adjusted according to the spatial dependency and temporal triggering conditions in the sample element association features.

[0148] A weighted fusion layer is used to aggregate the initial matching scores generated by the three branch networks. Weighted fusion is a method that combines multiple scores by assigning a weight coefficient to each score, then multiplying the scores by the weight coefficients and adding them together to obtain a composite score.

[0149] The weight coefficient is dynamically adjusted based on the spatial dependencies and temporal trigger conditions within the sample element association features. For example, if the spatial dependencies in a scene indicate that the location of static resources has a significant impact on the overall effect of the scene, then the weight coefficient corresponding to the initial matching score of static resources will be relatively large. If the temporal trigger conditions indicate that the time sequence of dynamic behavioral resources is important to the logic of the scene, then the weight coefficient corresponding to the initial matching score of dynamic resources will be relatively large.

[0150] Assume the initial matching score for static resources is S_score, the initial matching score for dynamic resources is D_score, and the initial matching score for special effects resources is E_score. The corresponding weight coefficients are w_s, w_d, and w_e, respectively, and w_s + w_d + w_e = 1. The comprehensive matching score C_score = w_s * S_score + w_d * D_score + w_e * E_score. This weighted fusion approach generates a comprehensive matching score that more comprehensively reflects the degree of match between the scene resource combination and the joint query vector.

[0151] Step S290: Calculate the cross entropy loss of the comprehensive matching score and the annotated matching score of the annotated scene resource combination, and superimpose the KL divergence of the static resource matching probability distribution, dynamic resource matching probability distribution, and special effect resource matching probability distribution as a regularization term to generate a joint loss function.

[0152] Calculate the cross-entropy loss between the comprehensive match score and the labeled match score for the combination of labeled scene resources. Cross-entropy loss is a loss function that measures the difference between two probability distributions and is often used in classification problems. The labeled match score is the true match probability distribution determined based on the combination of labeled scene resources, while the comprehensive match score is the matching probability distribution predicted by the model. By calculating the cross-entropy loss between these two distributions, the accuracy of the model's predictions can be measured.

[0153] The KL divergence of the static resource matching probability distribution, dynamic resource matching probability distribution, and special effect resource matching probability distribution is superimposed as a regularization term. KL divergence is an asymmetric measure of the difference between two probability distributions and is used to constrain the similarity between the model's output distribution and the true distribution. The role of the regularization term is to prevent model overfitting and improve the model's generalization ability.

[0154] Assume that the cross entropy loss is CE_loss, the KL divergence of the static resource matching probability distribution is KL_S, the KL divergence of the dynamic resource matching probability distribution is KL_D, and the KL divergence of the special effect resource matching probability distribution is KL_E. The joint loss function L=CE_loss+α*KL_S+β*KL_D+γ*KL_E, where α, β, and γ are regularization coefficients used to control the weight of the regularization term.

[0155] Step S2100: Use an adaptive gradient optimizer to iteratively train the scene resource matching model with a preset initial learning rate and batch size. After each round of training, use a validation set to calculate the topological consistency index and motion smoothness index. When the standard deviation of the topological consistency index of multiple consecutive rounds of validation sets is less than the preset convergence threshold, terminate the training and save the model parameters.

[0156] An adaptive gradient optimizer is used to iteratively train the scene resource matching model. This optimizer adaptively adjusts the learning rate based on the parameter gradients, improving training efficiency and stability. Presetting the initial learning rate and batch size are important training parameters. The initial learning rate determines the step size for parameter updates, while the batch size determines the number of samples used in each training session.

[0157] After each round of training, the validation set is used to calculate the topological consistency and motion smoothness metrics. The topological consistency metric measures the topological consistency of the static resources matched by the model with the annotated resources, while the motion smoothness metric measures the smoothness of the dynamic resources matched by the model during motion. By calculating these metrics, the model's performance on the validation set can be evaluated.

[0158] When the standard deviation of the topological consistency metric across multiple validation rounds is less than the preset convergence threshold, the model's performance has stabilized. Training is terminated and the model parameters are saved. The convergence threshold is a pre-set value used to determine whether the model has converged. The saved model parameters can be used in subsequent scene resource matching tasks, enabling the model to accurately match corresponding scene resource combinations when new scene description text is input.

[0159] Throughout the training process, attention must be paid to protecting data privacy and preventing data leakage. Encryption technology is used to store and transmit scene description text and annotation information in training data to prevent data theft during storage and transmission. During model training, data is anonymized to remove any potentially sensitive information, ensuring data security and privacy. Furthermore, regular security inspections and maintenance are performed on the servers storing data to promptly identify and address potential security vulnerabilities and ensure data security.

[0160] In addition, during the implementation of the entire method for rapid generation of film and television animation scenes, attention should also be paid to the coordination and compatibility between the various steps. For example, when matching scene resources, it is necessary to ensure that the matched resources can be reasonably combined and applied in the subsequent spatiotemporal layout optimization and rendering process. When rendering, the animation scene frame sequence should be accurately generated according to the scene layout parameters and dynamic adjustment parameters, and the quality and smoothness of the frame sequence should be guaranteed. At the same time, when performing audio and video synchronization processing, it is necessary to ensure the time alignment of audio and video to provide the audience with a high-quality film and television animation experience.

[0161] In practical applications, the entire method can be adjusted and optimized based on the needs of different film and television animations. For example, if more complex scenes need to be generated, the level of detail in the scene description text can be increased, providing more information about element-related features, so that the model can more accurately match and generate scenes. If the real-time requirements for animation are high, the rendering algorithm and processing flow can be optimized to increase the speed of generating animation scene frame sequences. In short, through continuous improvement and optimization, this method can better meet the needs of various film and television animation scene generation requirements.

[0162] Figure 2A schematic diagram illustrates exemplary hardware and software components of a system 100 for rapidly generating film and television animation scenes using an AI algorithm, which can implement the concepts of the present invention, according to some embodiments of the present invention. For example, a processor 120 can be used in the system 100 for rapidly generating film and television animation scenes using an AI algorithm to perform the functions of the present invention.

[0163] The system 100 for rapidly generating film and television animation scenes using AI algorithms can be a general-purpose server or a special-purpose server, both of which can be used to implement the method for rapidly generating film and television animation scenes using AI algorithms of the present invention. Although only one server is shown in the present invention, for convenience, the functions described in the present invention can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0164] For example, the system 100 for quickly generating film and television animation scenes in combination with an AI algorithm may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and storage media 140 in different forms, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the system 100 for quickly generating film and television animation scenes in combination with an AI algorithm may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present invention can be implemented according to these program instructions. The system 100 for quickly generating film and television animation scenes in combination with an AI algorithm also includes an I / O interface 150 between the computer and other input and output devices.

[0165] For ease of explanation, only one processor is described in the system 100 for rapid generation of film and television animation scenes in combination with an AI algorithm. However, it should be noted that the system 100 for rapid generation of film and television animation scenes in combination with an AI algorithm in the present invention may also include multiple processors, so the steps performed by one processor described in the present invention may also be performed jointly or individually by multiple processors. For example, if the processor of the system 100 for rapid generation of film and television animation scenes in combination with an AI algorithm executes step A and step B, it should be understood that step A and step B may also be performed jointly by two different processors or individually in one processor. For example, the first processor executes step A, the second processor executes step B, or the first processor and the second processor execute steps A and B together.

[0166] In addition, an embodiment of the present invention also provides a readable storage medium, in which computer-executable instructions are preset. When the processor executes the computer-executable instructions, the method for quickly generating film and television animation scenes combined with the AI ​​algorithm as described above is implemented.

[0167] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, multiple features are sometimes combined into one embodiment, figure or description thereof.

Claims

1. A method for rapidly generating film and television animation scenes in combination with AI algorithms, characterized in that: The method comprises: Receive a scene description text input, perform semantic parsing on the scene description text, and obtain a scene element set and element association features; Calling a pre-trained scene resource matching model to match corresponding scene resource combinations in a preset multimodal resource library according to the scene element set and the element association features; Performing spatiotemporal layout optimization processing on the scene resource combination to generate scene layout parameters and dynamic adjustment parameters; Constructing a three-dimensional space topology structure according to the scene layout parameters and the dynamic adjustment parameters, and rendering and generating a target animation scene frame sequence in the three-dimensional space topology structure; The target animation scene frame sequence is synchronized and calibrated with a preset plot timeline to output a complete animation scene stream.

2. The method for rapidly generating film and television animation scenes in combination with AI algorithms according to claim 1, characterized in that: The semantic parsing of the scene description text to obtain a scene element set and element association features includes: Sentence processing is performed on the scene description text to generate multiple semantic units; Calling a pre-trained semantic encoder to perform context encoding on each semantic unit to generate a unit semantic vector; Performing cluster analysis on the unit semantic vectors to determine core semantic clusters and subsidiary semantic clusters; Extracting main scene elements from the core semantic cluster and extracting auxiliary scene elements from the subsidiary semantic cluster, and merging them to form the scene element set; Construct a spatial dependency relationship and a timing trigger condition between the main scene element and the auxiliary scene element to generate the element association feature; wherein the spatial dependency relationship includes the relative position constraints and physical interaction rules between the elements, and the timing trigger condition includes the time threshold of the element state change and the event response logic.

3. The method for rapidly generating film and television animation scenes in combination with AI algorithms according to claim 2, characterized in that: The calling of the pre-trained semantic encoder to perform context encoding on each semantic unit to generate a unit semantic vector includes: Performing word segmentation processing on the semantic unit to generate a vocabulary sequence; Adding special markers at the beginning and end of the vocabulary sequence to generate an extended vocabulary sequence; Inputting the expanded vocabulary sequence into the pre-trained semantic encoder, and extracting the contextual dependencies between the vocabulary through a multi-layer attention mechanism; Extracting position features corresponding to the special marker in the last hidden state of the semantic encoder, and performing weighted fusion on features output from different layers to generate the unit semantic vector; The unit semantic vectors are subjected to dimensionality reduction processing, and the unit semantic vectors after dimensionality reduction are spliced ​​in the order of the semantic units to generate global semantic context features.

4. The method for rapidly generating film and television animation scenes in combination with an AI algorithm according to claim 2, characterized in that: The calling of the pre-trained scene resource matching model to match the corresponding scene resource combination in a preset multimodal resource library according to the scene element set and the element association features includes: Constructing a resource index structure of the multimodal resource library, wherein the resource index structure includes a static model resource layer, a dynamic behavior resource layer, and an environmental special effect resource layer; Invoking a first matching network of a pre-trained scene resource matching model to match a three-dimensional model resource that matches the geometric forms of the main scene element and the auxiliary scene element from the static model resource layer, and extracting topological features and material features of the three-dimensional model resource; Invoking a second matching network of a pre-trained scene resource matching model to match action clip resources that conform to the physical interaction rules in the element association features from the dynamic behavior resource layer, and extracting motion trajectory features and key frame features of the action clip resources; Invoking a third matching network of a pre-trained scene resource matching model to match lighting special effects resources that match the timing trigger condition from the environmental special effects resource layer, and extracting color distribution characteristics and intensity change characteristics of the lighting special effects resources; Combining and optimizing the three-dimensional model resources, the action clip resources, and the lighting special effect resources according to the spatial dependency relationship in the element association features to generate an initial scene resource combination; Resource conflict detection is performed on the initial scene resource combination. When a resource attribute conflict is detected, resource parameters are adjusted based on the timing trigger conditions in the element association characteristics to generate an optimized scene resource combination.

5. The method for rapidly generating film and television animation scenes in combination with AI algorithms according to claim 4, characterized in that: The performing resource conflict detection on the initial scene resource combination, and when a resource attribute conflict is detected, adjusting resource parameters based on the timing trigger condition in the element association feature to generate the optimized scene resource combination, includes: Detecting a spatial overlap area between a collision volume of the three-dimensional model resource and a motion trajectory of the action clip resource; Convert the collision volume of the three-dimensional model resource and the motion trajectory of the action clip resource into parameters in a relative coordinate system, and calculate the area ratio of the spatial overlapping area; When the area ratio exceeds a preset threshold, it is determined that there is a geometric conflict; Adjusting the motion trajectory of the action clip resource according to priority or scaling the collision volume of the three-dimensional model resource based on relative proportions; detecting compatibility between intensity changes of the lighting effects resource and light reflection properties of the material feature; When overexposure or color distortion is detected, the intensity curve of the lighting effect resource is adjusted in segments based on the time threshold in the timing trigger condition.

6. The method for rapidly generating film and television animation scenes in combination with AI algorithms according to claim 1, characterized in that: The performing spatiotemporal layout optimization processing on the scene resource combination to generate scene layout parameters and dynamic adjustment parameters includes: Analyzing the spatial occupancy attributes and motion attributes of each resource in the scene resource combination to generate an initial spatial distribution matrix; Constructing spatial constraints according to the spatial dependencies in the element association features, and iteratively optimizing the initial spatial distribution matrix based on the spatial constraints; During each iteration, priority weights are generated according to the spatial dependencies in the element association features, and the visual focus weight matrix is ​​dynamically adjusted based on the priority weights. A matching score between the current spatial distribution matrix and the adjusted visual focus weight matrix is ​​calculated, and the position offset and rotation angle of the resource are adjusted in priority order according to the matching score. When the maximum number of iterations is reached or the matching score meets a convergence threshold, locking the final spatial distribution matrix and extracting the scene layout parameters; According to the timing trigger conditions in the element association characteristics, key time nodes are marked in the final spatial distribution matrix, and the dynamic adjustment parameters are generated; wherein, the dynamic adjustment parameters include resource display and hide timing, motion speed curve and special effect intensity gradient function.

7. The method for rapidly generating film and television animation scenes in combination with an AI algorithm according to claim 6, characterized in that: The step of marking key time nodes in the final spatial distribution matrix according to the timing trigger conditions in the element association features and generating the dynamic adjustment parameters includes: Analyze the event response logic in the timing trigger condition and extract the event trigger condition and the corresponding response action; Mapping the event triggering condition to a corresponding time interval in the plot timeline; According to the rendering frame rate of the target animation scene frame sequence, a plurality of sampling time points are set at equal intervals within the corresponding time interval, and a resource state parameter corresponding to each sampling time point is calculated; generating a parameter adjustment curve according to the type of the response action, wherein the parameter adjustment curve includes a linear change curve, an exponential decay curve, and a step transition curve; Associating the parameter adjustment curve with the sampling time point to generate a time-driven parameter control instruction; The time-driven parameter control instruction is embedded in the final spatial distribution matrix to form an enhanced spatial distribution matrix with a timing mark.

8. The method for rapidly generating film and television animation scenes in combination with AI algorithms according to claim 1, characterized in that: The step of constructing a three-dimensional space topology structure according to the scene layout parameters and the dynamic adjustment parameters, and rendering and generating a target animation scene frame sequence in the three-dimensional space topology structure includes: Converting the scene layout parameters into a vertex data set and a material map mapping relationship in a three-dimensional coordinate system; generating a time axis control instruction according to the dynamic adjustment parameter, wherein the time axis control instruction includes a vertex displacement instruction, a material switching instruction, and a light intensity adjustment instruction; Creating a virtual camera perspective in the three-dimensional space topology structure, and setting perspective transformation parameters according to a preset lens motion trajectory; Initialize a scene rendering context based on the vertex data set and the material map mapping relationship; In the scene rendering context, updating vertex positions, material properties, and lighting effects frame by frame according to the timeline control instructions, and adjusting the observation matrix of the virtual camera according to the perspective transformation parameters; The rendering result of each frame is post-processed to generate the target animation scene frame sequence, wherein the post-processing includes anti-aliasing processing, dynamic blur overlay and color correction.

9. The method for rapidly generating film and television animation scenes in combination with an AI algorithm according to claim 8, characterized in that: The post-processing of the rendering result of each frame to generate the target animation scene frame sequence includes: Calculating a pixel displacement vector according to the movement speed and direction of the virtual camera, generating a motion blur effect based on the pixel displacement vector, and after completing the motion blur process, smoothing the edge contour of the rendered frame using multi-sampling anti-aliasing technology; Calculating a pixel displacement vector according to the movement speed and direction of the virtual camera, and generating a motion blur effect based on the pixel displacement vector; Extract the brightness histogram of the current frame and adjust the color saturation and contrast according to the preset tone mapping curve; Detect detail loss in highlight and shadow areas and restore texture information through local contrast enhancement algorithms; Compare the difference between the processed rendered frame and the previous frame to eliminate flicker artifacts between frames; The continuously processed rendering frames are arranged in chronological order to generate a visually coherent target animation scene frame sequence.

10. A film and television animation scene rapid generation system combined with AI algorithm, characterized by: It includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the method for quickly generating film and television animation scenes combined with AI algorithms as described in any one of claims 1 to 9 above.

Images