An intelligent video animation generation system based on AIGC

Abstract

The invention relates to the field of graphic generation technology, and in particular to an AIGC-based video animation intelligent generation system, comprising: a collector for inputting a basic picture and a target picture vocabulary; a target editor for generating a target picture corresponding to the basic picture according to the target vocabulary; a mapping compiler for determining static key points and key mappings between the basic picture and the target picture according to a basic mapping mode, and determining a target moving line of the animation according to the key points and key mappings according to an action mapping mode; an action inserter for generating a plurality of intermediate pictures according to a preset number of frames according to the target moving line, and connecting the basic picture, the plurality of intermediate pictures and the target picture according to the target moving line to form an animation; the invention segments a video, and simulates corresponding actions by using a model, so that the animation produced by using the AIGC can run continuously, and effectively improves the video quality and the continuity of the animation operation.

Description

Technical Field

[0001] The present invention relates to the field of graphics generation technology, and in particular to an AIGC-based video animation intelligent generation system. Background Art

[0002] AIGC technology is widely used in text-to-video and image-to-video technologies, such as Runway, Pika labs, SD +Deforum, etc.; video-to-video technologies, including frame-by-frame generation, keyframe + interpolation, motion capture and video repair technology; and AI Avatar + voice generation technology, such as Synthesia, HeyGen AI, D-ID, etc. These technologies involve a variety of functions such as conversion from text to video, conversion from image to video, and automatic generation of video content, which greatly improves the efficiency of high-quality video production.

[0003] Chinese patent authorization announcement number CN118632049B provides an AIGC skit generation information processing method and system, which generates multiple facial action key frame identifiers through the expected emotional information of the character speech text of the target storyboard and the semantic emotional information of each line in the target storyboard; determines at least two target facial action key frame identifiers according to the user's second selection operation for multiple facial action key frame identifiers, and the video time range corresponding to the at least two target facial action key frame identifiers includes at least the first video time range; adjusts the target storyboard according to the at least two target facial action key frame identifiers to obtain an updated target storyboard. This application realizes the function of presenting the expected emotional information of the character plot in three dimensions when the device processes the short drama video data, which is conducive to improving the precision and accuracy of the device processing the facial images of the characters in the storyboard.

[0004] Chinese patent authorization announcement number CN118051944B provides a method and system for realizing generative content personalization by using AIGC technology, which relates to the field of artificial intelligence technology. The main problems to be solved are low system data quality, inability of the system to maintain long-term stability and low user privacy. The generative content personalization system realized by using AIGC technology includes a data collection module, a data processing module, a content generation module, a feedback module and a system management module. The data type and data encoding are converted by a feature extraction algorithm, thereby improving the efficiency of data use; the recognition accuracy of the AIGC model is trained by an iterative algorithm, thereby improving the long-term stability of the system during use; the user's data information is securely protected by electronic encryption and firewalls, thereby improving the protection of user privacy.

[0005] However, the above method has the following problem: when the generated image changes greatly, the continuity of the video cannot be guaranteed. Summary of the invention

[0006] To this end, the present invention provides a video animation intelligent generation system based on AIGC, which is used to overcome the problem in the prior art that when the generated image and the basic image change greatly, the continuity of the video cannot be guaranteed.

[0007] To achieve the above object, the present invention provides a video animation intelligent generation system based on AIGC, comprising:

[0008] Collector, used to input basic pictures and target picture vocabulary;

[0009] A target editor, connected to the collector, for generating a target image corresponding to the basic image according to the target vocabulary;

[0010] A mapping compiler is provided with a basic mapping mode and an action mapping mode, wherein:

[0011] The basic mapping mode is to determine several static key points and several key mappings according to the basic image and the target image;

[0012] The action mapping mode is to determine the target motion line of the animation according to the key points and the key mapping;

[0013] An action inserter, which is connected to the target editor and the mapping compiler, is used to generate a number of intermediate pictures according to a preset number of frames according to the target action line, and,

[0014] Connecting the basic picture, the plurality of intermediate pictures and the target picture along the target moving line to form an animation;

[0015] The key points include at least a number of points with the same definition in the base image and the target image;

[0016] The key map includes at least the same region defined in the base picture and the target picture;

[0017] The definition corresponds to the target vocabulary, and the region includes at least an animation space and / or an animation object.

[0018] Further, when the target editor generates the target image according to the target vocabulary, the target vocabulary is disassembled to form a plurality of key definitions;

[0019] Mapping the key definitions onto the basic image according to semantics, and generating a plurality of key definition spaces and a plurality of key definition models;

[0020] Solidifying each key definition space into an animation space, and mapping each key definition model onto the animation space according to the target vocabulary to form the target image;

[0021] The key definition model includes at least one point that can be synchronously displayed on the basic image and the target image and is attached to the animation object.

[0022] Furthermore, the target editor expands the key definition space according to the key definition when generating the key definition space, and the size of the animation space composed of the key definition spaces is not less than the size corresponding to the target image.

[0023] Furthermore, when determining the key mapping process, the mapping compiler compares the target image and the base image to obtain the animation amplitude, and sets the action threshold of the animation according to the animation amplitude;

[0024] The target editor is responsive to the key definition model exceeding the action threshold being set as a key mapping model, and determining a key mapping of the animation according to each key mapping model;

[0025] The animation amplitude is the overlap ratio of the key definition model of the target image and the basic image on the projection of the corresponding surface of the image;

[0026] The action threshold is related to the magnitude of change of the key definition model, and for a single animation, a single action threshold is set.

[0027] Furthermore, when determining the key points, the mapping compiler divides each key definition model into a plurality of sub-areas according to the key mapping, and sorts the sub-areas exceeding the action threshold from large to small;

[0028] Determine the largest sub-regions in the sorting, and use the geometric center of each sub-region as the key point.

[0029] Furthermore, when determining the target moving line of the animation, the action inserter connects the corresponding key points of the base image and the target image using a smooth curve, determines the key mapping corresponding to each key point, and determines the corresponding position of the key point on the corresponding key definition model;

[0030] The motion inserter defines the motion trajectory of the model according to the key setting of each smooth curve, and

[0031] The motion trajectories are combined according to the key definition to form the target motion line.

[0032] Furthermore, when setting the intermediate picture, the action inserter continuously shoots a plurality of frames of pictures according to the target moving line as the intermediate pictures;

[0033] When acquiring the intermediate picture, each frame of the picture is inserted into the basic picture and the target picture in chronological order.

[0034] Further, when generating the key definition space, the mapping compiler sets a corresponding animation space according to the base image and the target image, and fills the animation space according to the key definition;

[0035] The animation space at least includes display areas of the basic image and the target image.

[0036] Furthermore, when generating the animation space, the mapping compiler sets the action threshold to a plurality of key definition models that do not change as the animation space.

[0037] Further, the animation space generated by the mapping compiler does not move within the animation space;

[0038] The animation space has a mapping corresponding to the target picture vocabulary in the base picture or the target picture.

[0039] Furthermore, the method of generating animation using the above system includes:

[0040] Input the basic picture and target picture vocabulary;

[0041] Generate a target image corresponding to the basic image based on the target vocabulary;

[0042] Determining key points and key mappings of the base image and the target image;

[0043] Determine a target moving line of the animation according to the key points and the key mapping;

[0044] Generate a number of intermediate images according to the target moving line and the preset number of frames;

[0045] Connecting the basic picture, the plurality of intermediate pictures and the target picture along the target moving line to form an animation;

[0046] Compared with the prior art, the beneficial effect of the present invention lies in that, by means of temporary modeling, the video is segmented and the corresponding actions are simulated using the model, so that the animation produced using AIGC can run continuously, which effectively improves the video quality and the continuity of the animation operation.

[0047] Furthermore, by inputting vocabulary and defining the model and animation space by key, the models in the picture are constructed, and the animation space and the model that can move are determined according to the constructed models, so that the corresponding animation process can be inserted by recording the model movement, making the whole animation process more reasonable and continuous.

[0048] Furthermore, by determining the action threshold of the animation, the animation space of the animation and the model that can move are separated, which effectively reduces the amount of calculation for animation generation, effectively improves the coherence of the model action, and maintains the consistency of the animation space, thereby effectively improving the coherence of animation generation.

[0049] Furthermore, by dividing the animation space and generating a number of corresponding key points, the key points in each area are mapped to each other. When generating the animation, the spatial relationship of the key points can influence each other, thereby enhancing the integrity of the model. While effectively avoiding the problem of animation instability caused by the scattered model, the consistency of the animation image is improved, thereby further effectively improving the coherence of the animation generation.

[0050] Furthermore, by generating corresponding animations by shooting model movements, each frame of the captured image is inserted into the corresponding animation, which effectively avoids the problem of the image not being effectively embedded due to differences in the models of each frame during the animation running process, thereby further effectively improving the coherence of the animation generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a connection diagram of the video animation intelligent generation system based on AIGC of the present invention;

[0052] Figure 2 A schematic diagram of animation generation of the video animation intelligent system of AIGC according to an embodiment of the present invention;

[0053] Figure 3 A basic picture schematic diagram of an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of a target image according to an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the relationship between the animation space and the moving lines of an embodiment of the present invention;

[0056] Among them: 1, the first definition point; 2, the second definition point; 3, the key point; 4, the motion trajectory; 41, the first motion trajectory; 42, the second motion trajectory; 5, the animation space. DETAILED DESCRIPTION

[0057] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0059] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0060] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] To facilitate reading of this plan, some of the terms involved in this plan are explained below:

[0062] AIGC (Artificial Intelligence Generated Content): AI-generated content refers to the use of artificial intelligence technology, especially methods based on generative adversarial networks and large pre-trained models, to generate relevant content with appropriate generalization capabilities by learning and identifying existing data. The core idea of ​​AIGC technology is to use artificial intelligence algorithms to generate content with certain creativity and quality. Through training models and learning from a large amount of data, AIGC can generate relevant content based on the input conditions or guidance. For example, by entering keywords, descriptions or samples, AIGC can generate articles, images, audio, etc. that match them.

[0063] There are mainly the following types of collectors:

[0064] Hard compression and soft compression video capture cards: Hard compression cards use built-in hardware for video encoding, providing high-speed processing and high image quality, suitable for applications with high requirements for real-time and stability; while soft compression cards rely on the computer's processor and are suitable for scenarios with high requirements for cost and compatibility.

[0065] Video capture cards with USB, HDMI, VGA and other interface types: Based on the function and interface type, video capture cards can be divided into USB, HDMI, VGA and other forms to meet the connection requirements between different devices.

[0066] Built-in video capture card: This capture card is installed on the computer motherboard and is usually connected using a PCI or PCI-E interface. It has high read and write speeds and stability and is suitable for applications that require accurate image and color performance.

[0067] External video capture card: an independent device connected to the computer via USB or other high-speed interface, with good compatibility and flexibility, and can support video input and output of various types and formats.

[0068] Model Mapping: usually refers to establishing a relationship between different models so that the data, structure or behavior of one model can correspond to or transform with another model. This mapping can be simple or complex, depending on the type of model and application scenario. The mapping in this solution is a composite mapping of geometric model mapping and machine learning mapping:

[0069] Among them, geometric model mapping may involve converting one geometric model into another geometric model to adapt to different rendering engines or design requirements.

[0070] In the field of machine learning, model mapping may involve converting the parameters or structure of one learned model to another model to facilitate transfer learning or model compression.

[0071] See also Figure 1 As shown, it is a connection diagram of the video animation intelligent generation system based on AIGC of the present invention, including:

[0072] Collector, used to input basic pictures and target picture vocabulary;

[0073] A target editor, which is connected to the collector and is used to generate a target image corresponding to the basic image according to the target vocabulary;

[0074] A mapping compiler is provided with a basic mapping mode and an action mapping mode, wherein:

[0075] The basic mapping mode is to determine several static key points and several key mappings according to the basic image and the target image;

[0076] The action mapping mode determines the target movement line of the animation based on key points and key mapping;

[0077] An action inserter, which is connected to the target editor and the mapping compiler, is used to generate a number of intermediate pictures according to the target motion line at a preset number of frames, and,

[0078] Connect the basic image, several intermediate images and the target image along the target moving line to form an animation;

[0079] The key points include at least a number of points with the same definition in the base image and the target image;

[0080] The key map includes at least the same defined areas in the base image and the target image;

[0081] The definition corresponds to the target vocabulary, and the region includes at least an animation space and / or an animation object.

[0082] The method of generating animation using the above system includes:

[0083] Step S1, input basic image and target image vocabulary;

[0084] Step S2, generating a target image corresponding to the basic image according to the target vocabulary;

[0085] Step S3, determining key points and key mappings of the base image and the target image;

[0086] Step S4, determining the target moving line of the animation according to the key points and the key mapping;

[0087] Step S5, generating a number of intermediate images according to a preset number of frames based on the target moving line;

[0088] Step S6, connecting the basic picture, the plurality of intermediate pictures and the target picture according to the target moving line to form an animation;

[0089] Through temporary modeling, the video is generated in segments, and the model is used to simulate the corresponding actions, so that the animation produced by AIGC can run continuously, which effectively improves the video quality and the continuity of the animation operation.

[0090] Specifically, when the target editor generates a target image based on the target vocabulary, it disassembles the target vocabulary to form several key definitions;

[0091] Map the key definitions onto the basic image according to semantics, and generate several key definition spaces and several key definition models;

[0092] Solidify each key definition space into an animation space, and map each key definition model onto the animation space according to the target vocabulary to form a target picture;

[0093] The key definition model includes at least one point that can be synchronously displayed on the base image and the target image and is attached to the animation object.

[0094] In particular, the process of generating the target image according to the target vocabulary includes:

[0095] Step S21, disassembling the target vocabulary to form a number of key definitions;

[0096] Step S22, mapping the key definition on the basic image according to the semantics, and generating a plurality of key definition spaces and a plurality of key definition models;

[0097] Step S23, solidifying each key definition space into an animation space, and mapping each key definition model onto the animation space according to the target vocabulary to form the target image;

[0098] In implementation, in order to ensure accurate identification of key points in complex animation spaces, the following strategies can be adopted, such as using advanced network structures such as Cascaded Pyramid Network (CPN) to improve detection capabilities in complex postures and occlusion situations through top-down detection strategies. Secondly, combine edge computing and cloud computing to perform multi-source information data fusion to reduce network latency and improve data processing efficiency. In addition, improve the robustness of the algorithm so that it can accurately detect key points in situations such as occlusion, posture changes, and lighting changes. At the same time, for multi-person scenes and occlusion problems, train the model to handle large-scale data sets and a large number of occlusion examples. Finally, combine target detection with key point prediction and use mature target detection models for optimization.

[0099] The above methods can improve the accuracy and robustness of the model in identifying key points in complex animation space.

[0100] See also Figure 2 As shown, it is an animation generation schematic diagram of the video animation intelligent system of AIGC according to an embodiment of the present invention, wherein the target editor expands the key definition space according to the key definition when generating the key definition space, and the size of the animation space composed of each key definition space is not less than the size corresponding to the target image.

[0101] By inputting vocabulary and defining the model and animation space, the models in the picture are constructed, and the animation space and the model that can move are determined according to the constructed model. The corresponding animation process can be inserted by recording the model movement, making the entire animation process more reasonable and continuous.

[0102] When expanding the key definition space, the options include:

[0103] Menu bar extensions: Custom buttons and menu items can be added to the menu bar through Unity editor scripts. This can be achieved by using the [MenuItem] property, allowing developers to add new options to Unity's menu bar that can trigger specific functions, thereby expanding the key definition space.

[0104] Component menu extension: Using the [AddComponentMenu] attribute, custom scripts can be added to Unity's component menu, allowing users to select these custom scripts when adding components, thereby expanding the key definition space.

[0105] Context menu extension: Through the [ContextMenu] property, you can add right-click menu options to components in the Unity editor to provide quick operations. This is also a way to expand the key definition space.

[0106] Property context menu options expanded: Using the [ContextMenuItem] attribute, you can add context menu options on specific properties in the Inspector panel, further expanding the key definition space.

[0107] Custom editor window: By inheriting the EditorWindow class, you can create a customized editor window and add more custom methods and interface elements to expand the key definition space.

[0108] EditorGUILayout class: Unity provides the EditorGUILayout class, which is similar to GUILayout, but adds automatic layout capabilities, allowing for more flexible display and editing of key definition spaces in the Inspector panel.

[0109] ReorderableList: This is a class inside the Unity editor that enables dragging and sorting of list elements, making the display of arrays or List collections in the Inspector panel more flexible, and can also be used to expand the key definition space.

[0110] It is understandable that the above functions can all be adjusted using a large model through AIGC.

[0111] Specifically, when determining the key mapping process, the mapping compiler compares the target image and the base image to obtain the animation amplitude, and sets the motion threshold of the animation according to the animation amplitude;

[0112] The target editor responds to the key definition model exceeding the action threshold being set as a key mapping model, and determines the key mapping of the animation according to each key mapping model;

[0113] The animation amplitude is the overlap ratio of the key definition model of the target image and the base image on the projection of the corresponding surface of the image;

[0114] The motion threshold is related to the magnitude of change of the key definition model, and for a single animation, it sets a single motion threshold.

[0115] By determining the action threshold of the animation, the animation space of the animation and the model that can move are separated. While effectively reducing the amount of calculation for animation generation, the coherence of the model action is effectively improved and the consistency of the animation space is maintained, thereby effectively improving the coherence of animation generation.

[0116] In implementation, animation amplitude refers to the degree of difference between the target image (usually a frame in the animation) and the base image (the starting or reference frame of the animation);

[0117] This difference can be quantified by projecting the key definition model (keypoint model) onto the corresponding surface of the image.

[0118] In particular, calculating the overlap ratio between the target image and the base image on the projection surface can reflect the amplitude of the animation, that is, the degree to which the object or character in the animation moves or deforms from the initial position.

[0119] If the overlap ratio is high, the animation amplitude is small; if the overlap ratio is low, the animation amplitude is large.

[0120] Motion threshold refers to the threshold used in animation or motion capture to determine when a change in movement or posture is significant enough to be recognized and recorded.

[0121] The action threshold is related to the magnitude of change in the key definition model, that is, when the change in the model exceeds the action threshold, the system will consider that a new action or posture has occurred.

[0122] In implementation, for a single animation, a single motion threshold may be set to determine a key frame or turning point in the animation.

[0123] It is understandable that the setting of the motion threshold is crucial to the smoothness and naturalness of the animation. A threshold that is too low may cause the animation to be too trivial, while a threshold that is too high may miss some important motion details.

[0124] Usually, the threshold is set to be above 2%. In practice, the threshold can be adjusted according to the specific data volume of the target image vocabulary, such as:

[0125] By semantically identifying the target image vocabulary, the target image described has little change from the base image, and the action threshold can be set to 3%;

[0126] The target image described has a large change from the base image, so the action threshold can be set to 5%;

[0127] The target image described does not have the same features as the base image, so the action threshold is not set and the user is prompted to re-enter the target image vocabulary;

[0128] In implementation, the above ratio corresponds to the degree of model change, which can be measured by simply superimposing the target image and the basic image. There are many specific measurement methods, which will not be described here.

[0129] Specifically, when determining the key points, the mapping compiler divides each key definition model into several sub-regions according to the key mapping, and sorts the sub-regions that exceed the action threshold from large to small;

[0130] Determine the largest sub-regions in the sorting, and use the geometric center of each sub-region as the key point.

[0131] In implementation, the process of determining the key points includes:

[0132] Step S41, dividing each key definition model into a plurality of sub-areas according to the key mapping;

[0133] Step S42, sorting the sub-regions exceeding the action threshold from large to small;

[0134] Step S43, determining a number of largest sub-regions in the sorting, and taking the geometric center of each sub-region as a key point.

[0135] Specifically, when determining the target motion line of the animation, the action inserter connects the corresponding key points of the base image and the target image using a smooth curve, determines the key mapping corresponding to each key point, and determines the corresponding position of the key point on the corresponding key definition model;

[0136] The motion inserter defines the model's motion trajectory based on the key settings of each smooth curve, and

[0137] Combine the motion trajectories according to the key definitions to form the target motion line.

[0138] The motion interpolator first connects the corresponding key points of the base image and the target image using smooth curves (such as Bezier curves, spline curves, etc.) These key points are previously determined in the base image and the target image, such as the geometric center and two endpoints of the football.

[0139] For each pair of corresponding keypoints, the action inserter determines their corresponding positions on the key definition model. The key map includes at least the same regions defined in the base image and the target image, which can be an animation space or an animation object.

[0140] In implementation, the animation space is the part of the animation where no movement occurs, and the animation object is the movable part of the animation.

[0141] The Motion Inserter sets the keys to define the model's motion trajectory based on these smooth curves. This means that for each key point a path is defined from a base position to a target position.

[0142] The motion inserter combines the motion trajectories of all key points to form a complete target motion line. This target motion line describes the complete motion process of the animation object from the initial state to the final state.

[0143] Based on the target motion line, the motion inserter generates in-between frames. These frames are created by inserting to the positions of the key points along the motion trajectory on the key definition model. This ensures smoothness and continuity of the animation.

[0144] Finally, the action inserter synthesizes the base image, intermediate frames, and target image in chronological order to form the final animation.

[0145] By dividing the animation space and generating a number of corresponding key points, the key points in each area are mapped to each other. When generating the animation, the spatial relationship of the key points can influence each other, thereby enhancing the integrity of the model. While effectively avoiding the problem of animation instability caused by the scattered model, the consistency of the animation image is improved, thereby further effectively improving the coherence of the animation generation.

[0146] Specifically, when setting the intermediate picture, the action inserter continuously takes several frames of pictures according to the target movement line as the intermediate pictures;

[0147] When obtaining the intermediate image, each frame of the image is inserted into the base image and the target image in chronological order.

[0148] It is understandable that the action inserter first needs to determine the target path, which is the path that the object (such as a football) in the base image moves to the target image position. This path is provided by the mapping compiler and is determined based on key points and key mappings.

[0149] In implementation, the motion inserter generates intermediate images according to the target motion line and a preset frame rate (eg, 30 frames per second), and the intermediate images can show the continuous position change of the object between the base image and the target image.

[0150] The action inserter inserts these intermediate frames between the base image and the target image in time order, where the first frame is the base image, the last frame is the target image, and the intermediate frames are generated intermediate images;

[0151] Taking 30 frames per second as an example, there should be 28 intermediate pictures to form 28 frames.

[0152] In particular, this process requires ensuring that each frame correctly corresponds to its position on the animation timeline to maintain animation coherence and temporal consistency.

[0153] Once all the frames (base, intermediate, and target) are in the correct order, the Motion Inserter combines them into a complete animation. The Motion Inserter outputs the final animation, which can be a video file, animation sequence frame, or other format, depending on the desired output and application scenario.

[0154] Specifically, when generating the key definition space, the mapping compiler sets the corresponding animation space according to the base image and the target image, and fills the animation space according to the key definition;

[0155] The animation space at least includes the display areas of the basic image and the target image.

[0156] The mapping compiler plays the role of converting the base image and the target image into a format that can be used for animation production in the process of generating the key definition space. The following are the detailed steps of how the mapping compiler works:

[0157] The mapping compiler first sets the corresponding animation space according to the base image and the target image. This animation space is a virtual container that defines the display area of ​​the animation, including at least the display area of ​​the base image and the target image.

[0158] The animation space is a three-dimensional space, and the specific size depends on the complexity of the animation and the desired visual effect.

[0159] The map compiler fills the animation space based on key definitions, such as key points and key maps. Key definitions are elements of the animation that require special attention and control, and they define the shape, position, and motion of the animated objects.

[0160] The key definition includes at least the same defined areas in the base image and the target image, which can be specific points, lines or surfaces in the animation space.

[0161] The mapping compiler maps the key definitions into the animation space and fills the corresponding areas of the animation space. This process may involve image processing techniques such as image segmentation, feature point detection, and image registration.

[0162] The purpose of filling the animation space is to create a complete, continuous animation path in the animation space, which describes the complete movement of the animation object from the base image to the target image.

[0163] The mapping compiler generates a key definition model, which contains all key points and key mapping information. The key definition model is the basis of animation production, which defines the shape, position and motion trajectory of the animation object.

[0164] The key definition model can be either a vector model or a pixel-based model, depending on the complexity of the animation and the desired visual effect.

[0165] Finally, the mapping compiler outputs the key definition space, which can be used directly for animation production. The output key definition space can be a file or a model that can be used directly in animation software.

[0166] In particular, for the set animation space, it can define the physical boundaries and working area of ​​the animation, ensuring that all animation elements are laid out and moved within this area. This helps animators organize and manage the various elements in the animation.

[0167] Animation space provides a frame of reference in which animators can accurately position and adjust the position, size, and angle of objects. This is essential for maintaining coherence and consistency in animation.

[0168] In keyframe animation, the animation space allows animators to set keyframes that define the position and state of an object at a specific point in time. The animation software will automatically generate intermediate frames (tween animation) between these keyframes.

[0169] In complex animation projects, there may be multiple objects and characters that need to be animated simultaneously. Animation Spaces ensure that the movement and interaction between these elements are coordinated and consistent.

[0170] Animation spaces can simulate different environments and conditions, such as lighting, shadows, and reflections, which helps enhance the visual impact and realism of animations.

[0171] By managing and optimizing animated elements in the animation space, you can improve the performance of animation rendering and playback, especially when working with high-resolution or high-frame-rate animations.

[0172] In some interactive animation or gaming projects, the animation space can respond to user input, allowing the animation to dynamically adjust based on the user's actions.

[0173] Animation Space facilitates post-production such as adding special effects, adjusting colors, and editing, because it provides a common working platform for these post-processing.

[0174] Simulating physical behavior:

[0175] Animation spaces can simulate physical behaviors such as gravity, collisions, and elasticity, which are important for creating realistic animation effects.

[0176] By generating corresponding animations by shooting model movements, each frame of the picture is inserted into the corresponding animation. This effectively avoids the problem of the picture not being effectively embedded due to differences in the models of each frame during the animation running process, thereby further effectively improving the coherence of the animation generation.

[0177] Specifically, when generating the animation space, the mapping compiler sets several key definition models whose action thresholds are unchanged as the animation space.

[0178] Specifically, the animation space generated by the mapping compiler does not move within the animation space;

[0179] The animation space is mapped in the base image or the target image.

[0180] See also Figure 3 As shown, it is a basic picture schematic diagram of an embodiment of the present invention. In the figure, a rugby ball can be seen in a picture with a window, wherein the rugby ball can be decomposed, and the geometric center of the rugby ball is set as the key point 3, and its two endpoints are respectively the first definition point 1 and the second definition point 2;

[0181] Please cooperate Figure 3 See also Figure 4 As shown, it is a schematic diagram of the target image of an embodiment of the present invention. In this figure, if an input is generated, an animation of the movement of a football toward a corner is generated. At this time, the first definition point 1 and the second definition point 2 of the football are displayed in the target image, and the corresponding key point 3 is located in the air near the corner.

[0182] See also Figure 5 As shown, it is a schematic diagram of the animation space and movement line relationship of an embodiment of the present invention. In the figure, an animation space 5 is first generated, the first definition point 1 corresponds to the first motion trajectory 41, the second definition point 2 corresponds to the second motion trajectory 42, and the key point 3 corresponds to the motion trajectory 4. The motion path is shot into a video corresponding to the basic image and the target image, and the content can be inserted to form the corresponding AIGC animation.

[0183] By following the above steps, you can create an AIGC (Artificial Intelligence Generated Content) animation where the football moves from the position in the base image to the position in the target image.

[0184] The above animation is produced by adding Figure 3 ) and the target image ( Figure 4 ) is achieved by inserting a series of intermediate images between Figure 5 The modeling shown is based on Figure 5 The continuous position change along the predefined motion trajectory is shown.

[0185] Finally, the above pictures are connected end to end according to the time sequence to form a coherent animation, showing the process of the football moving to the corner of the wall.

[0186] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0187] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video animation intelligent generation system based on AIGC, characterized in that: include: Collector, used to input basic pictures and target picture vocabulary; A target editor, connected to the collector, for generating a target image corresponding to the basic image according to the target vocabulary; A mapping compiler is provided with a basic mapping mode and an action mapping mode, wherein: The basic mapping mode is to determine several static key points and several key mappings according to the basic image and the target image; The action mapping mode is to determine the target motion line of the animation according to the key points and the key mapping; An action inserter, connected to the target editor and the mapping compiler, is used to set a number of intermediate pictures according to a preset number of frames based on the target action line, and, Connecting the basic picture, the plurality of intermediate pictures and the target picture along the target moving line to form an animation; The key points include at least a number of points with the same definition in the base image and the target image; The key map includes at least the same region defined in the base picture and the target picture; The definition corresponds to the target vocabulary, and the region includes at least an animation space and / or an animation entity; When the target editor generates the target image according to the target vocabulary, the target vocabulary is disassembled to form a plurality of key definitions; Mapping the key definitions onto the basic image according to semantics, and generating a plurality of key definition spaces and a plurality of key definition models; Solidifying each key definition space into an animation space, and mapping each key definition model onto the animation space according to the target vocabulary to form the target image; The key definition model at least includes at least one point that can be synchronously displayed on the base image and the target image, and is attached to the animation entity; When determining the key mapping process, the mapping compiler compares the target image and the base image to obtain an animation amplitude, and sets an action threshold of the animation according to the animation amplitude; The target editor responds to the key definition model exceeding the action threshold being set as a key mapping model, and determines the key mapping of the animation according to each key mapping model; The animation amplitude is the overlap ratio of the key definition model of the target image and the basic image on the projection of the corresponding surface of the image; The action threshold is related to the change amplitude of the key definition model, and for a single animation, a single action threshold is set; When determining the target moving line of the animation, the action inserter connects the corresponding key points of the base image and the target image using a smooth curve, determines the key mapping corresponding to each key point, and determines the corresponding position of the key point on the corresponding key definition model; The motion inserter defines the motion trajectory of the model according to the key setting of each smooth curve, and The motion trajectories are combined according to the key definition to form the target motion line.

2. The AIGC-based video animation intelligent generation system according to claim 1 is characterized in that: When generating the key definition space, the target editor expands the key definition space according to the key definition, and the size of the animation space composed of the key definition spaces is not less than the size corresponding to the target image.

3. The AIGC-based video animation intelligent generation system according to claim 1 is characterized in that: When determining the key points, the mapping compiler divides each key definition model into a plurality of sub-areas according to the key mapping, and sorts the sub-areas exceeding the action threshold from large to small; Determine the largest sub-regions in the sorting, and use the geometric center of each sub-region as the key point.

4. The AIGC-based video animation intelligent generation system according to any one of claims 2 or 3, characterized in that: When setting the intermediate picture, the action inserter continuously shoots a number of frames of pictures according to the target moving line as the intermediate pictures; When acquiring the intermediate picture, each frame of the picture is inserted into the basic picture and the target picture in chronological order.

5. The AIGC-based video animation intelligent generation system according to claim 1 is characterized in that: When generating the key definition space, the mapping compiler sets a corresponding animation space according to the base image and the target image, and fills the animation space according to the key definition; The animation space at least includes display areas of the basic image and the target image.

6. The AIGC-based video animation intelligent generation system according to claim 5 is characterized in that: The animation space generated by the mapping compiler does not move within the animation space; The animation space has a mapping corresponding to the target picture vocabulary in the base picture or the target picture.

Images