Automatic animation of visual content

Abstract

Various disclosed embodiments relate to automatic animation of visual content. In particular, some embodiments first receive a design document. Some embodiments then generate a rendered image of the design document. Some embodiments then generate a mask indicating one or more regions of visual importance in the image. Some embodiments further generate a scene graph from the design document. Some embodiments detect one or more dominant angle elements based at least in part on the mask, filtering of the scene graph, clustering, and / or one or more dominant angle element rules. Some embodiments additionally or alternatively determine one or more animation rules based on significance data and / or protagonist element detection in order to generate an animation sequence or output.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 699,643, filed September 26, 2024, entitled “AUTO-ANIMATE EXPRESS DESIGNDOCUMENTS USING SALIENCY DETECTION AND HEURISTIC PROGRAMMING,” and U.S. Patent Application No. 19 / 004,073, filed December 27, 2024, the entire contents of which are incorporated herein by reference. Background Technology

[0002] Existing media design technologies allow users to create a variety of visual content, including digital posters, digital greeting cards, slideshows, infographics, and more. These tools offer useful features for creating static designs using a range of templates, design elements, and customization options, making it easier for users to produce visually appealing graphics. However, significant technical challenges arise when transforming these static designs into engaging animations. Animating static content typically requires manual adjustments, in-depth knowledge of animation techniques, substantial time investment, and excessive computational resource consumption, among other challenges. Summary of the Invention

[0003] One or more embodiments relate to the automatic animation of visual content. Specifically, for example, some embodiments first receive a design document (e.g., a digital manual, digital poster, web mockup, or slideshow presentation). Then, some embodiments generate a rendered image of the design document. For example, some embodiments combine all visual elements and layers from the design document into a single flat image that shows the final look indicated in the design document. Then, some embodiments generate a mask that indicates one or more visually significant regions in the image. For example, some embodiments generate a saliency mask by feeding a representation of the rendered image as input to a saliency model, wherein the saliency mask indicates one or more regions that are likely to attract human attention.

[0004] Some embodiments further generate a scene graph from the design document. The scene graph represents each element in the design document in a hierarchical structure, where each node in the scene graph corresponds to an element or group of elements in the design document. For example, the scene graph may organize elements in a tree-like hierarchy, where each node represents an element (or object) in the scene. Some embodiments detect one or more protagonist elements based at least in part on masks, scene graph filtering, clustering, and / or one or more protagonist element rules. Protagonist elements are key or visually prominent components within the design document or scene that are useful for the overall message, user interaction, or visual hierarchy. Protagonist elements typically attract the most attention and are emphasized in animations or visual processing due to their importance. These elements may include a main title, a main image, and / or a key call to action that is critical to the design purpose. For example, some embodiments overlay saliency data from a saliency mask onto the scene graph to determine which specific elements correspond to highly saliency regions identified in the rendered image.

[0005] Some embodiments additionally or alternatively determine one or more animation rules based on saliency data and / or protagonist element detection. For example, a saliency map might show high saliency for title text and product images, medium saliency for call-to-action buttons, and low saliency for decorative snowflakes. Based on saliency data and / or elements detected as protagonist elements, various embodiments use heuristic animation rules to determine how to animate each element. For example, a rule might be “Apply dynamic animation to high-saliency elements to immediately capture the viewer’s attention.” Since the title text and product images are high-saliency, this rule specifies the use of dynamic animation for these elements. Animation is the process of creating the illusion of motion or movement and change (e.g., by continuously displaying a series of still images or frames). In the context of digital design, animation involves applying movement, transitions, or effects to visual elements (such as text, images, or shapes) to make them appear dynamic or interactive. For example, when a sequence of frames showing a ball at different locations is played rapidly in succession, the image of the ball appearing to bounce up and down is animation. An animation sequence is a specific series of frames or steps that defines the movement or change of visual elements within the animation over time.

[0006] The various embodiments disclosed herein offer a variety of technical effects and improvements over existing media design techniques. For example, as described in more detail herein, some technical effects include improved user interfaces, improved user experiences (e.g., by eliminating the need for in-depth animation knowledge), and reduced computational resource consumption (e.g., reduced computer input / output (I / O), reduced memory consumption, etc.). Attached Figure Description

[0007] The patent or application documents contain at least one drawing executed in color. Copies of this patent or patent application publication having multiple color drawings will be provided by the Patent Office upon request and payment of the necessary fees. The invention is described in detail below with reference to the appended drawings, wherein:

[0008] Figure 1 This is a block diagram of an example computing system architecture suitable for implementing some embodiments of the present disclosure;

[0009] Figure 2 The illustration shows an example pipeline for generating output animations according to some embodiments;

[0010] Figure 3 The illustration shows an example pipeline of a saliency model according to some embodiments;

[0011] Figure 4A These are screenshots of user interface pages according to some embodiments, illustrating a static design document prior to requesting the generation of an animation;

[0012] Figure 4B According to some embodiments Figure 4A A screenshot of the user interface page, illustrating the initial steps of a "waterfall" animation style in response to a user request to generate the corresponding animation sequence;

[0013] Figure 4C According to some embodiments Figure 4A A screenshot of the user interface page, illustrating... Figure 4B The intermediate step in the "waterfall" animation style sequence initiated in the middle;

[0014] Figure 4D According to some embodiments Figure 4A A screenshot of the user interface page, illustrating the transition from... Figure 4C The last time step of the continuing "waterfall" animation style sequence;

[0015] Figure 5 This is a flowchart of an example process for filtering scene graphs according to some embodiments;

[0016] Figure 6 This is a flowchart of an example process for detecting one or more sets of protagonist elements according to some embodiments;

[0017] Figure 7 This is a flowchart of an example process for training a machine learning model to generate a saliency mask, according to some embodiments;

[0018] Figure 8 This is a flowchart of an example process for generating an animation sequence according to some embodiments;

[0019] Figure 9 These are example computer environments in which various aspects of this disclosure are employed according to some embodiments; and

[0020] Figure 10 This is a block diagram of a computing device incorporating various aspects of the present disclosure according to some embodiments. Detailed Implementation Overview

[0021] As described above, animate content, especially static content, presents technical challenges due to the often manual adjustments required, in-depth knowledge of animation techniques, significant time investment, and excessive computational resource consumption. For example, regarding manual adjustments, excessive tweaking is typically required at the user interface level. When animate static content, users often need to perform numerous manual adjustments, such as setting keyframes, adjusting timings, or modifying the properties of individual elements (e.g., position, opacity, rotation). Each of these actions usually requires navigating several menus, panels, or layers within the software, which can be cumbersome and time-consuming. For instance, a user might have to select an object, then delve into its properties to adjust the animation timing, and repeat this process for every element on the page, resulting in a repetitive and labor-intensive workflow.

[0022] Furthermore, these techniques require a deep understanding of animation techniques. Animation techniques involve principles such as timing, easing, motion paths, and layering, which specify how objects move, change, or interact over time. To create smooth and visually appealing animations, users typically must understand these principles and how to apply them effectively. This often requires expertise beyond basic design skills. For example, users need to know how to set appropriate keyframes to control the start and end points of the animation and how to use easing features to create natural motion (e.g., objects smoothly accelerating or decelerating). Without this knowledge, animations may appear uneven, unprofessional, or fail to convey the intended message.

[0023] Even with software that provides animation capabilities, manually animate each element can be a slow and labor-intensive process, especially for complex designs with multiple animated components. Creating animations often involves a great deal of detailed work, including designing motion paths, setting keyframes, and fine-tuning the animations to ensure they look correct. For example, a user animates a slideshow with multiple text boxes and images might spend hours setting up individual animations for each element, adjusting timings to synchronize the animations, and repeatedly previewing the results to ensure everything looks harmonious—a process far too tedious for the user.

[0024] Existing technologies are also associated with increased consumption of computing resources, such as increased input / output (I / O). I / O operations refer to reading data from and writing data to storage devices, or rendering animations on the screen. Excessive I / O operations can cause wear and tear on storage devices (e.g., disks) due to excessive mechanical movement. As described above, users typically manually navigate through various panels, layers, and properties in the user interface to set up animations. Each click, drag, or adjustment requires the software to read data (input) from the storage device and update the interface (output). For example, when a user adjusts the position of an object frame by frame, the software must load the data for the current frame, apply the changes, and then save or render the new state. This repetitive cycle of loading, modifying, and saving increases I / O operations, causing unnecessary wear and tear on storage components. Furthermore, every time a user modifies parameters (such as position, scale, rotation, or opacity), the software must read the current state of the element, process the changes, and write the updated state back to memory or disk. This is especially true for applications that frequently save changes to prevent data loss. For example, if a user is animates a complex scene with multiple keyframes, each adjustment (whether adding, deleting, or modifying a keyframe) will result in multiple excessive read / write operations. The system reads the current data, updates it with the new keyframe information, and writes it back to storage.

[0025] Existing technologies are also associated with increased memory consumption. When users interact with complex animation software interfaces, especially by digging into multiple layers and features, the software needs to load all relevant data for each element. This includes metadata, visual characteristics, animation states, etc. All of this information is typically stored in RAM for quick access, thus increasing memory consumption. Many animation tools maintain a history of user actions to support undo / redo functionality. Every user input that changes the project state is logged, which requires memory. As users make numerous adjustments (over-drilling), the memory required to store these states increases, especially for complex projects with multiple layers and elements.

[0026] As described herein, embodiments of this disclosure provide one or more technical solutions to one or more of these technical problems. Various aspects relate to the automatic animation of images. Specifically, for example, some embodiments first receive a design document. A design document is a file created in graphic design or layout software. Digital design documents contain structured visual elements and are used to create various visual content. For example, design documents may include digital manuals, digital posters, digital business cards, web mockups, social media graphics, infographics, slideshow presentations, UI designs, digital logos, brand guides, or digital magazine layouts.

[0027] Then, some embodiments generate a rendered image of the design document. For example, a painter's algorithm can be used to combine all visual elements and layers from the design document into a single flat image that shows the final look indicated in the design document. Then, some embodiments generate masks that indicate one or more visually significant regions in the image. For example, some embodiments generate saliency masks by feeding a representation of the rendered image as input to a saliency model, wherein the saliency mask indicates one or more regions that are likely to attract human attention.

[0028] Some embodiments further generate scene diagrams from the design document. The scene diagram represents each element in the design document in a hierarchical structure, where each node in the scene diagram corresponds to an element or group of elements in the design document. For example, the scene diagram may organize elements in a tree-like hierarchy, where each node represents an element (or object) in the scene. Nodes may have parent-child relationships, indicating how elements are grouped and positioned relative to each other. Some embodiments filter the scene diagram by selecting or discarding specific elements based on predefined criteria. For example, the position of an element in the design can be a criterion. Elements placed in prominent positions (such as the center or top of the design) can be prioritized, while those in less conspicuous areas (such as corners or edges) can be discarded.

[0029] Some embodiments detect one or more protagonist elements based at least in part on masks, scene graph filtering, clustering, and / or one or more protagonist element rules. One or more protagonist elements indicate one or more visually significant regions. For example, some embodiments overlay saliency data from a saliency mask onto the scene graph to determine which specific elements correspond to highly saliency regions identified in the rendered image. Various embodiments additionally apply protagonist element rules. These rules are predefined strategies or criteria that consider visual saliency and / or contextual importance to identify protagonist elements. For example, these rules may include: elements overlapping with highly saliency regions in a saliency mask are considered visually more important. Alternatively, certain types of elements (such as titles, key images, or buttons) may be prioritized based on their role in the design. Alternatively, larger or centrally positioned elements may be prioritized as potential protagonist elements. Using a combination of information from the saliency mask and protagonist element rules applied to the scene graph, various embodiments then identify or detect protagonist elements. These elements are considered important for the visual and communicative impact of the design.

[0030] Regarding clustering for identifying protagonist elements, some embodiments group elements that are physically close to each other (e.g., via Euclidean distance) into clusters (e.g., via agglomerative hierarchical clustering). This ensures that elements close to each other are considered relevant, thus preserving the spatial organization of the design. Additionally or alternatively, some embodiments group elements that share similar visual characteristics (such as color, size, or shape) together. This helps create a consistent visual experience by treating similar elements consistently. Additionally or alternatively, some embodiments group elements that overlap with or are close to highly saliency areas in the saliency map. This step focuses on grouping visually important elements based on saliency analysis. Once elements are grouped into clusters, protagonist element rules are applied to determine which clusters (or elements within clusters) contain protagonist elements. For example, a cluster including large, centered title text and key images located in highly saliency areas can be identified as containing protagonist elements because these components are visually and contextually important.

[0031] Some embodiments additionally or alternatively determine one or more animation rules based on saliency data and / or protagonist element detection. For example, a saliency map might show high saliency for title text and product images, medium saliency for call-to-action buttons, and low saliency for decorative snowflakes. Based on saliency data and / or the detection of elements as protagonist elements, various embodiments use heuristic animation rules to determine how to animate each element. For example, a rule might be “Apply dynamic animation to high-saliency elements to immediately capture the viewer’s attention.” Since the title text and product images are high-saliency, this rule specifies the use of dynamic animation for these elements. For example, some embodiments apply a “zoom in” animation to the title text and a “slide in” animation to the product image from the left. Both animations are short in duration (e.g., 1 second) and have a “ease out” easing feature to create a fast, eye-catching effect. Animation is the process of creating the illusion of movement and change by continuously displaying a series of still images or frames. In the context of digital design, animation involves applying movement, transitions, or effects to visual elements (such as text, images, or shapes) to make them appear dynamic or interactive. For example, when a sequence of frames showing a ball in different locations is played rapidly in succession, the image of the ball bouncing up and down is animation. An animation sequence is a specific series of frames or steps that defines the movement or change of visual elements within the animation over time. It details the order in which the animation occurs, including the timing, duration, and transitions of each element's movement or effect. An animation sequence can involve multiple elements and effects working together to create a coordinated animation. For example, in a presentation slideshow, an animation sequence might include first fading in the title text, then sliding in an image from the left, and finally animate the bulleted list, showing one item at a time.

[0032] Various aspects of this disclosure employ a variety of technical solutions with significant advantages. For example, one technical solution generates animation output or sequences by generating masks (e.g., saliency masks) and / or detecting one or more protagonist elements to determine which elements are visually and contextually most important. This functionality instructs automatic animation. This reduces the need for manual adjustments, the mining described above, and minimizes user interaction with complex interfaces. Various embodiments intelligently determine the animation parameters for each element based on pre-established animation rules and visual saliency, significantly reducing the time and effort required to set up animations.

[0033] Some implementations eliminate the need for extensive animation expertise by automatically applying animation parameters or rules that take into account the importance of visual elements. This approach ensures that animation is both effective and visually coherent without requiring users to understand or manually apply complex animation principles. The system effectively handles the complexities of animation (such as setting keyframes and applying easing functions), allowing users to access the animation process without specialized knowledge.

[0034] Various implementations further streamline the animation process by automating element selection and animation based on element salience and / or relevance. Instead of manually animating each element, these implementations automatically detect which elements need animation (the main elements) and apply predefined animation rules to them. This significantly reduces the time and effort required to create animations, allowing users to quickly achieve professional results with less manual intervention in the user interface.

[0035] Various embodiments reduce I / O operations by automating the animation process, minimizing the need for repetitive manual adjustments and frequent updates to project data. By automatically applying animations based on generation masks (e.g., saliency masks), detection of protagonist elements, and / or animation rules, the system reduces the number of times data needs to be read from or written to storage. This reduces the overall I / O load on the system, thereby reducing wear on storage components and improving performance. Additionally or alternatively, various embodiments reduce memory usage by automating the animation process, thereby reducing the need to store large amounts of data in memory for manual editing and adjustments. By determining animations based on animation rules, detecting protagonist elements, and / or generating masks (e.g., saliency masks), the system can efficiently manage memory by loading only the elements and features necessary for each animation process. Additionally, memory consumption can be further reduced because the process relies less on storing a large undo / redo history of manual actions. Example System

[0036] Now for reference Figure 1This document provides block diagrams illustrating various aspects of an example computing system architecture suitable for implementing some embodiments of this disclosure, and generally designated as System 100. System 100 represents only one example of a suitable computing system architecture. Other arrangements and elements may be used in addition to or in lieu of the arrangements and elements shown, and some elements may be omitted entirely for clarity. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or combined with other components, and in any suitable combination and location. For example, some or each of the components of the system may reside on a single computing device (e.g., Figure 10 The system 100 is located within a computing device 1000. Alternatively, some or each of the components may be distributed across various computing devices, such as in a distributed cloud computing environment. As described in more detail herein, in some embodiments, system 100 and each of the components are located within a computing device 1000. Figure 9 Within the server and / or user equipment.

[0037] System 100 includes (multiple) networks 110, combined with Figure 9 The system 100 is described and communicatively coupled to its components, including a scene graph generator 102, a rendered document image generator 104, an animation preset generator 106, a scene graph filter component 108, a saliency model 114, a protagonist element component 112, an animation heuristic component 120, an animation generator 122, a user interface and integration layer 124, and a storage device 105. The components of system 100 may be implemented as a collection of compiled computer instructions or functions, program modules, computer software services, logic gates, hardware accelerators, or a process arrangement that executes on one or more computer systems. System 100 typically operates to generate animation output or sequences from a single image or document.

[0038] Scene diagram generator 102 is typically responsible for generating data structures that represent one or more elements in a design document, such as scene diagrams that represent the hierarchical structure of all elements in the design document, including their characteristics and relationships. For example, in web design, the scene diagram generator organizes elements such as text, images, and buttons into a structured format that shows the position of each element and its relationship to other elements.

[0039] In some embodiments, the scene graph generator 102 generates a scene graph by parsing a design document to identify all individual elements and their characteristics. This involves reading the file format and extracting information about each element, including its type (e.g., text, image, and shape), position, size, color, layer order, and / or any other attributes. The scene graph generator 102 analyzes the spatial and logical relationships between elements to determine their hierarchical structure. Elements are organized into parent-child relationships based on their grouping and nesting within the design document. For example, text boxes and images grouped together can form parent nodes, with text boxes and images as child nodes. For each identified element, the scene graph generator 102 creates a node in the scene graph. Each node represents a separate element or group of elements and contains data about its characteristics and relationships with other nodes. This data includes transformation information (e.g., translation, rotation, scaling) and other attributes such as opacity and blending mode. Nodes are connected to form a tree structure, where each parent node represents a group of elements, and child nodes represent elements within that group. This hierarchy reflects the organization of elements in the design document. For example, if a button consists of text and a shape, the button will be the parent node, and the text and shape will be its child nodes. The scene graph generator 102 then calculates transformations (such as position, rotation, and scale) for each node based on its parent node. This means that any transformations applied to a parent node are also applied to its child nodes, allowing for efficient manipulation of grouped elements. For example, moving a group node containing multiple elements will move all its child elements accordingly. After building the hierarchy, the scene graph generator 102 optimizes the scene graph for efficient processing. This may involve removing redundant nodes, merging similar nodes, or optimizing transformations. The final scene graph is then stored in memory (e.g., to storage device 105) in preparation for rendering, animation, or further processing.

[0040] The render document image generator 104 is typically responsible for producing a flat visual representation of the entire design document by rendering all elements from the scene graph into a single image. For example, it generates a visual preview of a flyer by combining all text, images, and graphics into a single image, accurately showing how the flyer will look when viewed. In an illustrative example, the render document image generator 104 traverses the scene graph from the root node, rendering each element in the order determined by its hierarchy and layers, and applying any transformations (such as scaling, rotation, and translation) and visual properties such as color, opacity, and effects. As each element is rendered, it is composited onto the canvas (considering depth order and blending modes) to produce a single flat image that visually represents the entire design document exactly as intended, showing the combination of all elements in the final layout. Additionally or alternatively, in some embodiments, the render document image generator 1004 generates any suitable image, such as a digital image (e.g., a photograph) containing pixel data or raster graphics data.

[0041] The Animation Preset Generator 106 is typically responsible for defining the initial animation presets to be applied to elements in a design document, such as type, timing, and effects. An "animation preset" is a collection of predefined parameters, properties, and configurations that can be applied to animated elements. For example, animation presets might include setting default animation effects, such as a "fade in" for text and a "slide in" for an image on a marketing banner for 2 seconds. Animation presets are automatically generated by the system (as a built-in option) and / or by the user (for customization needs). For example, users can create their own animation presets by setting specific combinations of animation parameters (such as keyframes, timing, motion paths, easing curves, etc.) and then saving these configurations for reuse. This process allows users to develop personalized animations suited to their specific project requirements or creative styles.

[0042] Scene graph filter component 108 is typically responsible for selecting and / or discarding elements from the scene graph based on predefined criteria, focusing on those elements most relevant to the animation. For example, in some embodiments, scene graph filter component 108 filters / discards background elements of the animation and selects key text and images for the animation in a promotional flyer. Scene graph filter component 108 is described in more detail below.

[0043] A saliency mask generator 114 is typically responsible for generating one or more masks that indicate visually important regions and / or regions most likely to attract human attention. For example, in some embodiments, a saliency model generates saliency masks. A saliency model is a computational model designed to predict which parts of an image are most likely to attract human attention. It generates saliency masks by analyzing various visual features, such as color, contrast, intensity, and / or spatial frequency, to identify regions that stand out from their surroundings. These regions are assigned higher saliency values, creating a saliency map that highlights the most attention-grabbing areas in the image. This model can be based on traditional methods using handcrafted features, or it can be based on deep learning techniques that learn from large image datasets and eye-tracking data to predict saliency.

[0044] While saliency models are specifically designed to predict visual saliency, other types of models (particularly those utilizing deep learning techniques) can also generate attention maps or masks that highlight important regions in an image. Such models can include, for example, convolutional neural networks (CNNs), transformer models, object detection and segmentation models (e.g., YOLO and SSD), and visual transformers (ViT). ViT can generate attention maps. In ViT, attention weights are calculated for each token (image patch) based on its relationship to other tokens. These attention weights can be visualized as a graph indicating which parts of the image are emphasized, effectively serving as a form of saliency map.

[0045] The protagonist element component 112 uses generated saliency maps(s), clustering module 118, and / or protagonist element rules 119 to detect and prioritize key design elements (protagonist elements) for animation. For example, the protagonist element component 112 uses a saliency model to highlight visually prominent areas in a poster and identifies the main title and product image as protagonist elements to be emphasized.

[0046] The protagonist element component 112 includes a clustering module 118, protagonist element rules 119, and protagonist element detector 116. The clustering model 118 is typically responsible for grouping design elements into clusters based on predefined criteria such as spatial proximity, visual similarity, and / or their association with highly visually salient areas in the saliency map. It analyzes the scene graph to identify elements that are geographically close, share similar visual characteristics (such as color, size, or style), and / or are located in highly salient areas. By grouping these elements into clusters, the module allows the system to treat related elements as a single unit, thereby facilitating coordinated animation and enhancing the visual coherence of the design. For example, in web design, the clustering module can group title text, subheadings, and product images into a cluster because they are geographically close and visually related in color and thematic content. Another cluster could include a collection of social media icons located together in the footer. By clustering these elements, the system can apply animations that make the title and product image appear simultaneously in a synchronized manner, while the animation of the social media icons is more subtle, thus maintaining a harmonious and visually appealing layout.

[0047] The protagonist element rule 119 is a heuristic criterion used to identify or detect protagonist elements that should be prioritized for animation based on their visual and / or contextual importance. These rules are determined by analyzing a saliency map generated from rendered document images, which highlights visually prominent areas, and / or by evaluating characteristics within the scene diagram, such as the element's type, size, position, and relevance to the design message. The protagonist element detector 116 uses these rules to determine which elements are most critical to conveying the design's main message and therefore should receive the most attention during animation. For example, in a marketing flyer, the protagonist element rule might prioritize large, bold headline text and a central product image appearing in a high-salience area of ​​the flyer. The rule might specify that elements with the highest visual salience and those occupying a central position in the design should be identified as protagonist elements. Therefore, the system will select the headline and product image as protagonist elements and apply prominent animations, such as zoom-in or slide-in effects, to effectively draw the viewer's attention to these key components.

[0048] The protagonist element detector 116 is typically responsible for detecting protagonist elements by analyzing the clusters generated by the clustering module 118 and applying protagonist element rules 119. It evaluates each cluster to determine if it contains elements that meet criteria set by protagonist element rules 119, such as high visual salience, central positioning, or significant size. By examining the visual salience of elements within each cluster and their contextual importance as defined by protagonist element rules 119, the protagonist element detector 116 selects specific elements or entire clusters as protagonist elements, prioritizing them for use in animation to enhance the overall impact of the design.

[0049] Animation heuristic component 120 generates and / or applies predefined animation rules to determine how protagonist elements and other components should be animated (e.g., based on their salience and importance). These rules consider factors such as the visual importance of elements (determined by protagonist element detector 116), their spatial relationships, and / or the overall design context to determine the most effective animation type, timing, duration, and / or style. The component dynamically adjusts these animation parameters based on the design's content and layout to ensure that each animation enhances visual focus and contributes to a harmonious and engaging presentation. For example, animation heuristic component 120 uses rules to animate protagonist elements (such as a central product image) where a "magnification" effect is used to draw attention, while more subtle animations are applied to secondary elements.

[0050] Animation generator 122 produces the final animation output by applying animation presets and / or animation rules (e.g., applied to selected elements in the scene graph). For example, animation generator 122 processes parameters or presets (such as animation type, duration, start time, easing, and path) by interpreting the hierarchy and relational data of the scene graph to animate elements according to a specified sequence and effects. In some embodiments, generator 122 utilizes keyframes and interpolation techniques to create smooth transitions and movements, rendering each frame sequentially to produce a final animation output aligned with the designed visual and contextual goals, ensuring synchronization and visual coherence of all animated elements. Keyframes involve setting specific "keyframes" at important points in the animation, where the characteristics of the element (such as position, rotation, scaling, or opacity) are explicitly defined. Animation generator 122 then fills in the frames between these keyframes, creating the illusion of motion or change. Interpolation techniques (e.g., linear interpolation, Bezier interpolation) are algorithms used to calculate intermediate frames between keyframes. In the illustrative example, animation generator 122 combines animations such as the “slide-in” of text and the “fade-in” of images to create a coherent animated advertisement.

[0051] The user interface and integration layer 124 provides users with an interface to interact and integrate with all components of the system 200 to create a seamless animation workflow. For example, layer 124 allows users to preview animation designs, adjust parameters / presets, and complete animations through an intuitive interface in the animation software program.

[0052] Storage device 105 typically stores information, including data used in embodiments of the techniques described herein (e.g., design documents, scene diagrams, images, etc.), generative text, computer instructions (e.g., software program instructions, routines, or services), data structures, and / or models (e.g., saliency models). Any of these components can be accessed via any suitable component of system 100. In some embodiments, storage device 105 represents any suitable data repository or device, such as a database, data warehouse, RAM, cache, disk, RAID, and / or storage network (e.g., storage area network (SAN)).

[0053] Figure 2 The illustration depicts an example pipeline 200 for generating output animations according to some embodiments. In some embodiments, a scene graph filter component 208 represents... Figure 1 The scene graph filter component 108 and the salience and protagonist element component 212 represent... Figure 1 The saliency mask generator 114 and the protagonist element component 112, represented by the heuristic generator 222 Figure 1 Animation generator 122.

[0054] First, the scene graph filter component 208 receives the scene graph 202 as input to generate filter elements 210. In other words, the output of the scene graph filter component 208 is a filtered scene graph that includes only design elements selected based on predefined criteria. This filtered scene graph contains elements deemed relevant or important to the animation, while discarding elements unimportant to the animation process. For example, in an e-commerce webpage design example, the scene graph filter component 208 outputs a filtered scene graph based on criteria such as visual prominence and importance, by selecting only key elements of the animation (such as the title text "Flash Sale - 50% Off!", product images of shoes, and the "Buy Now" call-to-action button). Decorative background shapes and footer text are discarded because they are considered less relevant to the animation, allowing the system to focus on effectively animateing the most impactful elements.

[0055] The salience and protagonist element component 212 takes the filtered elements 210 and / or the rendered document image 204 as input to generate protagonist element groups (e.g., clusters). For example, in an e-commerce webpage design example, the scene graph filter component 208 outputs a filtered scene graph based on criteria such as visual prominence and importance by selecting only key elements for animation, such as the title text "Flash Sale - 50% Off!", product images of shoes, and the "Buy Now" call-to-action button. Decorative background shapes and footer text are discarded because they are considered less relevant to animation, allowing the system to focus on effectively animateing the most impactful elements. In response, the protagonist element component 112 detects protagonist elements. For example, in an e-commerce webpage design example, after the salience model generates a salience mask highlighting visually prominent areas such as title text and product images, the protagonist element component 112 detects protagonist elements in 210 by analyzing this salience data along with the filtered scene graph. This component applies heuristics to determine which elements within the high-salience areas are most critical for conveying the design message. For example, it can identify the title text "Flash Sale - 50% Off!" and the product image of the shoes as the main elements because they are visually prominent and central to the design goal of promoting sales, thus ensuring that these elements are given priority in the animation.

[0056] Then, the heuristic generator 222 takes animation presets 206 and / or (multiple) protagonist element groups 214 as input to generate output animation 214. In the e-commerce webpage design example, after the protagonist element component 112 detects protagonist elements such as the title text "Flash Sale - 50% Off!" and product images of shoes, the animation generator 222 creates output animations by applying one or more (e.g., subsets) of appropriate animation presets 206 and / or animation rules to these elements. For example, the animation generator 222 takes predefined animation presets and adjusts them based on the importance and visual prominence of the detected protagonist elements to ensure that these key elements are effectively highlighted. For example, it can apply a dynamic "zoom in" effect to the title text and a "fade in" effect to the product images, thereby creating an engaging sequence of animations that draws the viewer's attention to the most critical elements in the design.

[0057] Figure 3 This is an example pipeline 300 based on a saliency model according to some embodiments. In some embodiments, pipeline 300 represents a pipeline composed of... Figure 1The saliency mask generator 114 performs the functions. In some embodiments, pipeline 300 represents the architecture of a unified model of saliency and importance (UMSI). This model is designed to analyze visual content to predict its saliency (e.g., where people are likely to look) and / or importance (which parts of the content are most meaningful or important).

[0058] First, encoder 304 takes image 392 (e.g., a rendered document image) as input to generate one or more feature maps 306. Encoder 304 then processes the input image (the image of the poster) to extract feature maps. These feature maps 306 capture basic details about the visual content, such as edges, texture, color, and shape. In some embodiments, the encoder includes multiple layers representing different stages of feature extraction, such as through a convolutional neural network (CNN). Each successive layer captures more complex features. For example, encoder 304 generates feature maps that highlight the edges of the title text “Flash Sale - 50% Off!” and the outline of the product image, providing detailed visual data that helps the model understand which areas of the design are visually most important and should be considered for their salience and importance.

[0059] The Atrous Spatial Pyramid Pooling (ASPP) module 312 acquires feature maps 306 generated by the encoder 304 and applies several atrous (dilated) convolutions at different rates. This technique allows the saliency model to capture information at multiple scales, effectively understanding fine details and broader contextual information in the image. The ASPP module 312 outputs a set of feature maps 314 that have been processed to account for various spatial scales. By using atrous convolutions, the ASPP module 312 effectively increases the receptive field without reducing resolution, allowing it to extract features at various scales. This enables the module 312 to identify fine details and broader patterns within the image, which helps to accurately identify elements of different sizes and importance in tasks such as semantic segmentation, or, in the context of e-commerce examples, helps to determine the visual saliency of different design components such as text and images.

[0060] Cascaded layer 316 is responsible for generating cascaded features 318. Following ASPP module 312, feature maps 314 are cascaded (combined) along the depth dimension. This step aggregates multi-scale features into a single unified representation, which includes information from all processed scales. For example, suppose ASP module 312 generates several sets of feature maps 314 with different levels of detail using various dilation rates. One set of feature maps 314 might capture fine details, such as the sharp edges of the text “Flash Sale - 50% Off!”, while another set captures broader patterns, such as the overall shape and color of a product image. Cascaded layer 316 takes these multiple sets of feature maps and combines them into a single output tensor 318 by stacking them along the channel dimension. This unified output now contains comprehensive information from all feature maps, enabling the model to analyze images with fine and broad contextual understanding.

[0061] Decoder module 320 acquires cascaded features 318 and transforms them back into a spatial representation, ultimately outputting a saliency map 322. This saliency map visually highlights the regions of the input image most likely to attract human attention (e.g., indicated by bright areas in the output image). For example, in e-commerce webpage design, decoder 320 starts with high-dimensional, multi-scale feature maps 314 generated by encoder and ASPP module 312, which contain detailed information about visual content, including title text and product images. Decoder 320 then applies transposed convolutions or upsampling operations to progressively increase the resolution of these feature maps 314, thereby refining and reconstructing details to produce the final output, such as the saliency map.

[0062] In parallel with the saliency pathway, the saliency model also includes a classification module 308. In some embodiments, the classification module 308 applies a series of fully connected layers or 1x1 convolutions to the feature map 306 to generate a probability distribution over predefined categories (such as "important" or "unimportant"). This probability distribution provides a preliminary assessment of the importance of individual regions in the image. The classification module 308 outputs a probability distribution indicating the likelihood that each region will attract the viewer's attention. The decoder 320 then uses this distribution to focus on regions with higher probabilities, guiding a refinement process to generate a precise saliency map 322 that highlights key elements such as title text and product images, which are considered visually most important. Therefore, the output 310 of the classification module 308 is fed into the decoder 320, which uses this information to guide an upsampling and refinement process, ultimately generating a detailed saliency map 322 that highlights the visually most important regions in the image, such as title text and product images in e-commerce web page design.

[0063] although Figure 3The saliency model described is illustrated with an architecture based on a convolutional neural network (CNN), but it should be understood that any suitable architecture can be used to generate saliency masks. For example, visual transformers (ViT), graph neural networks (GNN), or recurrent neural networks (RNN) with attention mechanisms are architectures that can be used alternatively or additionally to generate saliency masks.

[0064] ViT is a transformer-based model that uses a self-attention mechanism to process image data. It splits an image into patches, encodes these patches, and learns the relationships between them to understand visual importance. ViT can generate saliency masks by using attention maps generated during the classification or prediction process. Self-attention weights highlight which parts of the image contribute most to the model's decisions. These weights can be aggregated to create saliency maps that show visually important regions. The output attention score acts as an importance metric and can be reshaped into a saliency mask emphasizing regions with high visual focus.

[0065] Generative Neural Networks (GNNs) operate on structured graph data, making them suitable for modeling relationships between different components of an image, such as pixels, regions, or object parts. In the context of saliency, GNNs can be used to model spatial relationships between image regions by treating each pixel or block as a node in the graph. The model can learn to propagate importance signals through the graph based on connectivity and feature similarity. By aggregating these signals, GNNs can highlight the most relevant nodes (pixels), forming a saliency mask that emphasizes key regions.

[0066] RNNs, typically used for sequential data, can be combined with attention mechanisms to focus on specific portions of the input sequence; in the case of images, this specific portion could be a sequence of pixels or regions. When applied to images, an attention-based RNN can process blocks or rows of pixels sequentially, using attention weights to determine which parts of the image are most important at each step. The attention scores can then be compiled to generate a saliency mask, highlighting the most-focused regions.

[0067] A reinforcement learning (RL) model can be trained to focus on image regions that are rewarded as important based on predefined criteria, such as visual saliency. The RL agent can learn a strategy for browsing images, assigning higher attention scores to regions that produce higher "rewards" based on their visual features. Over time, the agent learns to map images onto saliency masks that highlight these key regions.

[0068] Attention-enhanced convolutional models are hybrid models that combine CNNs with attention mechanisms. They enhance the model's ability to focus on specific regions of an image by learning spatial attention maps. The attention module learns to weight different regions of the feature maps generated by the CNN layers, effectively creating spatial focal points that can be directly interpreted as saliency masks. Other examples include the transformer models described in this paper, object detection and segmentation models (e.g., YOLO and SSD).

[0069] Figure 4A This is a screenshot of a user interface page 400 according to some embodiments, illustrating a static design document 420 prior to requesting animation generation. Page 400 includes a left pane of user interface elements 402, 404, 406, 408, and 410, each element corresponding to an animation preset (e.g., generated by an animation preset generator 106) or a sequence of animations of different styles. The static design document 420 includes multiple elements 420, 422, 424, 426, 428, and 430 that make up the design document 420.

[0070] The "Fade All" animation style (corresponding to 402) involves simultaneously and smoothly transitioning all elements from completely transparent (invisible) to completely opaque (fully visible). This effect creates a soft, gradual display of all elements on the screen, adding a subtle and elegant introduction to a design. This effect can be used in presentations or slideshows where a calm and harmonious look is needed for elements, without requiring attention to individual components. The "Pop Up" animation style (corresponding to 404) makes elements appear to pop rapidly from a smaller size to their full size on the screen. This effect is characterized by a rapid zoom-in motion, giving the impression of elements bursting or bouncing into the view. This style is ideal for highlighting specific elements or creating a dynamic, energetic feel in a design, and is often used in advertisements or engaging slideshows to quickly grab the audience's attention.

[0071] The "Slide All" animation style (corresponding to 406) moves all elements from their off-screen positions to their final positions on the screen, typically from the left, right, top, or bottom. All elements slide in together, maintaining their relative positions as the animation enters the view. This effect can be used to present content in a structured way, suitable for slides or interfaces that require a clean, organized presentation. The "Sunrise" animation style (corresponding to 408) gradually raises elements from the bottom of the screen upwards, mimicking the natural movement of a sunrise. Elements start at the bottom of the screen and slowly rise to their designated positions, usually accompanied by a fade-in effect. This style is used to create calming, uplifting animation sequences, ideal for presentations or designs aimed at conveying warmth, optimism, or a fresh start.

[0072] The "Waterfall" animation style (corresponding to 410) makes the appearance of elements stagger one after another, like a waterfall cascading down. Each element appears or moves in sequence, creating a smooth, continuous animation that draws attention downwards or through the design. This effect can be used in infographics or tutorials, where information is presented in a step-by-step or layered order, enhancing understanding and engagement.

[0073] Figure 4B According to some embodiments Figure 4A A screenshot of the user interface page 400 illustrates the start step of a “waterfall” animation style in response to a user request to generate a corresponding animation sequence. In some embodiments, in response to receiving an indication that the user has selected user interface element 410 (corresponding to the “waterfall” animation style), element 422 is first presented as part of a first animation step in the animation sequence, followed by a second step in the animation sequence where element 426 is presented in a “waterfall” style.

[0074] More specifically, in some embodiments, in response to receiving an indication that a user has selected a user interface element 410, a rendered document image generator 104 converts the design document 400 into an image. Then, a scene graph filter component 108 filters the corresponding scene graph, a saliency mask generator 114 responsively generates corresponding masks indicating which elements in the rendered document image of the design document 400 are likely to attract human attention, a protagonist element component 112 detects (multiple) protagonist elements, an animation heuristic component 120 determines one or more animation rules, and an animation generator 122 initiates the process of a waterfall animation sequence, such as... Figure 4B As shown in the diagram. Figure 4B The illustration also shows user interface element 440, which illustrates the different layers (i.e., steps in the animation sequence) and timings. The start time determines the order and rhythm of the animation effects. For example, important elements should appear first, and text should appear in a left-to-right and top-to-bottom order.

[0075] Figure 4C According to some embodiments Figure 4A A screenshot of the user interface page 400, which illustrates... Figure 4B This is an intermediate step in the "waterfall" animation style sequence initiated in [the context]. For example... Figure 4C As illustrated, after the first two steps of rendering the waterfall animation style for elements 422 and 426 (also... Figure 4B (Illustrated in the image), presenting the third element 424, which corresponds to the third step in the waterfall animation sequence.

[0076] Figure 4D According to some embodiments Figure 4A A screenshot of the user interface page 400, which illustrates the transition from... Figure 4C The last time step in the continuing "waterfall" animation style sequence. (Example) Figure 4D As illustrated, the third element 424 (as shown) corresponds to the third step in the waterfall animation sequence. Figure 4C Following the illustration, the fourth element 428, corresponding to the fourth step in the waterfall animation sequence, is presented, followed by the fifth element 430, corresponding to the fifth step in the waterfall animation sequence. Example Flowchart

[0077] Figure 5 This is a flowchart of an example process 500 for filtering a scene graph according to some embodiments. In some embodiments, process 500 represents a function performed by scene graph filter component 108. Process 500 (and / or any function described herein, such as processes 600, 700, 800, and 900) may be performed by processing logic, which includes hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions running on a processor to perform hardware emulation), firmware, or a combination thereof. Although specific boxes described in this disclosure are referenced in a specific quantity and in a specific order, it should be understood that any box may appear substantially in parallel with any other box or before or after any other box. Furthermore, there may be more (or fewer) boxes than illustrated. The added boxes may include boxes that implement any of the functions described herein (e.g., as per [reference to...]). Figures 1 to 10 (As described herein). The computer-implemented methods, systems (including at least one computing device having at least one processor and at least one computer-readable storage medium), and / or computer-readable media described herein can perform or be used to perform process 500 or any other function described herein.

[0078] According to box 502, some embodiments first receive a scene graph. According to box 504, some embodiments detect one or more background elements in the scene graph and discard them for animation. For example, when the scene graph filter component 108 analyzes the scene graph, it calculates the bounding box region of each element. If the bounding box of an element occupies more than 80% of the canvas area (or other threshold-related value), the scene graph filter component 108 classifies it as a background element. This is because such a large element is likely intended to act as a background rather than a focal point. Identifying background elements helps ensure that the main content (foreground elements) is properly highlighted or animated without being obscured by the background.

[0079] In the illustrative example at box 504, scene graph filter component 108 determines the rectangular boundary that completely surrounds an element within the design document (and / or scene graph). It typically identifies the element's position, width, and height on the canvas by analyzing its coordinates in the scene graph or its placement within the design. Scene graph filter component 108 then calculates the area of ​​the bounding box by multiplying its width and height. This calculation is performed for each element to determine its size relative to the total canvas area, which helps classify the element as background or foreground based on predefined criteria such as whether the bounding box area exceeds 80% of the canvas area. Some embodiments then discard the background area. As described herein, "discarding" means masking (e.g., creating a mask to hide the background), marking (e.g., labeling the background element with a tag indicating it will be ignored during animation), complete removal from the graphics / design document, and / or layer separation.

[0080] According to box 506, some embodiments then detect a specific parent element with child elements and discard the child elements for animation. For example, in design documents, icons often consist of multiple smaller shapes grouped together to form a coherent symbol or graphic. For example, a star icon may consist of several smaller polygons or lines. Animating each of these small, individual child shapes can be computationally intensive and can lead to animation artifacts—unexpected visual glitches or irregularities in the animation. These artifacts can occur because animating many closely packed, detailed elements can result in overlapping, jittery movement, or unexpected interactions between shapes. To avoid these problems, scene graph filter component 108 identifies a top-level group that represents the entire icon (or other specific parent) rather than its individual components. This top-level group is a single parent node that encompasses all the child shapes of the icon. By animateing this group as a whole rather than individual shapes, the animation is smoother and more coherent, thus reducing the risk of artifacts.

[0081] The scene graph filter component 108 begins by traversing the scene graph from its root. The scene graph contains nodes representing all elements in the design, including their grouping and nesting. Each node corresponds to an element or group of elements and has information about its parent and child nodes. As it traverses, the scene graph filter component 108 identifies nodes marked as groups. A group node is a node that contains child nodes representing multiple elements grouped into a single unit. For example, in vector graphics, icons are typically stored as group nodes, which encapsulate all the individual shapes that make up the icon.

[0082] Scene graph filter component 108 examines each group node to see if it contains child nodes that are shapes or smaller elements. If a group node has child nodes that represent parts of the icon (such as paths, lines, or polygons), the group is considered a potential top-level group for the icon. Scene graph filter component 108 evaluates the characteristics of these group nodes, such as their size, position, and relationship to other elements in the design. If a group node encompasses multiple sub-shapes (or other threshold quantities) that form a coherent symbol and has no parent group, it is identified as the top-level group representing the icon. The graph filter component 108 then selects this group node as the top-level group for the icon. All animations are applied to this node, and child elements (individual shapes within the icon) are discarded during the animation process to prevent unnecessary complexity and animation artifacts.

[0083] In an illustrative example, consider a design document containing an icon of a house, composed of several shapes: the main body of the house is a rectangle, the roof is a triangle, and the doors and windows are small rectangles. A specific embodiment traverses the scene graph and finds a group node labeled "House Icon". It checks if this "House Icon" group node has child nodes representing the rectangle (main body), triangle (roof), and small rectangles (doors and windows). Since all these sub-shapes are part of a coherent symbol (House Icon) and are grouped together under the "House Icon" node, the specific embodiment identifies this node as the top-level group of the House Icon. The specific embodiment then selects the "House Icon" group node for animation and discards the child nodes (individual shapes) from the animation process. By animate the "House Icon" group as a whole, the scene graph filter component 108 ensures smooth animation of the entire icon, thus avoiding the complexity and potential artifacts that might result from animateing each shape independently.

[0084] According to box 508, a particular embodiment detects (multiple) rotation groups and assigns animations as a whole to (multiple) groups. A “rotation group” refers to a set of elements that are grouped together and then rotated as a single unit. This group can be any collection of elements, such as multiple shapes or images that are transformed together. For example, cropped images within a group that are rotated to fit a design layout are considered a rotation group. When a group is rotated, applying animations to individual child elements (such as moving or fading each element individually) can cause problems. Specifically, it can lead to incorrect animation start times and directions because the transformation (such as rotation) changes how the animation of each child element should be interpreted. This can cause animations to be misaligned, appear at the wrong time, or move in unexpected directions. To manage this, some embodiments track the top-level groups that have been rotated. The top-level group is the master node in the scene graph that encompasses all child elements. By identifying these groups, the system ensures that animations are correctly applied to the entire group, not just individual children.

[0085] Some implementations assign animation presets to the top-level group as a whole, controlling how the group is animated. This can involve applying a specific type of animation (such as a fade-in or slide) to the entire group, maintaining its visual coherence despite the group's rotation. Some implementations additionally or alternatively assign animation start times to all children within the group. This means that while the group is animated together based on its preset, each child element still has its own specific animation start timing relative to the group's overall animation. This approach ensures that the animations are properly synchronized and aligned, taking into account the group's rotation and maintaining visual consistency.

[0086] In the illustrative example of box 508, consider a design document for a digital presentation slide that includes a group of rotating elements representing a rotated photo collage. This group consists of the following elements: Image 1: a cropped square image of a landscape; Image 2: a circular clipping of a person; Image 3: a hexagonal image of a building; and a background shape: a decorative triangle behind the image. All these elements are grouped together and rotated 45 degrees to create a dynamic, tilted collage effect on the slide. The designer groups these images and background shapes into a single top-level group called “Photo Collage.” The entire “Photo Collage” group is then rotated 45 degrees clockwise to create a visually interesting tilting effect. If the embodiment animates each image and background shape individually without considering rotation, it could result in incorrect animation start timing and direction. For example, if Image 1 (the landscape image) is animated as sliding in from the left without considering the group's rotation, it might not follow the correct angular path and might appear out of sync with the rest of the group. Various embodiments identify “Photo Collage” as the rotated top-level group because it has already been rotated 45 degrees. Note that this group requires special handling to ensure all animations are correctly synchronized and aligned. Therefore, some implementations assign animation presets to the entire "Photo Collage" group. For example, it can apply a "fade in" animation to the entire group, ensuring the collages appear together as a unified rotating element.

[0087] When animate the entire group using the "Fade In" preset, each child element (Image 1, Image 2, Image 3, and the background shape) is assigned a specific animation start time. For example, Image 1 can begin fading in first, followed by Images 2 and 3, with a slight delay between them. The background shape can appear last to complete the effect. By treating the "Photo Collage" as a rotating top-level group and applying a uniform animation preset while managing the start time of each child element, visual coherence and synchronization are ensured. The group fades in together, and each image appears sequentially, maintaining correct rotation and creating a harmonious and dynamic animation that enhances the overall presentation slides.

[0088] Figure 6This is a flowchart of an example process 600 for detecting one or more sets of protagonist elements according to some embodiments. In some embodiments, process 600 represents Figure 1 The function of the protagonist element component 112. Given filtered elements from the scene graph, a specific embodiment identifies the protagonist element group. Detecting protagonist elements allows for the writing of custom animation rules to improve the quality of the animation output. According to box 603, some embodiments first receive a rendered image (e.g., from...). Figure 1 (The rendered document image generated by the rendered document image generator 104). According to box 605, some embodiments derive a saliency mask by running a saliency model on the rendered image. For example, as... Figure 3 As illustrated, the saliency model uses image 302 to generate the final saliency map 322.

[0089] According to box 607, some embodiments then use a threshold X (e.g., 0.5) to convert the saliency mask into a binary image. For example, a particular embodiment sets the saliency mask threshold to 0.5 and runs connectivity components on the output to identify connected regions (blobs) of highly saliency areas. In some embodiments, the saliency mask / map is a grayscale image where the intensity of each pixel represents a saliency value, indicating the likelihood that the pixel will attract human attention based on visual features. The thresholding process involves converting the grayscale saliency map into a binary image, where, for example, pixels are classified into two categories: highly saliency and low saliency. A threshold of 0.5 means that any pixel with a saliency value greater than 0.5 (on a scale of 0 to 1) is considered part of a highly saliency region and is set to 1 (white in the binary image), while pixels with a value of 0.5 or less are considered low saliency and are set to 0 (black in the binary image). Running connectivity components on the output is the process used to identify and label clusters of connected pixels in the binary image. In this context, it identifies a set or group of white pixels (or other pixels sharing the same value) that are connected to each other horizontally, vertically, and / or diagonally. Various embodiments can identify different “connected regions” or clusters of highly saliency pixels by running connectivity component analysis on a thresholded binary image. Each connected region represents a region of the image where pixels are closely packed together and all exceed a saliency threshold, indicating a highly saliency region.

[0090] In the example illustration at box 607, the saliency map M is first analyzed to apply a threshold of 0.5, transforming it into a binary image that highlights highly saliency regions. Then, a particular embodiment performs connectivity component analysis on this binary image to group all adjacent highly saliency pixels together into connected regions or areas. This process helps identify the image and segment it into visually salient regions, which can then be further analyzed or used in subsequent steps of the animation process, such as determining which regions or elements should be emphasized or animated.

[0091] According to box 609, some embodiments group similar elements within a threshold distance from each other into groups (e.g., clusters) from the filtered scene graph using binary images. For example, some embodiments (e.g., clustering module 118) run an agglomerative hierarchical clustering algorithm on the filtered scene graph elements. Agglomerative hierarchical clustering is a hierarchical clustering method that first treats each element as its own cluster and then progressively merges them based on their similarity until a desired number of clusters is reached. Some embodiments then set the requested maximum number of clusters to be the same as the number of salient connected regions extracted in box 607. Various embodiments then use a scoring function described below to compute a pairwise distance matrix for clustering. The pairwise distance matrix measures the distance or dissimilarity between each pair of elements to be clustered. This matrix is ​​based on a scoring function that quantifies the similarity or dissimilarity of elements according to several criteria.

[0092] The scoring function is designed to group elements that are 1) close to each other, 2) of the same type, and 3) overlap with the same saliency region. In other words, the scoring function used in the clustering process is specifically designed to group elements that are spatially close to each other, of the same type (e.g., grouping text elements or image elements), and that overlap with the same saliency region identified in the saliency map. This helps ensure that visually and context-dependent elements are clustered together. The mathematical representation of the scoring function is as follows: D[i,j]=iOU(i,j)+saliencyDiff(i,j)+distanceDiff(i,j)+typeDiff(i,j) Where D[i,j] is the distance between element i and element j, iOU is the intersection-union ratio (IU) between the bounding boxes of elements i and j, saliencyDiff is the absolute difference between the average saliency scores of elements i and j, and distanceDiff is calculated by taking the exponent of the L2 distance between the centers of the bounding boxes of elements i and j, and typeDiff is 0.5 if both elements are text types, otherwise alternatively 0.0 (e.g., if both elements are image types). Therefore, this equation is a scoring function used to calculate the distance or dissimilarity between two elements i and j during clustering.

[0093] In some embodiments, although the binary image created in block 607 itself is not directly used in the agglomerative hierarchical clustering algorithm in step 609, information from salient connected regions extracted from the binary image is useful. The number of clusters in the agglomerative hierarchical clustering algorithm is set to match the number of identified salient connected regions. Additionally, saliency information influences the clustering process through the "saliencyDiff" component of the distance metric. The output of 609 is a group of protagonist elements.

[0094] Figure 7 This is a flowchart of an example process for training a machine learning model to generate a saliency mask, according to some embodiments. Following box 702, some embodiments receive input (image)-output ((multiple) importance features) pairs. In other words, a particular embodiment collects a set of images (e.g., rendered design document images) that will be used as the training dataset. Various embodiments obtain ground truth saliency maps for each image, which are created, for example, by a human annotator or derived from eye-tracking data. These saliency maps serve as the target output of the model. Eye-tracking data refers to information collected by tracking and recording the movement and position of a person's eyes as they view various stimuli, such as images, text, or user interfaces. This data typically includes details such as fixation point (the focus of a gaze), saccades (rapid eye movements between gazes), gaze duration, and visual exploration sequences, indicating areas that may attract human attention.

[0095] According to box 704, some embodiments then initialize the model parameters. For example, some embodiments initialize the network's weights and / or biases. The weights can be initialized randomly or using specific strategies such as Xavier or He initialization. According to box 706, some embodiments then generate a predicted saliency map for each image in the dataset. Thus, various embodiments feed images into the model. Images pass through multiple layers of the network, such as convolutional layers, activation functions, pooling layers, and / or potentially fully connected layers, depending on the architecture. The model outputs a predicted saliency map, which represents the regions in the image where attention is likely to be focused. This is the raw output from the last layer of the network, such as a sigmoid or softmax activation of the probability map.

[0096] According to box 708, some embodiments then calculate the loss via a loss function. In other words, some embodiments use a suitable loss function to compare the predicted saliency map with the ground truth saliency map to quantify the degree of difference between the increment or the predicted saliency map and the ground truth saliency map. For example, some embodiments calculate the binary cross-entropy loss for a pixel-by-pixel classification problem or the mean squared error (MSE) of the regression type output. For example, some embodiments calculate the loss by computation:

[0097] According to box 710, some embodiments then compute the gradient of the loss function. For example, some embodiments compute the gradient of the loss function relative to each parameter (weight and bias) in the model. In some embodiments, this is done by backpropagating the error through the network layers using a chain rule. Some embodiments also perform backpropagation, which adjusts the network weights in a manner that minimizes the loss. The gradient of each weight is used to determine how much to change the weight to reduce the error.

[0098] According to box 712, some embodiments then adjust the model parameters based on the computed gradients and optimization algorithms. For example, some embodiments use optimization algorithms such as stochastic gradient descent (SGD), Adam, or RMSprop to update the model parameters. The optimizer uses the gradients computed during backpropagation to adjust the weights. Some embodiments additionally adjust the learning rate during training to improve convergence, typically reducing the learning rate over time as the model begins to converge.

[0099] According to box 714, some embodiments determine whether a convergence minimum is met. Convergence is achieved when the model's parameters (such as weights and biases) are adjusted through backpropagation and gradient descent such that the loss function consistently decreases and eventually stabilizes around a minimum. At convergence, the model's predictions (e.g., box 706) are close to the ground truth saliency map, indicating that the model has learned to accurately identify visually important regions of the image. When the model converges, it means that further training will no longer significantly improve the model's ability to predict saliency maps. This indicates that the model has learned the optimal representations and patterns necessary to effectively perform the task. Achieving convergence helps ensure that the model can generalize well to new, unseen images and consistently generate high-quality saliency maps.

[0100] If the convergence minimum is not met, box 706 is repeated to introduce another epoch / forward pass and generate another saliency map (e.g., from different inputs of the input-output pair). Boxes 710, 712, and 714 are also repeated until the convergence minimum is met, at which point process 700 stops. The above steps (from generating the saliency map (forward pass) to optimization) are repeated for each batch of images in the dataset. In each epoch, batches are processed one after another. The entire training set is processed multiple times (multiple epochs) to allow the model to learn and generalize. Each epoch consists of all training batches fed through the network.

[0101] Figure 8 This is a flowchart of an example process 800 for generating an animation sequence according to some embodiments. At block 802, some embodiments receive an image or file comprising one or more elements. In some embodiments, the image represents any suitable image, such as a digital photograph or video frame. In some embodiments, the image represents a rendered document image (e.g., Figure 2 The rendered document image 204). The rendered document image is a design document (e.g., Figure 4A A visual representation of a design document (420). In some embodiments, the design document represents a file in frame 802, which is created in graphic design or layout software. For example, some embodiments first receive a design document and then generate a rendered image of the design document (e.g., as per [reference to design document]). Figure 1 (As described in the rendering document image generator 104). Some embodiments generate scene graphs from design documents, which represent each element in the design document in a hierarchical structure, wherein each node in the scene graph corresponds to an element or group of elements in the design document, for example, as per [reference to...]. Figure 1 The scene graph generator 102 is described.

[0102] According to box 804, some embodiments generate a mask indicating one or more visually significant regions in an image or file (e.g., at least partially automatically in response to receiving the image or file). For example, in some embodiments, box 804 includes information about... Figure 1 The saliency mask generator 114 describes the functionality described therein. A "mask" is any suitable data structure and / or technique used to control which parts of an image, document object, or surface are affected by certain operations or transformations. Masks can be used in graphics and image processing to selectively hide or reveal parts of an image, allowing for more precise and localized adjustments. In the illustrative example of box 804, some embodiments generate saliency masks (e.g., by providing a representation of the rendered image (e.g., a preprocessed version, such as a matrix, vector, or grayscale version) as input to the saliency model. Figure 3 (Saliency Figure 322). A saliency mask indicates one or more visually important regions within a rendered image. For example, in some embodiments, a saliency mask is a grayscale image or heatmap that indicates which pixels in one or more regions are likely to attract human attention. In some embodiments, the mask is generated based on a machine learning model trained on an image dataset with labeled visually important regions, e.g., as per [reference to image model 322]. Figure 7 The training process described in 700.

[0103] Saliency refers to the likelihood that a specific area of ​​an image or document will stand out and attract human attention. A salient area is typically a region with a different color, texture, edges, or contrast compared to its surroundings. In contrast to a grayscale image, a saliency mask is an image where the intensity (from black to white) of each pixel indicates the level of saliency. Higher intensity (closer to white or the white threshold) means the area is more likely to attract attention, while lower intensity (closer to black or the black threshold) indicates less saliency. In contrast to a heatmap, this is a more colorful representation where colors (such as red, yellow, blue, etc.) are used to indicate the level of saliency. Generally, warmer colors (such as red and yellow) represent areas of higher saliency, while cooler colors (such as blue) represent areas of lower saliency. The purpose of a saliency mask is to predict or identify which parts of an image or document are most likely to catch the viewer's eye. This is based on various visual features such as contrast, color, edges, and / or texture. The areas highlighted by a saliency mask are those that are prominent and therefore more likely to be noticed by a human observer first.

[0104] According to box 806, some embodiments determine one or more animation rules, at least in part, based on a mask. In some embodiments, box 806 includes information about... Figure 1The animation heuristic component 120 describes the functionality described therein. In the illustrative example at box 806, if a saliency mask indicates that a particular character's face is likely to grab the viewer's attention, the animation rules can ensure that the character's facial expressions are more pronounced and dynamic. Additionally, various embodiments can use smoother or more exaggerated movements of the character to maintain engagement, thereby guiding the viewer's focus based on saliency data. This approach allows the animation to adapt dynamically, emphasizing the most compelling elements in the scene.

[0105] In some embodiments, box 806 additionally or alternatively includes defining one or more animation parameters. An "animation parameter" refers to one or more properties (e.g., animation presets) associated with the animation output and / or one or more animation rules indicating in box 806 specifying how the animation output should be applied to different elements within a file or image. For example, in the context of an animation preset, animation parameters may include attributes such as the duration of the animation, a delay before the animation begins, the type of movement / animation output (such as fade, slide, or scale), and easing features (how the speed of the animation changes over time, such as ease in or ease out). If the animation preset is set to "fade in," the animation parameters might specify that the fade lasts for 2 seconds, begins 0.5 seconds after the element appears on the screen, and uses an ease out easing feature to create a smooth, gradual appearance. These parameters ensure that the animation behaves consistently across different elements when the preset is applied. In some embodiments, each animation parameter includes an animation preset name (e.g., "spin," "roll," "fade") and / or a set of parameters such as duration, personality, or direction.

[0106] In some embodiments, block 806 additionally or alternatively includes determining an animation preset, which includes a set of predefined animation styles selectable by the user, such that determining one or more animation rules includes determining how to apply or modify the animation preset, and wherein the generation of an animation sequence is based on determining how to apply or modify the animation preset. Therefore, the animation preset in various embodiments is a set of predefined animation styles and effects selectable by the user. About Figure 4A UI elements 402, 404, 406, 408, and 410 illustrate this point. These presets simplify the animation process by providing a variety of ready-made animation options, such as "fade in" or "swipe left," which users can easily apply to different elements within their design. Various embodiments then determine how to implement or modify these selected animation presets based on specific criteria or user input. This includes adjusting preset parameters (such as timing, duration, or sequence) to fit the specific needs or context of the design, ensuring the animation aligns with the desired visual outcome.

[0107] According to box 808, some embodiments generate an animation sequence of multiple elements of an image or document by applying at least multiple animation rules, at least in part, based on at least one of the animation rules. Alternatively or additionally, box 808 represents generating an animated output associated with an image or document by at least in part based on the detection of one or more protagonist elements. Alternatively or additionally, box 808 represents generating animated output by applying one or more animation parameters at least in part based on a saliency mask to a design document.

[0108] The generation of animation output transforms the design document into an animated design document. An "animation sequence" is a series of steps that defines how the animation unfolds over time, outlining the specific order and timing in which different elements are animated. Each step in the sequence can involve applying different animation effects (such as fade, slide, or scale) to one or more elements, typically with specific parameters (such as duration and delay) to control the animation flow. For example, in a presentation slideshow, an animation sequence might begin with a step that fades in the title text, followed by a second step that slides in an image from the left, and a final step that animates the bullet points one by one with a pop-up effect.

[0109] In some embodiments, "animation output" includes an animation sequence. In some embodiments, "animation output" additionally or alternatively represents only a single step or effect (e.g., in a sequence of steps / effects). In some embodiments, "animation output" additionally or alternatively is the final rendered result of the animation process, which includes all applied animation effects, transitions, and sequences. It represents the complete visual presentation after the animation steps have been performed, and can be in the form of a video file, an interactive web animation, or a series of animation frames. For example, after creating an animation sequence and applying it to elements in a design document (e.g., text fade-in and image slide-in), the animation output is the final video or interactive content that the viewer sees, showcasing the expected smooth transitions and animations.

[0110] In some embodiments, the generation of the animation sequence in box 808 is based on a single user input representing a request to convert an image or document into an animation sequence. This "single user input" represents a "one-click" solution, making it different from existing technologies where no additional user input selection is required to animate the document besides a single user input. For example, returning to the reference... Figures 4A to 4D In response to receiving an indication that the user has selected the "Waterfall" UI element 410, a specific embodiment automatically executes the following... Figure 4B , Figure 4C ,and Figure 4D The corresponding waterfall animation steps are all completed without the user having to manually perform or otherwise participate in user input to complete the waterfall animation sequence.

[0111] In some embodiments, the generation of an animation sequence of one or more elements is further based on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria. For example, some embodiments filter the representation of one or more design elements of an image (e.g., an indicated scene graph) based on predetermined criteria. Regarding Figure 5 Process 500 and Figure 1 The scene graph filter component 108 describes an example of this filtering.

[0112] Some implementations use a threshold to convert the mask into a binary image (e.g., as per [reference]). Figure 6 (as described in box 607), and based on using binary images to group elements of the scene graph that are within a threshold distance from each other into one or more clusters (e.g., as per [reference to...]). Figure 6 (As described in box 609). In this way, the generation of animation sequences is further based on transformation and combination.

[0113] Some embodiments detect one or more protagonist elements in a scene graph based on analyzing the overlap between one or more design elements in the scene graph and one or more highly salient regions in a mask, and the generation of an animation sequence is further based on the detection of one or more protagonist elements. In some embodiments, the process involves thresholding the saliency map to highlight the visually most important regions, identifying highly salient connected regions (e.g., such as...) Figure 6 (as described in boxes 605 and / or 607), and clustering design elements based on their spatial and visual relationships with these areas (e.g., as described in...). Figure 6 (As described in box 609). Then, based on these detected protagonist elements, the generation of the animation sequence is further customized to ensure that they occupy a prominent position in the final animation. Exemplary operating environment

[0114] Now go to Figure 9This document provides an illustrative depiction of an example computing environment 900 for recommending one or more color values ​​to be applied to an input image, which may employ some embodiments of the invention. It should be understood that such and other arrangements described herein are presented by way of example only. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groupings, etc.) may be used in addition to or in place of the arrangements and elements shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or in combination with other components, and in any suitable combination and location. For example, there may be multiple servers 910 representing nodes in a cloud computing network. The various functions described herein as being performed by one or more entities can be performed by hardware, firmware, and / or software. For example, various functions can be performed by a processor executing instructions stored in memory.

[0115] Figure 9 The environment 900 depicted includes a prediction server (“server”) 910 communicating with network 110. Environment 900 also includes a client device (“client”) 920 that also communicates with network 110. Furthermore, client 920 can communicate with server 910 via network 110 and generate requests to animate images and / or design documents for transmission to server 910. In various embodiments, client 920 is implemented in a computing device, which may be referred to herein as a client device or user device, such as regarding… Figure 10 The computing device 1000 is described.

[0116] In some embodiments, Figure 1 Each component is included in either server 910 or client device 920. Alternatively, in some embodiments, Figure 1 The components are distributed between server 910 and client device 920.

[0117] Server 910 can receive requests transmitted from client 920 and can search for relevant data via any number of data repositories that server 910 can access remotely or locally. Data repositories may include one or more local or remote computing devices, each of which server 910 can access directly or indirectly via network 110. According to some embodiments described herein, a data repository may include one or more remote servers, any node in a distributed plurality of nodes (e.g., computing devices such as those typically maintaining a distributed ledger (e.g., blockchain) network), or any remote server coupled to or communicating with any node in the distributed plurality of nodes. Any of the data repositories mentioned above may be associated with one of a plurality of data storage entities, which may or may not be associated with each other. As described herein, a data storage entity may include any entity (e.g., retailer, manufacturer, e-commerce platform, social media platform, web host) that stores data associated with its customers, clients, sales, relationships, website visitors, or any other subject of interest to the entity (e.g., name, demographic data, purchases, browsing history, location, address). Considering that each data repository is typically associated with a different data storage entity, although some data storage entities may be associated with multiple data repositories, and some data repositories may be associated with multiple data storage entities. In various embodiments, server 910 is implemented in a computing device, such as regarding... Figure 10 The computing device 1000 is described.

[0118] Having described embodiments of the invention, the following describes exemplary operating environments in which embodiments of the invention may be implemented, in order to provide a general context for the various aspects of the invention. Specifically, initial references are made to… Figure 10 An exemplary operating environment for implementing embodiments of the present invention is shown and is generally designated as computing device 1000. Computing device 1000 is merely an example of a suitable computing environment and is not intended to imply any limitation on the scope of the purpose or functionality of the invention. Nor should computing device 1000 be construed as having any dependency or requirement on any of the components or combinations of components illustrated.

[0119] Looking at it now Figure 10 The computing device 1000 includes a bus 10 that directly or indirectly couples to the following devices: a memory 12, one or more processors 14, one or more presentation components 16, input / output (I / O) ports 18, input / output components 20, and an illustrative power supply 22. The bus 10 can represent one or more buses (such as an address bus, a data bus, or a combination thereof). Although lines are used for clarity... Figure 10Various boxes are used to depict the components, but in reality, the descriptions of the various components are not so clear, and metaphorically, the lines would be more accurately described as gray and blurry. For example, presentation components (such as display devices) can be considered as I / O components. Moreover, the processor has memory. The inventors recognize this as the nature of the art and reiterate... Figure 10 The accompanying drawings merely illustrate exemplary computing devices that can be used in conjunction with one or more embodiments of the invention. There is no distinction between categories such as "workstation," "server," "laptop," and "handheld device," as all of these are considered within the scope of the invention. Figure 10 Within that scope, it is referred to as a "computing device".

[0120] Computing device 1000 typically includes a variety of computer-readable media. Computer-readable media can be any available medium accessible by computing device 1000, and includes volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Universal Disc (DVD) or other optical disc storage devices, magnetic cartridges, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by computing device 1000. Computer storage media itself does not include signals. Communication media typically implement computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. The term "modulated data signal" refers to a signal whose one or more characteristics are set or altered in a manner that encodes information in the signal. By way of example, and not limitation, communication media include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media). Any combination of the above media should also be included within the scope of computer-readable media. In various embodiments, computing device 1000 represents... Figure 9 The client device 920 and / or the server 910.

[0121] Memory 12 includes computer storage media in the form of volatile and / or non-volatile memory. The memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Computing device 1000 includes one or more processors that read data from various entities such as memory 12 or I / O components 20. Multiple presentation components 16 present data instructions to a user or other device. Exemplary presentation components include display devices, speakers, printing components, vibration components, etc. In some embodiments, the memory includes program instructions that, when executed by one or more processors, cause one or more processors to perform any of the functions described herein (such as processes 500, 600, 700, 800, and / or 900), or to perform actions related to... Figures 1 to 9 Any of the functions described.

[0122] I / O port 18 allows computing device 1000 to be logically coupled to other devices, including I / O components 20, some of which may be built-in. Illustrative components include microphones, joysticks, game controllers, satellite antennas, scanners, printers, wireless devices, etc. I / O component 920 can provide a natural user interface (NUI) for processing hovering gestures, voice, or other physiological input generated by the user. In some instances, the input can be sent to appropriate network elements for further processing. The NUI can implement any combination of speech recognition, stylus recognition, facial recognition, biometric recognition, on-screen and near-screen gesture recognition, hovering gestures associated with the display of computing device 1000, head and eye tracking, and touch recognition (described in more detail below). Computing device 1000 may be equipped with depth cameras, such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof, for gesture detection and recognition. Additionally, computing device 1000 may be equipped with accelerometers or gyroscopes capable of motion detection. The output of an accelerometer or gyroscope can be provided to the display of a computing device 1000 to render immersive augmented reality or virtual reality.

[0123] It is understood that, in addition to the above, embodiments of the present invention also provide notifications of proof and authentication services corresponding to the authenticity of the determined claims. The invention has been described with respect to specific embodiments, which are intended in all respects to be illustrative and not restrictive. Alternative embodiments will become apparent to those skilled in the art to which this invention pertains without departing from its scope.

[0124] As will be seen from the foregoing, the present invention is well suited to achieving all the objectives and purposes stated above, as well as other advantages that are obvious and inherent to the system and method. It will be understood that certain features and sub-combinations are practical and can be employed without reference to other features and sub-combinations. This is as contemplated by and within the scope of the claims.

[0125] This document specifically describes the subject matter of the invention to meet legal requirements. However, the specification itself is not intended to limit the scope of this patent. Rather, the inventors have considered that the claimed subject matter may also be implemented in other ways to include different steps or combinations of steps similar to those described herein, while incorporating other prior art or future technology. Furthermore, although the terms “step” and / or “box” may be used herein to refer to different elements of the method employed, these terms should not be construed as implying any particular order among or between the various steps disclosed herein, unless the order of the individual steps is explicitly described.

Claims

1. A method comprising: Receive design documents; Generate a rendered image of the design document; By providing a representation of the rendered image as input to a machine learning model, a saliency mask is generated via the machine learning model, which indicates one or more visually important regions within the rendered image; Define one or more animation parameters; as well as Animation output is generated by applying at least one or more animation parameters to the design document, based at least in part on the saliency mask, the generation of which transforms the design document into an animated design document.

2. The method of claim 1, wherein the one or more animation parameters include at least one of the following: one or more properties associated with the animation output, or one or more animation rules specifying how the animation output should be applied to different elements within a design document.

3. The method of claim 1, wherein the generation of the animation output is further based on a single user input representing a request to convert the design document into the animation output.

4. The method according to claim 1, further comprising: A scene diagram is generated based on the design document. The scene diagram represents each element in the design document in a hierarchical structure, wherein each node in the scene diagram corresponds to an element or group of elements in the design document.

5. The method of claim 4, wherein the generation of the animation output is further based on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria.

6. The method according to claim 1, further comprising: The saliency mask is converted into a binary image using a threshold. as well as The generation of the animation output is based on combining elements of the scene graph that are within a threshold distance from each other using the binary image into one or more clusters, and the generation of the animation output is also based on the transformation and the combination.

7. The method of claim 1, wherein the generation of the saliency mask is based on training the machine learning model on a dataset of images with labeled visually important regions.

8. The method according to claim 1, further comprising: One or more protagonist elements in the scene graph are detected based on the overlap between one or more design elements in the scene graph and one or more highly salient regions in the saliency mask, and the generation of the animation output is further based on the detection of the one or more protagonist elements.

9. The method according to claim 1, further comprising: The determination includes an animation preset comprising a set of predefined animation styles selectable by the user, wherein the determination of one or more animation parameters includes determining how the animation preset is applied or changed, and wherein the generation of the animation output is based on determining how the animation preset is applied or changed.

10. A system comprising: Memory components; as well as A processing device coupled to the memory component, the processing device being configured to perform operations including: Receive an image or file containing one or more elements; Generate a mask that indicates one or more visually important regions in the image or file; One or more animation rules are determined, at least in part, based on the mask; and An animation sequence of one or more elements of the image or file is generated by applying at least one or more of the animation rules.

11. The system of claim 10, wherein the mask is a saliency mask, and wherein the automatic generation of the mask comprises automatically generating the saliency mask via a saliency model, and wherein the saliency mask is a grayscale image or heatmap indicating which pixels of the one or more regions are likely to attract human attention.

12. The system of claim 10, wherein the generation of the animation sequence is further based on a single user input representing a request to convert the image or file into the animation sequence.

13. The system of claim 10, wherein the image represents a rendered document image, the rendered document image is a visual representation of a design document, the design document being the file created in graphic design or layout software, and wherein the operation further comprises: A scene diagram is generated based on the design document. The scene diagram represents each element in the design document in a hierarchical structure, wherein each node in the scene diagram corresponds to an element or group of elements in the design document.

14. The system of claim 13, wherein the generation of the animation sequence of one or more elements is further based on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria.

15. The system of claim 10, wherein the operation further comprises: The mask is converted into a binary image using a threshold. as well as The generation of the animation sequence is based on combining elements of the scene graph that are within a threshold distance from each other using the binary image into one or more clusters, and the generation of the animation sequence is also based on the transformation and the combination.

16. The system of claim 10, wherein the generation of the mask is based on providing a representation of the image as input to a machine learning model and training the machine learning model on a dataset of images with labeled visually important regions.

17. The system of claim 10, wherein the operation further comprises: One or more protagonist elements in the scene graph are detected based on the overlap between one or more design elements in the scene graph and one or more highly salient regions in the mask, and the generation of the animation sequence is further based on the detection of the one or more protagonist elements.

18. The system of claim 10, wherein the operation further comprises: The determination includes an animation preset comprising a set of predefined animation styles selectable by the user, wherein the determination of one or more animation rules includes determining how the animation preset is applied or modified, and wherein the generation of the animation sequence is based on determining how the animation preset is applied or modified.

19. A non-transitory computer-readable medium storing executable instructions, which, when executed by a processing device, cause the processing device to perform an operation, the operation comprising: A mask is generated using a machine learning model to indicate one or more parts of an image that may attract human attention, the image including one or more design elements; The representation of one or more design elements of the image is filtered based on predetermined criteria; Based at least in part on the mask and the filtering, one or more protagonist elements are detected from the filtered representation of the one or more design elements, the one or more protagonist elements indicating one or more visually important regions; as well as Animation output associated with the image is generated, at least in part, based on the detection of the one or more main character elements.

20. The computer-readable medium of claim 19, wherein the operation further comprises: Based on the detection of the one or more main character elements, the filtered representations of the one or more design elements are grouped into one or more clusters, wherein the generation of the animation output is also based on the grouping of the filtered representations of the one or more design elements into one or more clusters.

Images