System and method for customized text animation

Abstract

The present disclosure relates to a system and method for custom textual animation. A system or method for animation implementation that presents or generates one or more alphabetic glyphs based on successive mappings that gradually reveal brushhead shapes in alphabetic stroke order is provided. A machine learning model is used to generate animated metadata from one or more alphabetic glyphs. The animation metadata includes a centerline, a continuous mapping of brush head shapes for sweeping along the centerline, and an alphabetic form stroke order for generating an animation implementation of one or more alphabetic glyphs.

Description

Background Technology

[0001] In the field of digital video editing, software applications offer a range of tools for creative manipulation. However, challenges remain in creating visual effects for unique text. Traditional methods of animate text may require laboriously masking text elements and then animateting the mask to reveal the text in front of the selected font. This process is typically time-consuming and requires manual intervention. Given these drawbacks, this patent aims to address the need for a more efficient text customization workflow that provides dynamic and seamless rendering of complex font effects. Attached Figure Description

[0002] To facilitate identification of any discussion of a particular element or action, the highest significant digit in the figure references refers to the figure number in which the element first appears.

[0003] Figure 1 An example system configured to support user accounts in creating and managing collaborative content files is shown, based on certain aspects of this technology.

[0004] Figure 2 An example collaborative content editing service configured to support user accounts in creating and managing collaborative animated text objects is shown, based on certain aspects of this technology.

[0005] Figure 3 An example method for implementing animation of one or more letter glyphs according to certain aspects of this technology is shown.

[0006] Figure 4 An example method is shown for training a machine learning model to generate an animation implementation of one or more letter glyphs, according to certain aspects of this technology.

[0007] Figure 5 An example graphical user interface depicting an animated playback of a set of letter glyphs in a first instance, according to certain aspects of the present technology, is shown.

[0008] Figure 6 An example graphical user interface depicting an animated playback of a set of letter glyphs in a second instance, according to certain aspects of this technology, is shown.

[0009] Figure 7 An example computing system based on certain aspects of this technology is shown.

[0010] Figure 8 An example neural network architecture based on certain aspects of this technology is shown. Detailed Implementation

[0011] Traditionally, text animation involves a laborious process where individual text elements are carefully masked, and then the masks are individually animated to reveal the letter shapes. This method demands significant effort and attention to detail, as each frame requires meticulous work to achieve the desired effect. The limitations of this approach become apparent when using variable fonts, as the fonts constantly change throughout the sequence, requiring frame-by-frame animation. Furthermore, the process can be time-consuming and error-prone, making it less than ideal for achieving smooth, seamless text animation.

[0012] Furthermore, simply instructing a traditional generative AI model to animate text is not a viable solution, as this requires the model to understand the nuances of letter forms and their relationships across various contexts. The model's ability to generate convincing animations will be hampered by a lack of understanding of typographical details such as font variations, stroke styles, and textures. Therefore, relying solely on generative AI can produce unsatisfactory results, highlighting the need for more sophisticated approaches that combine dynamic frameworks with AI-based methods.

[0013] As an example, a machine learning model can be customized and used as a dynamic framework for animate text. This might involve integrating a robust metadata input system that can accept and process centerline and font information for one or more glyphs. The machine learning model can then infer a continuous mapping of brush shapes tailored to the unique features of each letter, which can be used to sweep along a specified centerline. By providing this framework, the machine learning model can be tuned and generated an animated implementation of a set of glyphs, taking into account subtle differences in stroke style, texture, and spacing that reflect how the text appears as if it were drawn by someone.

[0014] In some cases, a machine learning model may be given one or more glyphs with novel outline shapes and may need to determine the center lines of one or more of these glyphs. Accordingly, the machine learning model can determine the center lines of a set of glyphs based on training data, which may include annotated images of hand-drawn or printed fonts and their corresponding center lines. The machine learning model can be trained on an annotated example dataset containing a variety of glyphs and font center lines, utilizing an iterative refinement process to converge to a stable representation of the center lines. This may involve presenting the model with a series of points representing potential candidate points for the center lines, which can then be refined through a series of adjustments until convergence is achieved, allowing the model to learn the patterns and features that define the center lines of each letter.

[0015] In some cases, the same or different machine learning models can be used to determine a continuous mapping of brush head shapes associated with one or more letter glyphs that will be used to sweep along a specified centerline. The continuous mapping of brush head shapes can vary continuously depending on their position on the centerline they are rendered from. In other words, the continuous mapping of brush head shapes can be a continuous mapping from the centerline position to a stamp, which can be a raster image or a parametric vector shape.

[0016] Machine learning models can learn to predict a range of brush head shapes to accurately capture the subtle differences and features of each glyph's outline shape. This process may involve learning from multiple predefined outline shapes across various fonts. Outline shapes can be parametric representations of the overall shape and structure of a font, defining the composition and layout of the letter glyphs. In some cases, a successive mapping of brush head shapes follows a corresponding tapering associated with the outline shape, determined by the original designer of the respective font. In other words, a successive mapping of brush head shapes can expand and / or contract at the correct locations to conform to the shape of the underlined glyph. For example, a font might have a brush-like effect, and the brush head shapes could create outline shapes by reflecting this effect with splatter-like patterns.

[0017] In some cases, machine learning models can learn to determine a continuous mapping of the centerline and brush head shape of alphanumeric or handwritten glyphs by training on a dataset containing annotated examples (represented as images or glyphs) of individual letter glyphs. By learning to manipulate this contour shape through subtle variations in stroke width, curvature, and other features, the model can effectively output a continuous mapping of the brush head shape. Therefore, in some cases, through an iterative refinement process guided by supervised and reinforcement learning techniques, machine learning models can produce continuous mappings of the brush head shape that accurately capture the essence of each letter shape.

[0018] In other words, the delivery of a font within an animation framework may not be related to traditional font files, but rather to a calculated centerline, the alphabetical stroke order, and a continuous mapping of the brush head shape used to sweep along that centerline to generate one or more glyphs. In some cases, the alphabetical stroke order also includes the direction of the strokes.

[0019] In some cases, the order of stroke paths is important if the font has more complex alphabetic forms or unique nuances. For example, when a font is created with watercolor brushes, if there is overlap and a later stroke overflows where it intersects with a previous stroke, a machine learning model can map the order of stroke paths when determining the stroke order of the alphabetic forms. As another example, if the font has an outline, to maintain the outline's characteristics throughout animation, when two strokes intersect, the next brush head shape can be used to correct for previous outlines that are no longer relevant after the intersection. Therefore, the order of stroke paths will determine how the intersecting brush head shapes will appear.

[0020] In some cases, the stroke order of letter forms can be determined based on well-known idiomatic handwriting patterns. For example, it is well known that for the letter "a", the circle is drawn first, followed by the tail. However, for certain fonts, the stroke order of letter forms may need to be determined by a machine learning model that uses a pre-trained set of annotated examples of letter form stroke orders that include the corresponding features to determine the stroke order of the corresponding letter glyphs.

[0021] In some cases, a custom interface can be provided. A custom interface can contain multiple controls for customizing the center line, the continuous mapping of the brush head shape, and / or the order of the strokes in the letterforms. The custom interface can serve as a starting point for editing or creating new fonts. For example, a font itself may not have an italic version. However, by defining the center line, the brush head shape can be changed or moved to create an italic version of the font. The thickness of the brush head shape can be changed to alter the thickness of the letterforms; furthermore, the center line itself can be moved to change the shape of the letterforms.

[0022] In some cases, after drawing letter glyphs, they can be transformed into other shapes or fonts by changing the parameters of the continuous mapping of the brush head shape. In some cases, each letter glyph can be a different font, and the interpolation between one letter glyph and another can be based on a smooth transition generated using a subset of brush head shapes.

[0023] In addition, various effects can be applied to the animation metadata, including center lines, stroke order of letterforms, and continuous mapping of brush head shapes, to further refine the letterforms. For example, shapes or letterform thickness can be applied after the animation metadata is generated as effects. These effects can be applied before the animation is rendered to modify or add elements to the letterforms. Some effects may need to be performed when generating the animation metadata. For example, applying sliders (such as italic sliders that can change the slant of letterforms) might also alter the center line, and therefore should be performed when generating the animation metadata.

[0024] Furthermore, in some cases, 2D letterforms can be transformed into 3D letter animations. One aspect to consider is the positioning of the letterforms in 3D space. To avoid complete intersection of letter strokes, the center line can be slightly shifted in 3D space, slightly forward or backward along the Z-axis, to prevent conflicts with other parts of the design. This process can be automated using heuristic algorithms that consider the geometry of the font and the center line, thus achieving more precise control over the 3D letterforms.

[0025] More specifically, machine learning models can be integrated as part of a content editing service that combines metadata associated with each layer in a hierarchical structure of a collaborative digital video editing service with letter animation modifiers guided by different features stored in association with individual layers. Animated implementations of one or more letter glyphs can be provided by the letter animation modifier, which can provide a set of one or more input layers as guidance for generating the animation implementation. Input layers can be selected layers used as guides for modifiers (e.g., letter animation modifiers). Each individual layer is a stackable, nested container for content added to an interactive editing interface, a graphical interface on a client device where users can manipulate and edit the visual aspects of content files.

[0026] Modifiers such as letter animation modifiers can be used to provide a variety of non-destructive editing protocols for processing image data and arranging them in a pipeline. A content file can use more than one modifier, and the output of a previous modifier in the pipeline can be treated as processed to export a certain output, which is then passed to the next modifier in the pipeline or rendered to the screen.

[0027] This type of integration provides users with combined control over how different features are represented in the output computer-generated content. The computer-generated content can be achieved by animates of one or more letter glyphs generated by a letter animation modifier. The computer-generated content can be 2D or 3D animation.

[0028] Therefore, this technology addresses current problems in the art by providing an integrated system that offers users a customizable, non-destructive, and holistic approach to content editing and creation. In this way, the technology reduces each of the multiple action steps, thereby reducing the burden on the computing system 700 (by requiring less navigation and rendering of the interface) and the user's burden (by reducing the number of steps they need to perform). Furthermore, this technology addresses current problems in the art by applying non-destructive editing holistically to the entire workflow of content creation and editing.

[0029] Figure 1An example system 100 is shown, configured to support non-destructive content editing and creation based on a hierarchical structure. Specifically, system 100 supports multiple user accounts interacting with each other within their respective communities.

[0030] System 100 illustrates an example architecture in which a user account interacts through an instance of client application 104 running on client device 106. Client application 104 may be provided by a webpage rendered in a web browser or by a downloaded client application executed by the operating system of a computing device. In some embodiments, client application 104 may provide several different sets of features or functionalities depending on the capabilities of the environment in which it is executed or rendered.

[0031] System 100 also includes a collaborative content editing service 102, which provides the infrastructure to support multiple user accounts in collaboratively creating and editing hierarchical content files. Content files can be self-contained sets of files created within the collaborative content editing service. The collaborative content editing service 102 can be a distributed service hosted in a cloud computing architecture. The collaborative content editing service 102 can be responsible for hosting various services of the client application 104 that are accessible to user accounts.

[0032] like Figure 1 As shown, the collaborative content editing service 102 provides a content manipulation service 114 that propagates state changes based on edits made to the content file by the client application 104. In some cases, such changes may propagate to the graphics engine service 120, which applies various features (such as filters and modifiers) to the content file. In some cases, when the content file is shared between two or more user accounts, such changes may propagate to the collaboration service 116 to spread them to other instances of the client application 104 associated with the user account. In some cases, such changes may propagate to the interface state management service 118, which assists in rendering and managing different states.

[0033] User accounts can operate their client application 104 instance to create or edit content files on an interactive editing interface via collaborative content editing service 102. The interactive editing interface can be a graphical interface on the client device where users can manipulate and edit the visual aspects of the content files. In some embodiments, this is performed by the client application 104 calling API layer 108 to request the creation or editing of content files. API layer 108 can then interact with internal services (such as content manipulation service 114, collaboration service 116, and interface state management service 118) and external services to create or edit content files. When a content file is created, the user account that created the content file can be considered the owner and / or administrator of the content file and can grant other user accounts permission to collaboratively process the content file. Content manipulation service 114 can use data service 110 to record information about the content files to store this information in database 112 and / or cloud service 122.

[0034] As described above, the collaborative content editing service 102 provides holistic, non-destructive editing throughout the workflow. To provide this non-destructive editing, layers are used to render the final representation of the content file while maintaining the independence and adjustability of all editing parameters. Layers can be simple, stackable, nested containers for content added to the interactive editing interface. Layers can be defined by various metadata, such as identifiers, parent layer identifiers, the order in which the layer appears in the parent layer's list of child layers, user-defined names, and applied transformations.

[0035] Modifiers can be various non-destructive editing concepts responsible for processing image data, arranged in a pipeline. They typically take the output of the previous modifier in the pipeline as input, process it to derive some output, and then pass the output to the next modifier in the pipeline or render it to the screen. Modifiers can include filter modifiers for implementing a range of filter types, such as color transformations (e.g., Hue / Saturation / Luminosity (HSL), Levels / Curves, Black & White (B&W) conversion, Shadow / Highlight adjustment, Color remapping, etc.) and convolutions (e.g., Blur, Sharpen, Edge detection, etc.). In some cases, the implementation of a filter modifier can be a fragment shader.

[0036] Furthermore, modifiers can animate certain features of the content file. In some cases, an animated modifier might include a letter animation modifier, which animates one or more letter glyphs by progressively revealing a continuous mapping of the brush head shape. This continuous mapping of the brush head shape can change continuously based on its position on the center line of its rendering. In other words, the continuous mapping of the brush head shape can be a continuous mapping from the center line position to the stamp, which can be a raster image or a parametric vector shape.

[0037] Depending on the layer order, in some cases, one or more animation features of the content file can be used as input to a letter animation modifier, causing the output frame with the animated features to be modified based on the letter animation modifier. In other cases, if the input contains other layers that modify the letter glyphs, such as changing their color or texture in certain areas, these modifications can also be output by the letter animation modifier.

[0038] As described above, the interface state management service 118 supports various functions related to state management and rendering the user interface. In some cases, the interface state management service 118 models the state of a content file that the user is editing in an interactive editing interface in the form of a content graph. The content graph can be a data structure and model representing the content and the relationships between different layers of the content file.

[0039] In some cases, the interface state management service 118 can also load bitmap data associated with the imported content, which can be stored in the database 112 or the cloud service 122. A fundamental function required by the collaborative content editing service 102 is to enable users to import content for use in their projects. Therefore, the interface state management service 118 assists in importing and storing binary bitmap data.

[0040] Furthermore, the interface state management service 118 can interface with the collaboration service 116 to synchronize document state with other users editing the same content file. The interface state management service 118 can also interface with the graphics engine service 120 by providing a content file for rendering. Additionally, the interface state management service 118 can directly capture and process user input. For example, the interface state management service 118 can respond to interactions with layers or toolbars presented in the interactive editing interface. However, in situations requiring low latency response times (e.g., dragging layers or drawing within layers), such input events can be directly pipelined to the graphics engine service 120 for processing.

[0041] As described above, the graphics engine service 120 supports various functions related to rendering graphical content. For example, the graphics engine service 120 can receive a content graph from the interface state management service 118 and convert it into a processor graph. A processor graph can be an abstract representation of the GPU and CPU operations and data flows required to render the content file. In some cases, the graphics engine service 120 evaluates the processor graph to visualize the content file on an interactive editing interface.

[0042] The collaborative content editing service 102 also provides a collaboration service 116, which can interface with a multi-user state management library. In some cases, the collaboration service 116 interfaces with the interface state management service 118 to use the multi-user state management library to synchronize content files with other users.

[0043] In some embodiments, the collaborative content editing service 102 is configured such that most communication between the collaborative content editing service 102 and the client application 104 passes through the API layer 108. The client application 104 can request responses from the API layer 108 to various services provided by the collaborative content editing service 102. Furthermore, services within the collaborative content editing service 102 can communicate with each other by sending messages via the API layer 108. In some cases, the API layer 108 includes a WebSocket server, which acts as a hub for the client application 104 to share information about the state of content files and their state changes. Additionally, the API layer 108 can assist in managing collaborative sessions by supporting extensions, thereby allowing the collaborative content editing service 102 to hook into the editing process for storage, metrics, logging, and management synchronization.

[0044] Although the collaborative content editing service 102 is described with respect to a specific system architecture and communication process, those skilled in the art will understand that other system configurations are also feasible. Furthermore, the collaborative content editing service 102 may not need to be collaborative, but may simply be a content editing service providing such modifiers (e.g., letter animation modifiers).

[0045] Figure 2 An example collaborative content editing service configured to support user accounts in creating and managing collaborative animated content files is shown, based on some aspects of this technology.

[0046] In some cases, the content file can be an animation file and / or contain animated features. Therefore, to generate animated features, multiple animation frames can be generated. Multiple animation frames can be generated for display, and in some cases, the animation frames can loop for several seconds, which can be displayed continuously in the preview interface. In some cases, looping animation frames can be displayed in the preview interface, and changes made in the interactive editing interface can be displayed in real-time or near real-time along with the interactive editing interface in the preview interface. Alternatively, the interactive editing interface can also be the preview interface, so that the animation plays in the same frame that is receiving changes.

[0047] In some cases, animated content files may include one or more layers with animated features, which are configured as input to a letter animation modifier, such that each frame of one or more animated layers can be generated by the letter animation modifier.

[0048] Although the collaborative content editing service 102 is described with respect to a specific system architecture and communication flow, those skilled in the art will understand that other system configurations are possible. For example, the letter animation modifier service 208 could be a service independent of the graphics engine service 120.

[0049] Figure 3 Example method 300 for rendering an animation of one or more letter glyphs according to some aspects of the present technology is shown. Although example method 300 describes a specific order of operations, this order can be changed without departing from the scope of this disclosure. For example, some of the described operations can be performed in parallel or in a different order without substantially affecting the functionality of method 300. In other examples, different components of the example device or system implementing method 300 can perform their functions substantially simultaneously or in a specific order.

[0050] In some cases, the runtime system of a content editing service (such as collaborative content editing service 102) can perform one or more of these steps. The runtime system can be a client runtime system, a server runtime system, or a combination of both.

[0051] According to some examples, the method includes, in step 302, receiving one or more letter glyphs. The one or more letter glyphs can be any letter glyph or ideographic character. In some cases, the letter animation modifier service 208 may receive one or more letter glyphs. In some cases, one or more glyphs may be received in various formats, including but not limited to: vector graphics (e.g., SVG (Scalable Vector Graphics), EPS (Encapsulated PostScript), AI (Adobe Illustrator)), raster images (e.g., JPEG, PNG, GIF, BMP, TIFF), scanned images (e.g., scanned PDFs, OCR (Optical Character Recognition) processed images), font files (e.g., TTF (TrueType Fonts), OTF (OpenType Fonts), WOFF (Web Open Font Format)), 3D models (e.g., STL (Stereolithography), OBJ (Object Files)), embedded formats (e.g., embedded images in documents (DOCX, PDF), embedded images in presentations (PPTX)), handwritten or printed samples (e.g., physical samples scanned or photographed into digital format), or digital screenshots (e.g., screenshots taken from digital displays or applications).

[0052] Once received, the letter animation modifier service 208 may first process the received media to extract one or more letter glyphs into a format, such as a vectorized format, that indicates the outline shape of each letter glyph.

[0053] According to some examples, the method includes: in step 304, using a machine learning model to determine animation metadata from one or more glyphs. In some cases, the letter animation modifier service 208 can determine the animation metadata. In some cases, the animation metadata includes a continuous mapping of a center line and a brush shape used to sweep along the center line to generate an animation implementation of one or more glyphs. In some cases, the animation metadata includes the stroke order of the letter forms. In some cases, the continuous mapping of the brush shape gradually reveals itself according to the stroke order of the letter forms. In some cases, the stroke order of the letter forms is established by a machine learning model. In other cases, the stroke order of the letter forms can be determined retrospectively.

[0054] In some cases, a machine learning model may be given one or more glyphs with novel outline shapes and may need to determine the centerlines of these glyphs. The machine learning model can determine the centerlines of a set of glyphs based on training data, which may include annotated images of hand-drawn or printed fonts and their respective centerlines. The machine learning model can be trained on an annotated example dataset containing various glyphs and font centerlines, utilizing an iterative refinement process to converge to a stable representation of the centerlines. This may involve presenting the model with a series of points representing potential candidate points for the centerlines, and then refining these points through a series of adjustments until convergence is achieved, enabling the model to learn the patterns and features that define the centerline of each letter.

[0055] In some cases, the same or different machine learning models can be used to determine a continuous mapping of the brush head shape associated with one or more letter glyphs that will be used to sweep along a specified centerline. In other cases, if different machine learning models are used, each model may receive the centerline and outline shape of each glyph to determine a continuous mapping of the brush head shape.

[0056] In either case, the machine learning model can learn to predict a series of brush shapes that accurately capture the subtle differences and features of the outline shape of each glyph. This process may involve learning from multiple predefined outline shapes across various fonts. Outline shapes can be parametric representations of the overall shape and structure of a font, defining the composition and layout of the letter glyphs. In some cases, a successive mapping of brush shapes follows a corresponding taper associated with the outline shape, determined by the original designer of the respective font. In other words, a successive mapping of brush shapes expands and contracts at the correct locations to match the shape of the underlined glyph. For example, a font might have a brush-like effect, and the brush shapes could reflect this effect through splatter-like patterns, thus creating the outline shapes.

[0057] In some cases, machine learning models can learn to determine a continuous mapping of the centerline and brush shape of alphanumeric or handwritten glyphs by training on datasets containing annotated examples of individual glyphs (represented as images or glyphs). By learning to manipulate this contour shape through subtle variations in stroke width, curvature, and other features, the model can effectively output a continuous mapping of the brush shape. Therefore, in some cases, through an iterative refinement process guided by supervised and reinforcement learning techniques, machine learning models can produce continuous mappings of the brush shape that accurately capture the essence of each letter's form.

[0058] The order of stroke paths is important when the font has complex alphabetic forms or unique nuances. For example, if the font is created with watercolor brushes, and at overlaps, a later stroke might overflow where it intersects with a previous stroke, a machine learning model can map the stroke path order in a way that aligns with the contours of each brush head shape when determining the stroke order of the alphabetic forms. As another example, if the font has a shape, to maintain shape characteristics throughout animation, when two strokes intersect, the next brush head shape can be used to correct the previous shape, which is no longer relevant after the intersection. Therefore, the order of stroke paths can determine how intersecting brush head shapes will look for certain alphabetic forms.

[0059] According to some examples, the method includes, in step 306, rendering an animated implementation of one or more letter glyphs based on a progressively revealing map of the brush head shape. In some cases, animation metadata may be used to render the animated implementation on client device 106. In other cases, frames of the animated implementation of the progressively revealing map of the brush head shape are generated and rendered on client device 106.

[0060] Figure 4 Example method 400 for training a machine learning model to generate animations of one or more letter glyphs, according to some aspects of the present technology, is illustrated. Although example method 400 describes a specific order of operations, this order can be changed without departing from the scope of this disclosure. For example, some of the described operations can be performed in parallel or in a different order without substantially affecting the functionality of method 400. In other examples, different components of the example device or system implementing method 400 may perform their functions substantially simultaneously or in a specific order.

[0061] According to some examples, the method includes: in step 402, training a machine learning model based on a first set of pre-trained data, which includes annotated centerline examples of corresponding letter glyphs. In some cases, the corresponding features and centerlines may be annotated by human experts and / or automated tools.

[0062] According to some examples, the method includes: at block 404, training a machine learning model to determine a continuous mapping of the brush head shape for the corresponding glyph. This training may be based on a second set of pre-trained data, which includes annotated examples of continuous mappings of brush head shapes defined by the contour shapes of the corresponding features. In some cases, the continuous mapping of brush head shapes is defined by the contour shapes of the corresponding glyphs. In other cases, the annotated examples of continuous mappings of brush head shapes may be annotated by human experts and / or automated tools.

[0063] According to some examples, the method includes: at block 406, training a machine learning model to determine the stroke order of the letter forms of the corresponding glyphs. In some cases, this training is based on a third set of pre-trained data, which includes annotated examples of the stroke order of the letter forms for the corresponding features. In some cases, the annotated examples of the stroke order of the letter forms for the corresponding features can be annotated by human experts and / or automated tools. In some cases, the stroke order of the letter forms can be determined based on well-known and idiomatic handwriting forms. However, for certain fonts, it may be necessary to determine the stroke order of the letter forms using a machine learning model that is trained to determine the stroke order of the letter forms of the corresponding glyphs based on a third set of pre-trained data, which contains annotated examples of the stroke order of the letter forms for the corresponding features.

[0064] and Figure 3 Similarly, according to some examples, the method may include: in step 408, receiving one or more glyphs. According to some examples, the method includes: using a machine learning model to determine animation metadata from one or more glyphs. In some cases, in step 410, the animation metadata may include a center line, the stroke order of the letter forms, and a continuous mapping of the brush head shape used for sweeping along the center line to generate the animation of one or more glyphs.

[0065] According to some examples, the method includes: in step 412, sending animation metadata that generates an animated implementation of one or more letter glyphs based on a successive mapping that gradually reveals the shape of the brush head.

[0066] Figure 5 An example graphical user interface depicting an animated playback of a set of letter glyphs in a first instance is shown, according to some aspects of the present technology.

[0067] The example graphical user interface 500 includes an interactive editing interface 502, which can receive edits of one or more letter glyphs and can also be used as a preview interface to display a playback of the animation. Alternatively, a separate preview interface can display the playback.

[0068] In some cases, the animation implementation of one or more letter glyphs based on a continuous mapping that gradually reveals the shape of the brush head can be displayed as a continuous loop of animation frames over a period of time.

[0069] As shown in layer interface 504, the letter animation modifier service 208 is applied to the sub-layer "LIG HT", but it can also be applied to a set of layers. In other words, the features of the sub-layer "LIG HT" can be input through the letter animation modifier service 208. The features extracted from the sub-layer can include the letter glyphs "L", "I", "G", "H", "T" and font metadata. In some cases, the letter glyphs "L", "I", "G", "H", "T" and / or font metadata can be used as input to the machine learning model of the letter animation modifier service 208.

[0070] The customizable interface 512 can also be provided as a panel, offering more insights and customization options for the selected letter glyphs. For example, in Figure 5 In the custom interface 512, "Content" includes letter glyphs "L", "I", "G", "H", and "T", the font "Roboto Mono", the variant "700 Bold", font size, spacing, etc. Some customizations can be performed before the animation is generated, while others can be customized in real time after the animation is generated.

[0071] like Figure 5 As shown, a preview of the animation implementation is displayed. In the first instance, "L" and "I" have already been displayed, and "G" is being displayed, aligned with the timeline displayed in the player interface 506. The continuous mapping of the brush head shape can be swept along the center line 510 (as shown). Figure 5 (As shown, but may or may not be shown to the user).

[0072] Figure 6 An example graphical user interface according to certain aspects of the present technology is shown, which depicts the animated playback of a set of letter glyphs in a second instance.

[0073] In the second example, the preview of the animation shows all the displayed letter glyphs, and... Figure 6 The timeline alignment is shown in player interface 506. While the font is quite simple, one can imagine that more complex fonts would have more complex brush strokes to sweep along the center line. In some cases, playback is controlled by player interface 506.

[0074] In some cases, providing a custom interface 512 allows for customization of at least one of the following: center line, continuous mapping of brush head shape, or stroke order of letter forms. By allowing customization of letter glyphs, the original font of letter glyphs can be transformed into a variable font. For example, a font itself may not have an italic version, but by defining the center line, the brush head shape can be changed or moved to create an italic version of the font.

[0075] In some cases, the thickness of the brush head shape can be changed to alter the thickness of the letter glyphs. In other cases, changing the thickness may change the appearance of pressure on the brush. In still others, each letter glyph can be a different font, and interpolation between one letter glyph and another can be based on a smooth transition generated using a subset of brush head shapes.

[0076] In some cases, after establishing the center line 510, the center line 510 can be further manipulated to change the shape of the letter glyphs. The brush head shape can also be changed, which may alter aspects such as thickness. In some cases, the user can choose to completely change the brush head shape. Such users can use the custom interface 512 as a starting point for editing or creating new fonts. In this case, the brush head shape can be morphed into other shapes or fonts by changing the parameters of the continuous mapping of the brush head shape. In some cases, the stroke order of the letter forms can be a random order, completing the letters non-continuously, selecting strokes in height order, or other types of custom rules determined by the user or the model.

[0077] In addition, different effects can be applied to the animation metadata, including continuous mapping of the center line, letter stroke order, and brush head shape, to further refine the letter glyphs. For example, the shape or thickness of the letter glyphs can be applied after the animation metadata has been generated as effects. These effects can be applied before the animation implementation is rendered to change or add elements to the letter glyphs. Some effects may need to be performed when generating the animation metadata. For example, applying an italic slider that can change the slant of the letter glyphs will also change the center line and therefore should be performed when generating the animation metadata. In some cases, users can use the custom interface 512 to select a new starting point for a new letter stroke order.

[0078] In some cases, custom features may allow one or more glyphs to be drawn in parallel, and the word may not be read from the first letter to the last. Furthermore, the center lines of each glyph may not intersect, or in some cases they may intersect, and no brush head shape may be visible in areas not belonging to the glyphs.

[0079] In some cases, 2D letter shapes can be transformed into 3D letter animations. One aspect to consider is the positioning of the letter shapes. To avoid the strokes of the letter shapes intersecting completely, the center line can be slightly shifted in 3D space, slightly forward or backward along the Z-axis, to prevent conflicts with other parts of the design. This process can be automated by considering heuristics of font geometry and center lines, allowing for more precise control over the 3D letter form.

[0080] Finally, since the center line 510 has been established, the brush shape itself can transform into other shapes or fonts arranged around the center line during animation. In other words, in some cases, once the letterform is fully revealed, it can transform into another shape or font with the same center line 510. Animation can also allow the center line to move and transform into other shapes, providing a smooth and dynamic way to animate letterforms. Other decorations, ornaments, or objects can also be rendered relative to the center line. For example, bolts can be drawn along the center line 510 of one or more letterforms. Therefore, other geometries can be created and placed relative to the center line 510. Such renderings can be stacked together, and the order of such renderings can be controlled, either as layers or as other types of parameters. For example, the watercolor brush example and shape example mentioned above can be stacked together.

[0081] Figure 7 An example of computing system 700 is shown, which can be any computing device, such as constituting collaborative content editing service 102 or client device 106, or any component in which system components communicate with each other using connection 702. Connection 702 can be a physical connection via a bus or a direct connection to processor 704 (e.g., in a chipset architecture). Connection 702 can also be a virtual connection, a network connection, or a logical connection.

[0082] In some embodiments, the computing system 700 is a distributed system, wherein the functions described herein may be distributed across a data center, multiple data centers, a peer-to-peer network, etc. In some embodiments, one or more of the described system components represent a plurality of such components, each performing some or all of the functions described therein. In some embodiments, these components may be physical devices or virtual devices.

[0083] The example computing system 700 includes at least one processing unit (CPU or processor) processor 704 and a connection 702 that couples various system components, including system memory 1108 (e.g., read-only memory (ROM) 710 and random access memory (RAM) 712), to the processor 704. The computing system 700 may include a cache of the high-speed memory 708 that is directly connected to, closely connected to, or integrated into the processor 704.

[0084] Processor 704 may include any general-purpose processor and hardware or software services, such as services 716, 718, and 720 stored in storage device 714, which are configured to control processor 704. It may also include dedicated processors, where software instructions are incorporated into the actual processor design. Processor 704 can essentially be a completely independent computing system, containing multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors can be symmetric or asymmetric.

[0085] To enable user interaction, the computing system 700 includes an input device 726, which can represent any number of input mechanisms, such as a microphone for voice, a touchscreen for gesture or graphical input, a keyboard, a mouse, motion input, voice, etc. The computing system 700 may also include an output device 722, which can be one or more of a variety of output mechanisms known to those skilled in the art. In some cases, a multi-mode system allows the user to provide multiple types of input / output to communicate with the computing system 700. The computing system 700 may include a communication interface 724, which typically controls and manages user input and system output. There are no limitations on operation on any particular hardware configuration; therefore, the basic functionality described here can be easily superseded by improved hardware or firmware configurations, as these are under development.

[0086] Storage device 714 may be a non-volatile memory device and may be a hard disk or other type of computer-readable medium that can store computer-accessible data, such as magnetic tape, flash memory card, solid-state memory device, digital multifunction disk, magnetic tape cassette, random access memory (RAM), read-only memory (ROM), and / or some combination of these devices.

[0087] Storage device 714 may include software services, servers, etc., which, when executed by processor 704, enable the system to perform functions. In some embodiments, hardware services performing specific functions may include software components stored in a computer-readable medium that are connected to hardware components (e.g., processor 704, connection 702, output device 722, etc.) to perform that function.

[0088] Figure 8An example neural network architecture according to certain aspects of the present technology is illustrated. Architecture 800 includes a neural network 810 defined by an example neural network description 820 in a rendering engine model (neural controller) 830. Neural network 810 may represent a neural network implementation of a rendering engine used to render media data. Neural network description 820 may include a complete specification of neural network 810, including neural network architecture 800. For example, neural network description 820 may include a description or specification of the architecture 800 of neural network 810 (e.g., layers, layer interconnections, number of nodes per layer, etc.); input and output descriptions indicating how inputs and outputs are formed or processed; indications of activation functions, operations or filters, etc., in the neural network; neural network parameters, such as weights, biases, etc.; and so on.

[0089] Neural network 810 reflects the architecture 800 defined in neural network description 820. In this example, neural network 810 includes neural network input layers 802a to 1402n (collectively referred to as "802"), which contain input data, such as... Figure 2 and Figure 3 The input layer described herein. In an exemplary example, the input layer may contain data representing a portion of the input media data, such as data related to the input media data (e.g., ...). Figure 2 and Figure 3 The data in the input layer described above corresponds to a block of data or pixels in the image (e.g., a 128x128 data block). For clarity, at least in Figure 2 and Figure 3 The term "input layer" described in the text is similar to... Figure 8 The input layer of the neural network described in the text is different.

[0090] Neural network 110 includes neural network output layers 806a to 1404n (collectively referred to as "1404"). The neural network hidden layers 804 may include a number of n hidden layers, where n is an integer greater than or equal to 1. The number of hidden layers may include the number of layers depending on the desired processing outcome and / or rendering intent. Neural network 810 also includes an output layer that provides the output (e.g., computer-generated content) produced by the processing performed by the hidden layers 104. In an exemplary example, neural network output layer 806 may provide one or more output layers of computer-generated content.

[0091] The neural network 810 in this example is a multi-layer neural network composed of interconnected nodes. Each node can represent a piece of information. The information associated with a node is shared between different layers, and each layer retains the information as it is processed. In some cases, neural network 810 may include a feedforward neural network, in which case the neural network has no feedback connections, and the output of the neural network is fed back to itself. In other cases, neural network 810 may include a recurrent neural network, which can have loops that allow information to be transferred across nodes while the input is being read.

[0092] Nodes can exchange information through inter-node interconnects between layers. A node 808a in the neural network input layer 802 can activate a set of nodes 808b in the first neural network hidden layer 804a. For example, as shown, each input node 808a of the neural network input layer 802a is connected to each node of the first neural network hidden layer 804a. Nodes 808b in the neural network hidden layer 804b can transform the information of each input node by applying activation functions to the information. The information obtained from the transformation can then be passed to nodes in the next hidden layer (e.g., 1404b) and can activate these nodes, which can perform their respective specified functions. Example functions include convolution, upsampling, data transformation, pooling, and / or any other suitable function. The output of the hidden layer (e.g., 1404b) can then activate nodes in the next hidden layer (e.g., 1404n), and so on. The output of the last hidden layer can activate one or more nodes in the neural network output layer 806, providing the output at this point. In some cases, although nodes in neural network 810 (e.g., nodes 1408a, 1408b, 1408c) appear to have multiple output lines, a node has only a single output, and all lines shown as outputs from a single node represent the same output value.

[0093] In some cases, each node or the interconnection between nodes can have weights, which are a set of parameters derived by training the neural network 810. For example, an interconnection between nodes can represent a piece of information learned about the interconnected nodes. The interconnections can have adjustable numerical weights (e.g., based on the training dataset), allowing the neural network 810 to adapt to the input and learn as more data is processed.

[0094] The neural network 810 can be pre-trained to process features from data in the neural network input layer 802 using different hidden layers 804, thereby providing output through the neural network output layer 806. In an example where the neural network 810 is used to generate computer-generated content, it can be trained using training data containing a diverse set of images covering a range of features and styles, allowing patterns, textures, and structures to be used as examples to generate novel images. For example, training images can be input into the neural network 810, which can process these images and generate outputs that can be used to adjust one or more aspects of the neural network 810, such as weights, biases, etc.

[0095] In some cases, the neural network 810 can use a training process called backpropagation to adjust the weights of its nodes. Backpropagation can include forward pass, loss function, backward pass, and weight update. Forward pass, loss function, backward pass, and parameter update are performed in one training iteration. For each set of training media data, this process can be repeated a certain number of iterations until the weights of each layer are accurately tuned.

[0096] For the first training iteration of the Neural Network 810, since the weights are randomly chosen during initialization, the output may contain values ​​that do not prioritize any particular category. For example, if the output is a vector with probabilities that an object contains different products and / or different users, the probability values ​​for each different product and / or user may be equal or at least very similar (e.g., for ten possible products or users, the probability value for each category might be 0.1). Using the initial weights, the Neural Network 810 cannot determine low-level features and therefore cannot accurately determine the possible classifications of the objects. A loss function can be used to analyze the error in the output. Any suitable loss function can be defined.

[0097] For the first training dataset (e.g., images), the loss (or error) may be high because the actual values ​​will differ from the predicted output. The goal of training is to minimize the amount of loss so that the predicted output matches the target or ideal output. The Neural Network 810 can perform backpropagation by determining which inputs (weights) contribute most to the loss of the Neural Network 810, and the weights can be adjusted to reduce and eventually minimize the loss.

[0098] The derivative of the loss with respect to the weights can be calculated to determine the weights that contribute the most to the loss of the neural network 810. After calculating the derivative, weight updates can be performed by updating the filter weights. For example, the weights can be updated so that they change in the opposite direction of the gradient. The learning rate can be set to any suitable value; a higher learning rate indicates larger weight updates, and a lower learning rate indicates smaller weight updates.

[0099] Neural Network 810 can include any suitable neural network or deep learning network. One example includes a Convolutional Neural Network (CNN), which consists of an input layer and an output layer, with multiple hidden layers between them. The hidden layers of a CNN include a series of convolutional layers, non-linear layers, pooling layers (for downsampling), and fully connected layers. In other examples, Neural Network 810 can represent any other neural network or deep learning network, such as an autoencoder, a Deep Belief Network (DBN), a Recurrent Neural Network (RNN), etc.

[0100] For ease of explanation, in some cases, this technology may be presented as comprising individual functional blocks, including functional blocks constituting a device, device components, steps or routines in a method implemented in software or a combination of hardware and software.

[0101] Any step, operation, function, or process described herein may be performed or implemented by a combination of hardware and software services, or by services alone or in combination with other devices. In some embodiments, a service may be software residing in the memory of one or more servers of a client device and / or a content management system, which performs one or more functions when the processor executes the software associated with the service. In some embodiments, a service is a program or collection of programs that performs a specific function. In some embodiments, a service may be considered a server. The memory may be a non-transitory computer-readable medium.

[0102] In some embodiments, computer-readable storage devices, media, and memories may include wired or wireless signals containing bit streams, etc. However, when referring to non-transitory computer-readable storage media, media such as energy, carrier signals, electromagnetic waves, and the signals themselves are explicitly excluded.

[0103] The methods described in the examples above can be implemented using computer-executable instructions stored in or otherwise accessible from a computer-readable medium. Such instructions may include, for example, instructions and data that cause or otherwise configure a general-purpose computer, special-purpose computer, or special-purpose processing device to perform a particular function or group of functions. The computer resources used may be accessible via a network. The executable computer instructions may be, for example, binary files, intermediate format instructions (e.g., assembly language), firmware, or source code. Examples of computer-readable media that can be used to store instructions used, information used, and / or information created during the methods according to the examples include hard disks or optical disks, solid-state storage devices, flash memory, USB devices equipped with non-volatile memory, network storage devices, etc.

[0104] Devices implementing the methods according to this disclosure may include hardware, firmware, and / or software, and may employ various form factors. Typical examples of such form factors include servers, laptops, smartphones, minicomputers, personal digital assistants, etc. The functionality described herein may also be embodied in peripheral devices or add-in cards. Further examples include implementations between different chips on a circuit board or between different processes executing within a single device.

[0105] Instructions, media for conveying such instructions, computing resources for executing such instructions, and other structures for supporting such computing resources are means for providing the functionality described in this disclosure.

Claims

1. A computer-implemented method comprising: receiving one or more letter glyphs of a content file; determining, using a machine learning model, animation metadata from the one or more letter glyphs, the animation metadata comprising a centerline and a continuous mapping of a brush head shape for panning along the centerline to generate an animated realization of the one or more letter glyphs; and sending the animation metadata, the animation metadata generating the animated realization of the one or more letter glyphs based on a gradual reveal of the continuous mapping of the brush head shape.

2. The computer-implemented method of claim 1, wherein, the animation metadata comprising a letterform stroke order, and wherein the continuous mapping of the brush head shape gradually reveals in the letterform stroke order.

3. The computer-implemented method of claim 2, further comprising: providing a customization interface, the customization interface comprising a plurality of controls for customizing at least one of the centerline, the continuous mapping of the brush head shape, or the letterform stroke order.

4. The computer-implemented method of claim 3, further comprising: displaying at least one of the centerline, the continuous mapping of the brush head shape, or the letterform stroke order; and receiving, from the customization interface, a customization change to at least one of the centerline, the continuous mapping of the brush head shape, or the order of the letterform stroke order.

5. The computer-implemented method of claim 4, wherein, the customization change comprising: receiving a new starting point for one of the letter glyphs to change the letterform stroke order; and determining, by the machine learning model, a new letterform stroke order based on the new starting point.

6. The computer-implemented method of claim 4, wherein, the customization change comprising changing at least one of the brush head shapes by changing a parameter on the customization interface.

7. The computer-implemented method of claim 4, wherein, the customization change comprising manipulating at least one of the centerline or the brush head shapes on an interactive editing interface, the interactive editing interface being a graphical interface in which visual aspects of the content file are edited.

8. The computer-implemented method of claim 4, wherein, the customization change comprising applying an effect or a stackable render to the one or more letter glyphs after generating the animated realization.

9. The computer-implemented method of claim 8, wherein, the effect being transforming the one or more letter glyphs into 3D letterforms and adjusting the centerline in 3D space along a Z-axis.

10. The computer-implemented method of claim 2, further comprising: training the machine learning model to determine the letterform stroke order for a respective letter glyph based on a third set of pre-training data comprising annotated examples of letterform stroke orders of respective features.

11. The computer-implemented method of claim 1, further comprising: training the machine learning model to determine a centerline for a respective letter glyph based on a first set of pre-training data comprising annotated examples of centerlines of respective features.

12. The computer-implemented method of claim 1, further comprising: training the machine learning model based on a second set of pre-training data comprising annotated examples of continuous mappings of brush head shapes defined by outline shapes of respective characters, wherein the continuous mappings of brush head shapes are defined by outline shapes of respective letterforms, to determine the continuous mappings of brush head shapes for the respective letterforms.

13. A computing device comprising: one or more processors; and memory to store instructions that, when executed by the one or more processors, configure the computing device to: receive one or more letterforms; determine, using a machine learning model, animation metadata for the one or more letterforms, the animation metadata comprising a centerline, a continuous mapping of a brush head shape for swiping along the centerline, and a letterform stroke order for generating an animated realization of the one or more letterforms; and present the animated realization of the one or more letterforms based on revealing the continuous mapping of the brush head shape step-by-step in accordance with the letterform stroke order.

14. The computing device of claim 13, wherein, the animation metadata comprises the letterform stroke order, and wherein the continuous mapping of the brush head shape is revealed gradually in accordance with the letterform stroke order.

15. The computing device of claim 14, wherein, the instructions further configure the computing device to: provide a customization interface comprising a plurality of controls for customizing at least one of the centerline, the continuous mapping of the brush head shape, or the letterform stroke order.

16. The computing device of claim 14, wherein, the instructions further configure the computing device to: display at least one of the centerline, the continuous mapping of the brush head shape, or the letterform stroke order; and receive, from a customization interface, a customizing change to at least one of the centerline, the continuous mapping of the brush head shape, or the letterform stroke order.

17. The computing device of claim 16, wherein, the customizing change comprises: receiving a new starting point for one of the letterforms to change the stroke order; and determining, by the machine learning model, a new stroke order based on the new starting point.

18. The computing device of claim 16, wherein, the customizing change comprises changing at least one of the brush head shapes by changing a parameter on the customization interface.

19. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to: receive animation metadata input comprising a centerline and one or more letterforms; infer, using a runtime system of a content editing service, a continuous mapping of a brush head shape for swiping along the centerline to generate an animated realization of a set of letterforms based on the animation metadata input and the one or more letterforms; and generate the animated realization of the one or more letterforms by revealing the continuous mapping of the brush head shape step-by-step following a letterform stroke order.

20. The non-transitory computer-readable storage medium of claim 19, wherein, the revealing of the continuous mapping of the brush head shape step-by-step generates a set of animation frames that are displayed in a continuous loop over a period of time.

Images