Code-defined visual content in computing system

Abstract

The invention relates to code-defined visual content in a computing system. Embodiments of the present disclosure provide techniques for generating visual content in a design environment. An example method includes generating, on a canvas in a design environment, a layer of visual assets containing code definitions based on external code assets. Input associated with a modification to be applied to a code-defined visual asset in the layer is received. Based on the received input, the visual asset of the code definition in the layer is modified, and the modified visual asset of the code definition is rendered onto a canvas.

Description

Technical Field

[0001] Embodiments of this disclosure generally relate to graphic design tools, and more specifically, to techniques for generating and modifying visual content within graphic design tools. Background Technology

[0002] Graphic design tools typically allow the creation of visual assets within a design environment. These visual assets can be defined, for example, as vector assets based on mathematical relationships between different components, or as raster assets based on absolute pixel positions. In some cases, visual assets may include one or more containers defined in vector space, into which raster assets (e.g., images) can be inserted. Visual assets can contain any number of visual components and can be defined based on the relative positioning or other spatial relationships between the visual components within the visual asset.

[0003] In many workflows, the process of designing visual assets can be separated from the process of generating the code used to implement the visual assets in the executable environment outside the design environment. Because of this separation, visual assets can be designed in the design environment and coded in the coding environment. However, the design and coding environments may not support the same features, so a design created in the design environment may not be replicable in the coding environment. In other words, the design of a visual asset created in the design environment can serve as a guide for creating the visual asset in the coding environment, and the appearance of the visual asset created in the coding environment may not match the appearance of the visual asset in the design environment.

[0004] Furthermore, while the processes of designing visual assets and generating the code to implement them can be separated, the code generated in the executable environment to implement the visual assets may be tightly coupled to the appearance of the visual assets. Due to this coupling between appearance and implementation code, implementing multiple instances of a visual asset may involve copying code within the codebase that implements the visual asset. Therefore, to change each instance of the visual asset, it may be necessary to repeatedly modify the same code.

[0005] As mentioned above, we need more efficient techniques to implement visual content within executable code. Summary of the Invention

[0006] One embodiment of this disclosure provides techniques for generating visual content in a design environment. An example method includes: generating a layer containing a code-defined visual asset on a canvas in the design environment based on an external code asset; receiving input associated with modifications to be applied to the code-defined visual asset in the layer; modifying the code-defined visual asset in the layer based on the received input; and rendering the modified code-defined visual asset onto the canvas.

[0007] One embodiment of this disclosure provides techniques for modifying code-defined visual assets in a design environment. An example method includes importing the code-defined visual assets into a canvas within the design environment for editing. A request to modify the imported code-defined visual assets is received, and the code file associated with the code-defined visual assets is modified according to the received request. The canvas is updated based on how the code-defined visual assets are imported to reflect the modified code file.

[0008] Compared to existing technologies, a key advantage of this disclosure is that it allows visual assets to be defined in code and modified visually or in code within a design environment. Visual assets can be defined as code within external code assets (e.g., code files) and imported as instances of the visual asset into a layer of the canvas within the design environment. By importing the external code asset into a layer of the canvas to generate instances of the visual asset, these instances can inherit the prior-defined properties of the visual asset. When changes are made to the instance visual asset at a layer of the canvas, the code associated with the visual asset instance can be modified, thus allowing local modification of the visual asset instance without altering the external code asset. Furthermore, since the design and coding of visual assets can be more tightly coupled, the embodiments proposed herein allow the creation of visual assets in code to replicate the design of the visual asset rather than approximating a visual design created in design space. Attached Figure Description

[0009] To gain a detailed understanding of the features of the above embodiments, the inventive concept briefly outlined above can be described in more detail with reference to the embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings only show typical embodiments of the inventive concept and should not be construed as limiting the scope in any way; other equally effective embodiments exist.

[0010] Figure 1 A computer system configured to implement one or more aspects of the embodiments of this disclosure is shown.

[0011] Figure 2 This illustrates a design environment, according to some embodiments, in which visual components are created and maintained as code-defined assets.

[0012] Figure 3 The diagram illustrates a design environment according to some embodiments, as well as the links between visual components within the design environment and design files associated with the visual components.

[0013] Figure 4A and Figure 4BThis is a message flow diagram based on some embodiments, illustrating the messages exchanged between a code management engine and a graphical design engine for importing and modifying visual assets defined in the code within a design environment.

[0014] Figure 5 This is a flowchart of method steps for modifying visual assets defined in a code definition within a design environment based on external code assets, according to some embodiments.

[0015] Figure 6 This is a flowchart of method steps for interacting with code-defined visual assets in a design environment, according to some embodiments.

[0016] Figure 7 This is an example of a network computing system for implementing an interactive graphical application platform, according to some embodiments. Detailed Implementation

[0017] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the various embodiments. However, those skilled in the art will understand that the inventive concept can be practiced even without one or more of these specific details.

[0018] Figure 1 A computing device 100 configured to implement one or more aspects of various embodiments of the present invention is illustrated. In one embodiment, the computing device 100 includes a desktop computer, laptop computer, smartphone, personal digital assistant (PDA), tablet computer, or any other type of computing device configured to receive input, process data, and optionally display images, and is suitable for practicing one or more embodiments. The computing device 100 is configured to run a code management engine 122 and a graphics design engine 124 residing in memory 116.

[0019] It should be noted that the computing device described herein is exemplary, and any other technically feasible configuration falls within the scope of this disclosure. For example, multiple instances of the code management engine 122 or the graphics design engine 124 may execute on a set of nodes in a distributed and / or cloud computing system to implement the functionality of the computing device 100. In another example, the code management engine 122 or the graphics design engine 124 may execute on various hardware, device types, or environments to adapt the code management engine 122 or the graphics design engine 124 to different use cases or applications. In a third example, the code management engine 122 or the graphics design engine 124 may execute on different computing devices and / or different sets of computing devices.

[0020] In one embodiment, computing device 100 includes, but is not limited to: interconnect (bus) 112 connecting one or more processors 102, input / output (I / O) device interface 104 coupled to one or more input / output (I / O) devices 108, memory 116, storage device 114, and network interface 106. Processor 102 can be any suitable processor implemented as a central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), artificial intelligence (AI) accelerator, any other type of processing unit, or combination of different processing units, such as a CPU configured to work in conjunction with a GPU. Typically, processor 102 can be any technically feasible hardware unit capable of processing data and / or executing software applications. Furthermore, in the context of this disclosure, the computing element shown in computing device 100 can correspond to a physical computing system (e.g., a system in a data center) or can be a virtual computing instance executed in a computing cloud.

[0021] I / O device 108 includes devices capable of providing input, such as a keyboard, mouse, touchscreen, microphone, etc., and devices capable of providing output, such as a display device or speaker. Furthermore, I / O device 108 may also include devices capable of receiving input and providing output, such as a touchscreen, Universal Serial Bus (USB) port, etc. I / O device 108 can be configured to receive various types of input from end users of computing device 100 (e.g., designers), and can also provide various types of output to end users of computing device 100, such as displayed digital images, digital video, or text. In some embodiments, one or more I / O devices 108 are configured to couple computing device 100 to network 110.

[0022] Network 110 is any technically feasible communication network that allows data exchange between computing device 100 and external entities or devices (such as web servers or other networked computing devices). For example, network 110 may include a wide area network (WAN), a local area network (LAN), a wireless network (Wi-Fi), and / or the Internet.

[0023] Storage device 114 includes non-volatile memory for storing applications and data, and may include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. Code management engine 122 and graphics design engine 124 may be stored in storage device 114 and loaded into memory 116 during execution.

[0024] Memory 116 includes random access memory (RAM) modules, flash memory cells, or any other type of memory cell or a combination thereof. Processor 102, I / O device interface 104, and network interface 106 are configured to read data from memory 116 and write data to memory 116. Memory 116 contains various software programs executable by processor 102 and application data associated with said software programs, including code management engine 122 or graphics design engine 124.

[0025] Exemplary code-defined visual assets in a design environment

[0026] In graphic design software (e.g., by...) Figure 1 The software implemented by the graphical design engine 124 shown herein can render and design visual assets, such as user interface components in a user interface being designed within the graphical design engine 124. Visual assets used herein can be a set of visual components rendered in a user interface. Visual assets can be defined a priori (e.g., as code templates or pre-designed code-defined visual components within the graphical design engine 124), or they can be designed by the user of the graphical design engine 124 and implemented in code.

[0027] To allow the design of graphical interfaces using visual assets tightly coupled to implementation code, while also ensuring these assets are easily modifiable, the embodiments described herein allow visual assets to be defined, used, and reused as code-defined assets. Typically, a visual asset containing one or more visual components can be defined in an external code asset (e.g., a code file) that can be imported into the design environment. When importing a visual asset into the design environment, it can typically be imported into a layer on the canvas within that environment. Layers on the canvas in the design environment allow for spatial organization of various visual assets within the canvas and can isolate visual assets from other visual assets and from external code assets that derive them. Within a layer, various attributes, actions, triggers, etc., can be modified, allowing customization of the appearance and / or functionality of visual asset instances in the design without modifying the underlying external code asset from which the visual asset is generated. Furthermore, visual asset instances and the code defining such visual assets can be exported to external code assets for reuse in other designs. By doing so, the embodiments described herein achieve reusability of code associated with visual assets and enable rapid development of functional user interfaces based on code-defined visual assets. Furthermore, since the design of the visual appearance of a visual asset and the implementation of its functionality can be tightly coupled, the embodiments presented herein can accelerate the process of building a working user interface and allow the design of such a user interface to achieve the desired visual appearance, rather than just approximating it.

[0028] Figure 2 A design environment 200 according to some embodiments is illustrated, in which visual components are created and maintained as code-defined assets. The design environment 200 can be... Figure 1 One or both of the code management engine 122 and / or the graphics design engine 124 shown are used for rendering.

[0029] To allow visual assets to be defined as code constructs and to tightly couple the appearance of visual assets in design environment 200 with the underlying implementation code, the embodiments proposed herein allow the creation of code that implements visual assets and the use of these code-defined visual assets as modifiable objects in design environment 200. As shown, design environment 200 includes a layout panel 210 for displaying one or more layers associated with visual components in the design, a canvas 205 on which instances of code-defined visual assets are displayed, a control panel 230, and a code panel 240.

[0030] Visual assets 222, 224, and 226 (and others not included) Figure 2 The assets shown typically represent instances of visual assets with different code definitions. For example, visual assets 222 and 224 show various examples of visual assets used to render non-interactive text in a graphics display. Meanwhile, visual asset 226 shows an example of an interactive visual asset (in this example, a calculator, but it should be recognized that the calculator illustration is merely an example, and any type of interactive visual asset can be considered), where interaction with a button modifies the text displayed in a text field and / or triggers the execution of an operation that modifies the text displayed in a text field (e.g., triggering the execution of a mathematical operation, the result of which is displayed in the text field). Each of visual assets 222, 224, and 226 can be included in the code layer of canvas 205 and listed side-by-side in layout panel 210 to allow independent modification of the properties and / or functionality of the visual assets.

[0031] Typically, external code assets can define visual assets (e.g., Figure 2 The appearance and / or functionality of visual assets (one of 222, 224, 226 shown) are described. The appearance of a visual asset can be configured through, for example, code-defined properties (e.g., size, foreground and background colors, transparency, rotation, border size and style, etc.), which can be modified by the user in the design environment. The functionality of a visual asset can be configured through, for example, triggers, actions, etc. Typically, a trigger can correspond to an activity for which a code-defined visual asset emits an event notification, which can be used by other code-defined visual assets to trigger the execution of other code. Similarly, an action can correspond to an activity on which the execution of code is based (e.g., interaction with a button in the user interface causes code to be executed).

[0032] For example, an external code asset associated with visual asset 222 that implements a static text block may contain code that, when executed, returns instructions for rendering the static text block in a graphical user interface. Since visual asset 222 does not contain interactive features, the code associated with the visual asset shown in code panel 240 may not contain action handlers, triggers, or other code that executes when the user interacts with the visual asset. Visual design properties associated with a visual asset can be modified by changing various design parameters displayed in control panel 230 and / or by directly manipulating the visual asset on canvas 205 (e.g., using in-canvas controls rendered on the visual asset). As shown, parameters of a static text block may include position-related parameters (e.g., position, text orientation, element orientation, fill, etc.), size-related parameters, and so on. In some embodiments, the external code asset associated with visual asset 222 may include prior-defined values ​​for one or more parameters shown in control panel 230. Users can change these parameters in design environment 200 by modifying their values ​​in control panel 230 or by modifying the appearance of visual asset 222 on canvas 205. Similarly, external code assets associated with visual asset 224 may include code that returns instructions for rendering multiple static text blocks in a graphical user interface, each of which may have different attributes.

[0033] Visual asset 226, containing various interactive attributes, can be associated with external code assets that define the appearance and functionality of visual asset 226. The code defining visual asset 226 may include code for rendering visual asset 226 in a graphical user interface, and code for performing various functions based on interactions with visual components within visual asset 226. For example, the code may include code that emits triggers that cause modification of text displayed in a text box or otherwise changes the state of visual asset 226. The code defining visual asset 226 may also include action handling code that causes specified actions to be performed in the graphical user interface. For example, action handling code may be attached to a specific button to define the action to be performed, clear buffers or other memory associated with the calculator, perform a previously specified action, and so on.

[0034] Typically, to create a code layer for instances of visual assets, external code assets can be imported into layers on canvas 205. External code assets can be imported directly from external code assets or from canvas contents containers, where the external code assets are imported into the canvas contents containers for use and modification within canvas 205, also known as code instances. For this purpose, code management engine 122 can maintain a collection of external code assets, from which code layers can be generated. The collection of external code assets can typically include code-defined visual assets designed and implemented in code within the design environment, implemented in code outside the design environment, and so on. When an external code asset associated with a code-defined visual asset is imported into a layer on the canvas in the design environment, code management engine 122 can copy the code into the container associated with the canvas and rewrite the code to conform to various rules specific to the language used to write the code asset. For example, for external code assets written in a statically defined programming language that supports exported code assets, code management engine 122 can rewrite the code to change the exported content and the private content within the exported code to conform to the namespace rules defined by the programming language, etc.

[0035] For example, in a language used to write code assets, a single default export (or public code export pointer) component can be defined within a layer on canvas 205 in design environment 200 (e.g., a named component referenced by any name in the code that imports that component). When the code associated with a code instance includes a default export and that default export component is exported to a layer on canvas 205, that code can be copied from the code instance to the layer on canvas 205. In another example, when the code associated with a visual asset defined by code does not include a default export (public code export pointer) component, static analysis and a code compiler can be used to identify functions associated with the visual asset defined by code to designate them as default export (public code export pointer) components. For example, semantic matching between the name of a code instance (or external code asset) imported into design environment 200 and the name of a visual component being converted from a code instance to a code layer in the canvas can be used to identify functions in that layer that are defined as default export (public code export pointer) components. In yet another example, if the code associated with the code instance contains a default exported (public code export pointer) component that is different from the component being converted from the code instance to the code layer in the canvas, the code management engine 122 can rewrite the code to change the default exported (public code export pointer) component from the component specified in the external code asset to the component being converted to the code layer in the canvas.

[0036] In some embodiments, the source code associated with the visual assets defined in the code instance can be analyzed for errors before converting the code instance into a code layer on the canvas. To do this, the source code can be compiled to determine if errors exist during compilation. Typically, errors may occur when the code does not conform to the syntax rules of the language used to write the visual assets defined in the code instance. If syntax errors exist in the source code associated with the visual assets defined in the code instance, the code management engine 122 can copy the source code to the code layer on the canvas without modifying the source code. It should be noted that source code analysis performed to ensure syntactic correctness may allow modification of source code that is syntactically valid according to the language during the conversion from code instance to code component. However, syntactically valid code may not produce the expected results when executed, and the source code modifications performed by the code management engine 122 when converting the visual assets from code instance to code layer may result in malfunctioning code being imported into the code layer.

[0037] Visual assets 222, 224, and 226 represent instances of code-defined visual assets rendered on layers of canvas 205. Visual assets 222, 224, and 226 can be modified in various ways within design environment 200. Visual assets can be imported as code instances within the design environment or as code layers within the design environment. Modifications to code instances can be global modifications, i.e., modifications to the appearance and / or functionality of all instances of the visual asset in design environment 200. That is, if a code-defined visual asset is imported as a code instance and one or more instances of that code-defined visual asset are generated in a code layer (e.g., as described above, by conversion from a code instance), modifications to external code assets associated with that code-defined visual asset may propagate to instances of the code-defined visual asset in the code layers of canvas 205. Simultaneously, modifications made to a code-defined visual asset in a layer on canvas 205 may be limited to the specific layer in which the modification was performed.

[0038] Modifications to code-defined visual assets can be performed based on changes to parameters in control panel 230, changes to code in code panel 240, and / or through controls associated with the visual asset and displayed on canvas 205. When a code-defined visual asset is selected in canvas 205, control panel 230 can be populated with one or more controls for modifying the visual appearance of the code-defined visual asset. Simultaneously, code panel 240 can be populated with the code defining the code-defined visual asset. When the properties of the code-defined visual asset are modified in control panel 230, or when the code defining the code-defined visual asset is changed in code panel 240, the appearance of the code-defined visual asset can be updated in canvas 205. In some embodiments, when the visual appearance of a code-defined visual asset changes, a previously generated static image (or snapshot) of the code-defined visual asset can be replaced by an updated static image of the code-defined visual asset. In some embodiments, the code associated with the code-defined visual asset can be overlaid on canvas 205 and updated using the code reflecting the updated code-defined visual asset when the visual appearance of the code-defined visual asset changes.

[0039] A code-defined visual asset in a layer on canvas 205 can be converted and exported to an external code asset (code instance). To export a code-defined visual asset from a layer in canvas 205 to an external code asset, code management engine 122 can generate a new code file in a file system (e.g., the file system of the computing device running design environment 200, a virtual file system provided by design environment 200, etc.). The code for that layer in canvas 205 can be copied to the new code file. In some embodiments, converting a code-defined visual asset from a code layer to a code instance can include converting the layer into a code instance node so that the code-defined visual asset can continue to be modified within design environment 200. As described above, instances of code-defined visual assets copied to layers in canvas 205 can be linked to code instance nodes, and changes to the visual asset at a code instance node can propagate to the layer in canvas 205 containing that visual asset instance.

[0040] Typically, layers in canvas 205 and their corresponding code-defined visual assets can be copied within design environment 200. When a layer is copied, the design environment creates a new layer (and its associated code container) and copies the contents of the source layer to that new layer. While copying the contents of the source layer to the new layer, the code associated with the code-defined visual assets in the source layer may be copied to the code container associated with the new layer, and a snapshot associated with the code-defined visual assets in the source layer may be copied to the new layer and displayed in design environment 200. By allowing layers to be copied within design environment 200, design environment 200 enables the replication of code-defined visual assets for iterative changes, branching, and other experiments. Furthermore, since each layer containing a code-defined visual asset is associated with an external code asset, changes to the external code asset may propagate to the code-defined visual assets in every layer of canvas 205 that contains that asset.

[0041] Figure 3 Example 300 of a design environment according to some embodiments is shown, as well as the links between visual components in the design environment and their associated design files.

[0042] like Figure 2 As shown, the design environment may include canvas 310 on which various code-defined visual assets can be imported and edited. This design environment allows these code-defined visual assets to be imported and edited as instances, which are object instances 312, 314, and 316 (and...). Figure 3 Other instances not shown) or layers 322, 324 and 326 (and Figure 3 Other instances not shown in the text are indicated.

[0043] Code instances 312, 314, and 316 are associated with the same external code asset (e.g., code asset 302 named ObjectInstance.tsx in this example). When a new code instance is added to canvas 310 (e.g., by copying one of code instances 312, 314, or 316), the new code instance can also be associated with code asset 302. Edits made to one of code instances 312, 314, or 316 in the design environment can propagate to code asset 302. Because code instances 312, 314, and 316 are references to code asset 302, modifications to code asset 302 are typically reflected on canvas 310 in each code instance associated with code asset 302.

[0044] In some embodiments, one or more code instances 312, 314, and / or 316 can be forked from other code instances imported into canvas 310. For this purpose, a new code asset can be created, for example, as a copy of code asset 302. The new code asset can be attached to or otherwise associated with code instances 312, 314, or 316 forked from the code instance associated with code asset 302. In this way, the code instance associated with code asset 302 can be edited and maintained separately from the code instance associated with the new code asset. Edits to the code instance associated with code asset 302 can propagate to other code instances associated with code asset 302, but not to the code instance associated with the new code asset. Similarly, edits to the code instance associated with the new code asset may not propagate to the code instance associated with code asset 302.

[0045] In contrast, code layers 322, 324, and 326 (and) Figure 3 Other layers (not shown) allow for the isolation of modifications to the visual assets defined in the code layer from that layer. As shown, each code layer 322, 324, and 326 can be associated with corresponding code assets 304, 306, and 308, respectively. Since each code layer 322, 324, and 326 can be a self-contained space where the design and / or functionality of the visual assets defined in the code layer can be modified, the design on canvas 310 can be iterated using code layers 322, 324, and 326 without modifying the content or underlying code associated with the visual assets contained in code layers 322, 324, and 326. That is, modifications to the visual assets defined in the code layer can propagate to the corresponding code asset 304, but not to the versions of the visual assets defined in code layers 324 and 326. Furthermore, when a new code layer is created, a corresponding code file can also be created to enforce isolation of the code layer and the visual assets it contains from other code layers on canvas 310.

[0046] In some embodiments, each code layer 322, 324, 326 (and Figure 3Other layers (not shown in the text) can be associated with layer-specific modification history and information describing the various objects contained within them. In code layers 322, 324, and 326, various techniques can be used to modify the visual assets defined as part of the code layer. For example, design changes can be made directly to visual assets in the design environment (e.g., by interacting with various design tools that interact with and modify objects loaded in the design environment). Design and / or functional changes can be made to assets in the design environment by modifying the source code associated with these assets (e.g., by modifying the code file associated with the layer where the visual asset or its components are being modified). These modifications can be performed manually (e.g., through a code editor in the design environment) or using various automated code generation techniques that take input prompts describing the functionality of the visual assets defined in the code and generate corresponding code to implement such functionality.

[0047] Figure 4A and Figure 4B This is a message flow diagram illustrating the messages exchanged between a code management engine and a graphic design engine when importing and modifying visual assets defined by code in a design environment, according to some embodiments.

[0048] Message flow diagram 400A illustrates, according to some embodiments, messages exchanged between code management engine 122 and graphical design engine 124 for importing external code assets associated with code-defined visual assets (e.g., code instances) into layers on a canvas in the design environment and modifying the code-defined visual assets in those layers. As shown, to import an external code asset into a layer on the canvas, a code layer import request 402 may be received at code management engine 122. Code layer import request 402 typically identifies the external code asset from which instances of code-defined visual assets are to be created, and the layer on the canvas from which the code-defined visual asset instances are to be inserted.

[0049] In response to code layer import request 402, code management engine 122 rewrites the source code from the external code asset at block 404 to conform to various rules defined for the code in the design environment. For example, as described above, the source code may be rewritten to conform to the limit on the number of named default exported components in code files associated with layers in the design environment, the design environment itself, etc. In some embodiments, the code may be rewritten if the source code from the external code asset compiles successfully. Typically, when the source code from the external code asset fails to compile (indicating that the source code does not conform to the syntax rules defined for the language used to write the external code asset), code management engine 122 may omit the code rewriting at block 404.

[0050] The rewritten code 406 can be provided to the graphics design engine 124 to generate instances of the code-defined visual assets in layers of the canvas rendered by the graphics design engine 124. To do this, at block 408, a code layer is generated, and the rewritten code 406 is copied to that code layer. Typically, as described above, a code layer represents a container that can contain code-defined visual assets and allow for local editing (e.g., in such a way that editing a code-defined visual asset in one layer of the canvas does not result in other instances of that code-defined visual asset in other layers of the canvas).

[0051] Subsequently, modification 410 to the code layer can be received at the graphics design engine 124. As described above, the modification may include changes to the visual properties of the code-defined visual asset (e.g., changing the size of the code-defined visual asset or its contained components, changing the color of the code-defined visual asset or its contained components, adding or removing components from the visual asset, etc.), changes to the underlying code associated with the code-defined visual asset, and so on. Based on the received modification 410, at block 412, the graphics design engine 124 modifies the code layer. As described above, since modification 410 is local to the code layer, this modification may affect instances of the code-defined visual asset in that layer, but may not affect other instances of the code-defined visual asset in the design environment.

[0052] Figure 4B The diagram illustrates messages exchanged between a code management engine 122 and a graphics design engine 124 according to some embodiments for importing external code assets associated with code-defined visual assets (e.g., code instances) into a layer on a canvas in the design environment and modifying the external code assets associated with the code-defined visual assets in that layer.

[0053] As shown in the figure, after generating the code layer at block 408 based on the code 406 rewritten at block 404 in response to code layer import request 402, external code asset modifications 420 can be received. Although Figure 4BThis is illustrated as a modification received at code management engine 122, but it should be recognized that modifications can also be performed by changing a specific global instance of a code-defined visual asset rendered in the design environment executed by graphics design engine 124. As previously described, modifications can include changes to the visual properties of a code-defined visual asset (e.g., changing the size of a code-defined visual asset or components contained therein, changing the color of a code-defined visual asset or components contained therein, adding or removing components from a visual asset, etc.), changes to the underlying code associated with the code-defined visual asset, and so on. Based on the received modification 420, graphics design engine 124 modifies the code layer at block 422 in which the code-defined visual asset instance is imported. Since the modification is performed on the underlying external code asset (or code instance), it can be a global modification that modifies the instance of the code-defined visual asset contained on the canvas rendered by graphics design engine 124.

[0054] Figure 5 This is a flowchart illustrating an example operation 500 of modifying a visual asset defined based on an external code asset in a design environment, according to some embodiments. Operation 500 may be performed by, for example, a process including one or more processors (e.g., Figure 1 The computing system of the processor 102 of the computing device 100 shown is executed.

[0055] As shown in the figure, operation 500 starts from block 510, generating a layer containing visual assets with code definitions on the canvas of the design environment based on external code assets. Typically, a layer in the design environment can act as a container that stores the code defining the visual assets, and also as a container that allows modification of the positioning and layout of the code-defined visual assets relative to other visual assets displayed in the design environment.

[0056] In some embodiments, to generate a layer containing visual assets with code definitions, source code from an external code asset can be copied to the source code associated with the layer in the canvas. The source code associated with the layer in the canvas can be modified based on the consistency between the default exported names in the source code and the names of the visual assets defined in the layer.

[0057] In some embodiments, modifying the source code associated with a layer in the canvas typically involves determining that the source code in an external code asset does not contain a defined default export name. In this case, the source code associated with the layer can be modified such that the function associated with the name of the code-defined visual asset is defined as the default export in the layer of the canvas. Modifying the source code so that the function associated with the name of the code-defined visual asset is defined as the default export in the layer of the canvas can be used to ensure that the graphical interface (e.g., a design environment) is provided with a way to identify the correct graphical asset for rendering.

[0058] In some embodiments, source code in an external code asset can export a single component. In this case, the source code associated with that layer can be modified such that the function associated with that component is defined as the default export in the layer within the canvas.

[0059] In some embodiments, modifying the source code associated with a layer in the canvas may include determining that the source code in an external code asset contains a default export name that is different from the export code associated with that layer. In this case, the source code associated with the layer in the canvas may be modified such that the default export name defined in the modified source code is associated with the exported code, rather than with the default export name in the external code asset.

[0060] In some embodiments, generating a layer containing visual assets with code definitions may include determining whether to modify the source code associated with the visual assets with code definitions. This determination may be based on the assumption that the visual assets with code definitions in the layer were generated using a default export identified in an external source code asset. In this case, the source code in the external source code asset may be copied to the layer on the canvas without modification. In some embodiments, this determination may be based on the assumption that the source code for the visual assets with code definitions conforms to specifications and is syntactically correct. If the code fails to compile or is syntactically incorrect, the source code of the visual assets with code definitions may be copied directly to the layer without modification.

[0061] In some embodiments, generating a layer containing a code-defined visual asset includes inserting a static image of the visual asset into the layer in a canvas. The static image of the visual asset may be an image captured from the design environment when the code-defined visual asset is created. Typically, by rendering the visual asset in the design environment using a static image of the visual asset, the code associated with the visual asset may not need to be executed, thereby reducing the computational overhead involved in rendering the visual asset in the design environment.

[0062] At block 520, operation 500 continues to receive input associated with modifications to the visual assets to be applied to the code definition in the layer.

[0063] At block 530, operation 500 continues to modify the code-defined visual assets in the layer based on the received input. In some embodiments, the code-defined visual assets in the layer can be modified without modifying external code assets. Furthermore, in some embodiments, the code-defined visual assets in the layer can be modified without modifying instances of code-defined visual assets in other layers on the canvas in the design environment.

[0064] In some embodiments, received input associated with modifications to visual assets defined in code within a layer includes changes made to one or more properties associated with one or more visual components of the visual assets defined in code within one or more control panels in a design environment. Modifying visual assets defined in code within a layer may include editing the code defining the visual assets in code within the layer based on the received input.

[0065] In some embodiments, the input received in connection with modifications to visual assets defined in code within a layer includes changes to the code associated with the visual assets defined in code.

[0066] At block 540, operation 500 continues to render the modified code-defined visual assets on the canvas.

[0067] In some embodiments, operation 500 further includes detecting changes performed in external code assets generated by its generation layer. Instances of the code-defined visual assets can be modified based on the detected changes. By doing so, global changes to the code-defined visual assets can be performed by propagating modifications to the external code assets to other instances of the code assets, thus eliminating the need for manual repetition of changes for each instance of the code-defined visual assets.

[0068] Figure 6 This is a flowchart illustrating example operation 600 for interacting with code-defined visual assets in a design environment, according to some embodiments. Operation 600 may be performed by, for example, one or more processors (e.g., Figure 1 The computing system of the processor 102 of the computing device 100 shown is executed.

[0069] As shown in the figure, operation 600 begins at block 610, importing code-defined visual assets into the canvas of the design environment for editing. As mentioned earlier, code-defined visual assets can be imported into the canvas as instances or layers. Typically, a layer represents a self-contained space within the canvas where modifications made to the visual assets in that layer are isolated to that layer. Instances represent references to underlying external code assets or code files, so that changes to the underlying external code assets or code files are reflected in every instance on the canvas that references the external code assets or code files.

[0070] At block 620, operation 600 continues to receive requests to modify imported code-defined visual assets. As described above, requests to modify imported code-defined visual assets may include changes to the visual properties of the code-defined visual asset on the canvas and / or in a control panel that specifies the properties of the code-defined visual asset. In some embodiments, requests to modify imported code-defined visual assets may also, or alternatively, include changes to the functionality of the code-defined visual asset on the canvas and / or in a code panel that displays the code associated with the code-defined visual asset for editing.

[0071] At block 630, operation 600 continues to modify the code files associated with the visual assets defined in the code based on the received request.

[0072] At block 640, operation 600 continues to update the canvas based on the method of importing visual assets defined in the code to reflect the modified code file.

[0073] In some embodiments, code-defined visual assets can be imported into a new layer on a canvas within the design environment. In some embodiments, this new layer can be a copy of another layer on the canvas within the design environment. Because layers represent self-contained spaces on the canvas, each layer can be associated with its own code assets that define the code-defined visual assets within that layer.

[0074] In some embodiments, importing code-defined visual assets into a new layer on the canvas may include creating a code file as a file associated with the new layer. The code associated with the code-defined visual assets can then be added to the created code file.

[0075] In some embodiments, when a visual asset defined in code is modified in a new layer on the canvas in the design environment, the modification can be isolated from the code files associated with other layers on the canvas in the design environment. The modification can also be isolated from external code files associated with imported visual assets defined in code.

[0076] In some embodiments, code-defined visual assets can be imported as instances on a canvas within a design environment. As described above, these instances typically reference (e.g., point to) an external code file associated with the code-defined visual asset. When a request to modify the code-defined visual asset is received (e.g., for an instance rendered on the canvas), the underlying external code file associated with the code-defined visual asset instance can be modified. Therefore, when the canvas is updated to reflect modifications to the code file associated with the code-defined visual asset, each instance of the code-defined visual asset referencing the external code file can be re-rendered on the canvas within the design environment.

[0077] In some embodiments, a request to separate the instance from other instances referencing external code files may be received. In response to this request, a new external code file may be generated from the external code file associated with the visual asset defined by the code, and the instance may be associated with the new external code file. In some embodiments, when a request to modify the visual asset instance defined by the code is received, a new code file may be modified based on the received request, while keeping the code file associated with the visual asset defined by the code unchanged, and the instance may be updated on the canvas to reflect the modified new code file.

[0078] Figure 7 A network computing system for implementing an interactive application platform on a user computing device is illustrated according to some embodiments. For example... Figure 7 The network computing system shown can be implemented using one or more servers that communicate with the user's computing device via one or more networks. Figure 5 The network computer system 750 shown can correspond to, for example... Figure 1 The computing device 100 shown can be used to generate and / or modify visual content based on a generative artificial intelligence model and input prompts that specify visual content design goals.

[0079] In some embodiments, the network computing system 750 performs operations to enable an interactive application platform (“IAP 700”) to be implemented on the user computing device 10. In some embodiments, IAP 700 can be implemented by receiving program resources of IAP 700 through a user-initiated session (e.g., a user visiting a website). A browser component executes program resources to implement IAP 700, whose functions include receiving user input and rendering content based on or in response to user input. As described above, the implementation of IAP 700 enables users to create various types of content, such as interactive graphic designs, artwork, whiteboard content, program code rendering, presentations, and / or text content. Further described, IAP 700 may include logic (“ASL 716”) for implementing one or more application services, each of which is implemented through IAP 700 to provide a corresponding set of functions and user experiences. IAP 700 also implements application services to share certain resources, such as canvases, workspace files, or design element libraries. In addition, IAP 700 supports the use of multiple application services during a given online session and / or for a specific application service.

[0080] According to some embodiments, a user of computing device 10 operates a web-based application 80 to access a website, retrieve and execute program resources on that website to implement IAP 700. The web-based application 80 can execute scripts, code, and / or other logic (“program components”) to implement the functionality of IAP 700. In some embodiments, the web-based application 80 may correspond to a commercially available browser, such as Google Chrome (developed by Google Inc.) or Safari (developed by Apple Inc.). In some embodiments, the process of IAP 700 can be implemented as scripts and / or other embedded code that can be downloaded from a website by the web-based application 80. For example, the web-based application 80 can execute code embedded in a webpage to implement the process of IAP 700. The web-based application 80 can also execute scripts to retrieve other scripts and program resources (e.g., libraries) from a website and / or other local or remote locations. For example, the web-based application 80 can execute JavaScript embedded in HTML resources (e.g., webpages built according to HTML 5.0 or other versions provided by standards published by the W3C or WHATWG consortium). In some embodiments, the rendering engine 720 may utilize graphics processing unit (GPU) acceleration logic, such as acceleration logic provided by a WebGL (Web Graphics Library) program that executes a graphics library shader language (GLSL) program running on the GPU.

[0081] IAP 700 can be implemented as part of a web service, in which a web-based application 80 communicates with one or more remote computers (e.g., servers for the web service) to perform the IAP 700 process. The web-based application 80 retrieves some or all of the program resources from a web site for implementing IAP 700. The web-based application 80 can also access various types of datasets to provide IAP 700. These datasets may correspond to files and design libraries (e.g., pre-designed design elements), which may be stored remotely (e.g., on a server, associated with an account) or locally. In some embodiments, the network computer system 750 provides a shared design library that the user computing device 10 can use with any application service provided through IAP 700. In this way, a user can initiate a session to implement IAP 700 in order to create or edit workspace files rendered on canvas 722 according to one of the multiple collaborative application services of IAP 700.

[0082] In some embodiments, IAP 700 includes a program interface 502, an input interface 718, and a rendering engine 720. Program interface 720 may include one or more processes that execute to access and retrieve program resources from local and / or remote sources. In one implementation, program interface 502 may generate, for example, canvas 722 using program resources associated with a web-based application 80 (e.g., an HTML5.0 canvas). As an additional or variant, program interface 502 may trigger or otherwise cause canvas 722 to be generated using program resources and datasets (e.g., canvas parameters) retrieved from local (e.g., storage) or remote sources (e.g., web services).

[0083] The program interface 502 can also retrieve program resources containing an application framework used with the canvas 722. This application framework may contain datasets that define or configure, for example, a set of interactive graphics tools integrated with the canvas 722, and constitute an input interface 718 to enable users to provide input to generate or update content rendered on the canvas 722.

[0084] According to some embodiments, input interface 718 can be implemented as a functional layer integrated with canvas 722 for detecting and interpreting user input. For example, input interface 718 can handle user interaction with input mechanisms of the user's computing device (e.g., a pointer device, a keyboard) to detect, for example, cursor positioning / movement relative to canvas 722, hover input (e.g., pre-selection input), selection input (e.g., a click or double-click), shortcut keys (e.g., keyboard input), and other inputs. When processing user interaction with a pointer device, input interface 718 can use a reference to canvas 722 to identify the position of the user's cursor on the screen as the user moves or otherwise interacts with the pointer device. Furthermore, input interface 718 can interpret user input actions based on the location of detected input (e.g., the location of the input indicating a selection tool, a rendered object on the canvas, or an area of ​​the canvas), the frequency of input detected within a given time period (e.g., a double-click), and / or the start and end positions of an input or a series of inputs (e.g., the start and end positions of a click and a drag), as well as various other input types that the user may specify through one or more input devices (e.g., a right-click, a screen click, etc.). In some embodiments, the input interface 718 may interpret, for example, a series of inputs as design tool selections (e.g., shape selection based on input location) and inputs for defining attributes (e.g., size) of the selected shape. In some embodiments, the input interface 718 may interpret continuous inputs (corresponding to continuous movement of the user's pointer device) as the canvas location where the selection tool (e.g., also including shapes) and the output of the selected tool will appear.

[0085] In some embodiments, IAP 700 includes application service logic 716 to support the use of multiple application services during a given user session, where each application service provides specific functionality and / or user experience to the user. As described in some embodiments, each application service is implemented by IAP 700, utilizing the corresponding application service logic 716 to configure the interface interface 718, rendering engine 720, and / or other components of IAP 700 to provide the functionality and user experience of the corresponding application service. In this way, IAP 700 allows a user to operate multiple application services during an online session. Furthermore, different application services can share resources, including programming resources of IAP 700, such as canvas 722. In this way, each application service can contribute content to canvas 722 and / or utilize the functionality and content provided by canvas 722 during a given session. Additionally, application services can be implemented as alternative modalities of IAP 700 so that users can switch between modes, where each mode provides specific functionality and user experience. In some embodiments, each application service can use a common workspace file associated with the user. By default, computing devices that open workspace files can use the default application service to access and / or update the workspace file. Users can also switch the IAP 700's operating mode to use different application services to access, use, and / or update the workspace file.

[0086] The network computing system 750 may include a site manager 758 for managing a website that provides a set of web resources 755 (e.g., web pages) for use by a web-based application 80 of the user computing device 10. Web resources 755 may include instructions, such as scripts or other logic (“ICAP instructions 757”), which can be executed by a browser or web component of the user computing device. Web resources 755 may also include: (i) resources shared between application services, combined with the user computing device's use of any application service, provided to the user computing device; and (ii) application-specific resources that execute on the user computing device for a specific available application service among the available application services. Web resources 755 may also include a design element library, which is partially or wholly shared between application services. This design element library can be combined with the user's use of any application service, allowing the user to select predetermined design elements for use on a canvas 722.

[0087] In some variations, once computing device 10 accesses and downloads web resource 755, web-based application 80 executes IAP instruction 757 to perform the functions described above. For example, IAP instruction 757 may be executed by web-based application 80 to launch program interface 502 on user computing device 10. Launch of program interface 502 may occur simultaneously with, for example, establishing a web socket connection between program interface 502 and service component 760 of network computing system 750.

[0088] In some embodiments, Web resource 755 contains logic that a web-based application 80 can execute to initiate one or more processes of program interface 502, thereby enabling IAP 700 to retrieve additional program resources and datasets for implementing the functionality described in the examples. For example, Web resource 755 may embed logic, including GPU-accelerated logic (e.g., JavaScript code), into an HTML page for download to a user's computing device. Program interface 502 can be triggered to retrieve additional program resources and datasets from, for example, web service 752 and / or local resources of computing device 10, thereby implementing each of the multiple application services of IAP 700. For example, certain components of IAP 700 may be implemented via web pages that can be downloaded to computing device 10 after authentication is performed, and / or after the user performs other actions (e.g., downloading one or more pages of a workspace associated with an account identifier). Therefore, in the example described, the network computing system 750 can transmit the IAP instruction 757 to the computing device 10 via a combination of network communication methods (including downloading activities via the web-based application 80), wherein the IAP instruction 757 is received and executed by the web-based application 80.

[0089] Computing device 10 can use web-based application 80 to access the website of network service 752 to download web pages or web resources. After accessing the website, web-based application 80 can automatically (e.g., via saved credentials) or manually input an account identifier to service component 760. In some embodiments, web-based application 80 may also transmit one or more additional identifiers associated with the user identifier.

[0090] Furthermore, in some embodiments, service component 760 may retrieve profile information 509 from user profile repository 766 using a user identifier or an account identifier of the user identifier. As a supplement or variation, the user's profile information 509 may be determined and stored locally on the user's computing device 10.

[0091] Service component 760 can also retrieve files (“Active Workspace File 763”) linked to an active workspace of a user account or identifier from file repository 765. Profile repository 766 can also identify workspaces associated with an account and / or user, and file repository 765 can store datasets that constitute that workspace. The datasets stored in file repository 765 may include, for example, pages of the workspace and one or more data structure representations 761 of the edited design that can be rendered from the corresponding active workspace file.

[0092] As a supplement or variation, each file may be associated with metadata that identifies the application service used to create the specific file. In some embodiments, the metadata identifier is used to view, use, or otherwise update the default application service of that application service.

[0093] Furthermore, in some embodiments, service component 760 provides web-based application 80 with a representation 759 of a workspace associated with a user, wherein the representation, for example, identifies a single file associated with a user and / or user account. Workspace representation 759 may also identify a set of files, wherein each file contains one or more pages, and each page contains objects as part of a design interface.

[0094] On user device 10, a user can view the workspace representation through web-based application 80 and may choose to open files in the workspace through web-based application 80. In some embodiments, when a user selects to open one of the active workspace files 763, web-based application 80 launches canvas 722. For example, IAP 50 may launch an HTML 5.0 canvas as a component of web-based application 80, and rendering engine 120 may access one or more data structure representations 711 of the content rendered on canvas 722.

[0095] IAP 700 utilizes application service logic 716 to implement multiple operating modes, each corresponding to an application service. As described above, the application service logic 716 associated with each service application can contain instructions and data for configuring components of IAP 700 to include the functionality and characteristics of the corresponding application service. Therefore, the application service logic 716 can, for example, configure the application framework and / or input interface 718 to differ in form, function, and / or configuration in alternative modes of IAP 700. Furthermore, the types of actions and interactions that the user can perform can vary depending on the operating mode. Additionally, different operating modes can include different input or user interface functions for the user to select and use on the canvas 722. For example, when IAP 700 runs in whiteboard service application mode, program interface 502 can provide input functionality, allowing the user to select design elements in the form of "sticky notes," whereas in the alternative mode of the interactive graphic design service application, the "sticky note function" is unavailable. However, in the alternative mode, the user can select any of a number of possible shapes or pre-designed objects, into which the user can write text messages to display on the canvas 722.

[0096] Furthermore, the application service logic 716 can configure the operation of the rendering engine 720, allowing the functionality and behavior of the rendering engine 720 to differ between different application services. In this way, the rendering engine 720 can be used to provide alternative behaviors for different operating modes and to be consistent with the specific service application in an active state. For example, the configuration of the rendering engine 720 can affect the appearance of the canvas 722, the appearance of the content elements rendered on the canvas 722 (e.g., visual attributes), the behavior or representation of user interactions (e.g., whether the user cursor or pointing device is represented on the canvas 722), the type or specific content rendered, the physics engine used by the rendering engine to represent dynamic events (e.g., moving objects), what user operations can be performed (e.g., whether the size of the selected object can be adjusted), and so on.

[0097] Furthermore, each application service can utilize a shared library of content elements (e.g., graphic design elements), as well as core functionality that allows sharing and updating design elements between different application services available on the platform. Additionally, a workspace file created and edited using one application service can be used with another. Moreover, the transition between application services can be seamless—for example, a user computing device 10 can open a workspace file using a first application service (e.g., an interactive graphic design application service for UIX design) and then seamlessly switch to using a second application service (e.g., a whiteboard application service) to work on the same file without closing the workspace file. In some embodiments, each application service allows the user to update the workspace file even when it is being used by other computing devices (e.g., in a collaborative environment). In some embodiments, the user can switch modes on the IAP 700 to switch the application service in use, each application service using the same workspace file.

[0098] Example Terms

[0099] Various aspects of this disclosure are described in the following numbered clauses.

[0100] 1. In some embodiments, a processor-implemented method includes: generating a layer containing visual assets with code definitions based on external code assets on a canvas in a design environment; receiving input associated with modifications to the visual assets with code definitions to be applied to the layer; modifying the visual assets with code definitions in the layer based on the received input; and rendering the modified visual assets with code definitions on the canvas.

[0101] 2. The method of Clause 1, wherein the visual asset defined in the code in the layer is modified without modifying the external code asset.

[0102] 3. The method of any of Clauses 1 or 2, wherein the code-defined visual asset in the layer is modified without modifying instances of the code-defined visual asset in other layers on the canvas in the design environment.

[0103] 4. The method of any one of Clauses 1 to 3, wherein generating the layer containing the visual asset of the code definition comprises: copying source code from the external code asset to the source code associated with the layer in the canvas; and modifying the source code associated with the layer in the canvas based on the consistency between the default exported name in the source code and the name of the visual asset of the code definition in the layer.

[0104] 5. The method as described in Clause 4, wherein modifying the source code associated with the layer in the canvas comprises: determining that the source code in the external code asset does not include a defined default export name; and modifying the source code associated with the layer in the canvas such that a function associated with the name of the visual asset defined by the code is defined as a default export in the layer in the canvas.

[0105] 6. The method of any one of Clauses 4 or 5, wherein modifying the source code associated with the layer in the canvas comprises: determining that the source code in the external code asset includes a default export name different from the export code associated with the layer; and modifying the source code associated with the layer in the canvas such that a default export name defined in the modified source code is associated with the export code, rather than with the default export name in the external code asset.

[0106] 7. The method of any one of Clauses 1 to 6, wherein generating the layer comprising the visual asset defined by the code comprises: inserting a static image of the visual asset into the layer in the canvas.

[0107] 8. The method as described in any one of Clauses 1 to 7 further includes: exporting the visual asset of the modified code definition to a new external code asset.

[0108] 9. The method of any one of Clauses 1 to 8, wherein: the received input associated with a modification to the visual asset of the code definition to be applied to the layer includes changes made to one or more attributes associated with one or more visual components of the visual asset of the code definition in one or more control panels in the design environment; and modifying the visual asset of the code definition in the layer includes editing the code of the visual asset that defines the code definition in the layer based on the received input.

[0109] 10. The method of any one of clauses 1 to 9, wherein the received input associated with the modification of the visual asset of the code definition to be applied in the layer includes changes to the code associated with the visual asset of the code definition.

[0110] 11. The method of any one of clauses 1 to 10, further comprising: detecting changes performed in the external code asset from which the layer is generated; and modifying the visual asset of the code definition on the canvas based on the detected changes.

[0111] 12. In some embodiments, a processor-implemented method includes: importing a code-defined visual asset into a canvas in a design environment for editing; receiving a request to modify the imported code-defined visual asset; modifying a code file associated with the code-defined visual asset based on the received request; and updating the canvas to reflect the modified code file based on the manner in which the code-defined visual asset was imported.

[0112] 13. The method as described in Clause 12, wherein importing the code-defined visual asset into the canvas comprises: importing the code-defined visual asset into a new layer on the canvas in the design environment.

[0113] 14. The method as described in Clause 13, wherein the new layer on the canvas in the design environment comprises a copy of another layer on the canvas in the design environment.

[0114] 15. The method of any one of Clauses 12 or 13, wherein importing the code-defined visual asset into a new layer on the canvas comprises: creating the code file as a file associated with the new layer; and adding the code associated with the code-defined visual asset to the created code file.

[0115] 16. The method of any one of Clauses 12 to 15, wherein modifications to the code file are isolated from code files associated with other layers on the canvas in the design environment.

[0116] 17. The method of any one of Clauses 12 to 16, wherein modifications to the code file are isolated from external code files associated with the visual assets of the imported code definition.

[0117] 18. The method of any one of clauses 12 to 17, wherein importing the code-defined visual asset into the canvas comprises: importing the code-defined visual asset as an instance onto the canvas in the design environment, the instance referencing an external code file associated with the code-defined visual asset.

[0118] 19. The method as described in Clause 18, wherein modifying the code file associated with the visual asset defined by the code includes modifying the external code file.

[0119] 20. The method of any of Clauses 18 or 19, wherein updating the canvas to reflect modifications to the code file associated with the visual asset defined by the code comprises: re-rendering each instance of the visual asset defined by the code that references the external code file on the canvas in the design environment.

[0120] 21. The method of any one of Clauses 18 to 20 further includes: receiving a request to separate the instance from other instances that reference the external code file; and in response to the request: generating a new external code file based on the external code file associated with the visual asset of the code definition, and associating the instance with the new external code file.

[0121] 22. The method of claim 21, further comprising: receiving a request to modify an instance of the visual asset of the code definition; modifying the new code file based on the received request, while keeping the code file associated with the visual asset of the code definition unmodified; and updating the instance on the canvas to reflect the modified new code file.

[0122] 23. A processing system comprising: at least one memory having executable instructions thereon; and one or more processors configured to execute the executable instructions to cause the processing system to perform the method of any one of clauses 1 to 22.

[0123] 24. A processing system comprising: means for performing the method of any one of clauses 1 to 22.

[0124] 25. A non-transitory computer-readable medium having stored executable instructions thereon, which, when processed by one or more processors, cause the one or more processors to perform the method of any one of clauses 1 to 22.

[0125] Any and all combinations of any claim element set forth in any claim and / or any element described in this application, in whatever manner, fall within the intended scope of this invention and its protection.

[0126] The descriptions of the various embodiments are for illustrative purposes only and are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0127] Various aspects of this embodiment can be embodied as a system, method, or computer program product. Therefore, various aspects of this disclosure can take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, collectively referred to herein as a “module,” “system,” or “computer.” Furthermore, any hardware and / or software technology, process, function, component, engine, module, or system described in this disclosure can be implemented as a circuit or a set of circuits. Additionally, various aspects of this disclosure can take the form of a computer program product contained in one or more computer-readable media that contain computer-readable program code.

[0128] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. More specific examples (not an exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0129] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to generate a machine. When executed by a processor of a computer or other programmable data processing apparatus, these instructions are capable of implementing the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such processors can be (but are not limited to) general-purpose processors, special-purpose processors, dedicated processors, or field-programmable gate arrays.

[0130] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, code segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions shown in the blocks may not be in the order shown in the figures. For example, two consecutively displayed blocks may actually execute substantially simultaneously, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the blocks and / or flowcharts, and combinations of blocks in the blocks and / or flowcharts, may be implemented by a dedicated hardware system, or a combination of dedicated hardware and computer instructions, that performs the specified function or action.

[0131] While the foregoing describes embodiments of this disclosure, other and further embodiments of this disclosure may be devised without departing from its basic scope, the scope of which is defined by the appended claims.

Claims

1. A processor-implemented method, comprising: On the canvas of the design environment, generate layers of visual assets containing code definitions based on external code assets; Receive input associated with modifications to the visual assets to be applied to the code definition in the layer; Modify the visual assets defined by the code in the layer based on the received input; as well as Render the modified code-defined visual assets on the canvas.

2. The method according to claim 1, wherein, Modify the visual assets defined by the code in the layer without modifying the external code assets.

3. The method according to claim 1, wherein, Modify the code-defined visual assets in the layer without modifying instances of the code-defined visual assets in other layers of the canvas in the design environment.

4. The method according to claim 1, wherein, Generating the layer containing the visual assets defined in the code includes: Copy the source code from the external code asset to the source code associated with the layer in the canvas; and Modify the source code associated with the layer in the canvas based on the consistency between the default exported name in the source code and the visual asset name defined in the code of the layer.

5. The method according to claim 4, wherein, Modifying the source code associated with the layer in the canvas includes: It is determined that the source code in the external code asset does not include a defined default export name; and Modify the source code associated with the layer in the canvas so that the function associated with the name of the visual asset defined by the code is defined as the default export in the layer in the canvas.

6. The method according to claim 4, wherein, Modifying the source code associated with the layer in the canvas includes: The source code in the external code asset is determined to include a default export name that is different from the exported code associated with the layer; and Modify the source code associated with the layer in the canvas so that the default export name defined in the modified source code is associated with the export code, rather than with the default export name in the external code asset.

7. The method according to claim 1, wherein, Generating the layer that includes the visual assets defined by the code includes inserting a static image of the visual assets into the layer in the canvas.

8. The method according to claim 1, further comprising: Export the modified visual assets defined in the code to a new external code asset.

9. The method according to claim 1, wherein: The input received in connection with modifications to the visual assets defined by the code to be applied to the layer includes changes made to one or more attributes associated with one or more visual components of the visual assets defined by the code in one or more control panels in the design environment; as well as Modifying the visual assets defined by the code in the layer includes editing the code that defines the visual assets in the layer based on the received input.

10. The method according to claim 1, wherein, The input received in connection with modifications to the visual assets defined in the code to be applied to the layer includes changes to the code associated with the visual assets defined in the code.

11. The method according to claim 1, further comprising: Detect changes performed in the external code assets from which the layer was generated; as well as Modify the visual assets defined by the code on the canvas based on the detected changes.

12. A processing system, comprising: At least one memory on which executable instructions are stored; as well as One or more processors are configured to execute the executable instructions to enable the processing system to: On the canvas of the design environment, generate layers of visual assets containing code definitions based on external code assets; Receive input associated with modifications to the visual assets to be applied to the code definition in the layer; Modify the visual assets defined by the code in the layer based on the received input; as well as Render the modified code-defined visual assets on the canvas.

13. The processing system according to claim 12, wherein, Modify the code-defined visual asset in the layer without modifying the external code asset or instances of the code-defined visual asset in other layers of the canvas in the design environment.

14. The processing system according to claim 12, wherein, In order to generate the layer containing the visual assets defined by the code, the one or more processors are configured to make the processing system: Copy the source code from the external code asset to the source code associated with the layer in the canvas; as well as Modify the source code associated with the layer in the canvas based on the consistency between the default exported name in the source code and the visual asset name defined in the code of the layer.

15. The processing system according to claim 14, wherein, In order to modify the source code associated with the layer in the canvas, the one or more processors are configured to make the processing system: It is determined that the source code in the external code asset does not include the defined default export name; as well as Modify the source code associated with the layer in the canvas so that the function associated with the name of the visual asset defined by the code is defined as the default export in the layer in the canvas.

16. The processing system according to claim 14, wherein, In order to modify the source code associated with the layer in the canvas, the one or more processors are configured to make the processing system: The source code in the external code asset is determined to include a default export name that is different from the export code associated with the layer; as well as Modify the source code associated with the layer in the canvas so that the default export name defined in the modified source code is associated with the export code, rather than with the default export name in the external code asset.

17. The processing system according to claim 12, wherein, In order to generate the layer containing the visual assets defined by the code, the one or more processors are configured to cause the processing system to insert static images of the visual assets into the layer of the canvas.

18. The processing system according to claim 12, wherein, The one or more processors are also configured to enable the processing system to export the visual assets of the modified code definition to new external code assets.

19. The processing system according to claim 12, wherein: The input received in connection with modifications to the visual assets defined by the code to be applied to the layer includes changes made to one or more attributes associated with one or more visual components of the visual assets defined by the code in one or more control panels in the design environment; as well as Modifying the visual assets defined by the code in the layer includes editing the code that defines the visual assets in the layer based on the received input.

20. The processing system according to claim 12, wherein, The one or more processors are further configured to enable the processing system to: Detect changes performed in the external code assets from which the layer was generated; as well as Modify the visual assets defined by the code on the canvas based on the detected changes.

Images