Code-defined visual content in computing systems

Abstract

This provides technology for generating visual content in a design environment. [Solution] In embodiments of the present disclosure, an exemplary method includes the step of generating a layer containing a code-defined visual asset on a canvas 722 in a design environment 200 based on an external code asset. Then, upon receiving input associated with changes to be applied to the code-defined visual asset in the layer, the code-defined visual asset in the layer is modified based on the received input, and the modified code-defined visual asset is rendered on the canvas 722.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to graphic design tools, and more particularly to techniques for generating and modifying visual content in graphic design tools.

Background Art

[0002] Generally, a graphic design tool is a tool for creating visual assets within a design environment. These visual assets can be defined, for example, as vector assets defined based on mathematical relationships between different components, or as raster assets defined based on the absolute positions of pixels. In some cases, a visual asset includes one or more containers defined within a vector coordinate space, and a raster asset (e.g., an image) can be inserted into this container. Also, any number of visual components can be included in a visual asset, and the visual asset can be defined based on spatial relationships such as the relative positional relationship between the visual components contained therein.

[0003] In many workflows, the process of designing a visual asset is separated from the process of generating code used to implement the visual asset in an executable environment outside the design environment. Since the two are thus separated, the design of the visual asset may be performed in the design environment, and the coding may be performed in the coding environment. However, since the functions supported in the design environment and the coding environment do not always match, the design created in the design environment may not be reproduced on the coding environment side. That is, the design of the visual asset created in the design environment only functions as a guide when creating the visual asset in the coding environment, and the appearance of the visual asset created in the coding environment may not match the appearance of the visual asset in the design environment.

[0004] Furthermore, even if the process of designing a visual asset and the process of generating the code used to implement that visual asset are separate, the code generated to implement the visual asset in the executable environment may be closely tied to the appearance of that visual asset. Because the appearance and the implementation code are thus tied, when implementing multiple instances of a visual asset, there was a possibility of code duplication within the codebase implementing that visual asset. Consequently, when making changes to each instance of the visual asset, it may have been necessary to repeatedly modify the same code. [Overview of the project] [Problems that the invention aims to solve]

[0005] As explained above, there is a need for techniques to implement visual content more effectively with executable code. [Means for solving the problem]

[0006] One embodiment of the present disclosure provides a technique for generating visual content in a design environment. An exemplary method includes the step of generating a layer containing a code-defined visual asset on a canvas in the design environment based on an external code asset. Then, upon receiving input associated with changes to be applied to the code-defined visual asset in the layer, the code-defined visual asset in the layer is modified based on the received input, and the modified code-defined visual asset is rendered on the canvas.

[0007] One embodiment of the present disclosure provides a technique for making changes to a code-defined visual asset in a design environment. An exemplary method includes the step of importing a code-defined visual asset into a canvas in the design environment for editing. When a change request is received for the imported code-defined visual asset, the code file associated with the code-defined visual asset is modified based on the received request, and the canvas is updated to reflect the modified code file based on how the code-defined visual asset was imported into the canvas.

[0008] One of the technical advantages of the technology disclosed herein over the prior art is that the technology disclosed herein allows visual assets to be defined in code and modified visually or by code in a design environment. Visual assets are defined as code within an external code asset (e.g., a code file) and can be imported as an instance of the visual asset into a layer on a canvas in the design environment. By importing an external code asset into a layer on a canvas to create an instance of the visual asset, this instance of the visual asset can inherit predefined properties for that visual asset. Furthermore, when a change is made to an instance of the visual asset on a layer on a canvas, the code associated with this instance of the visual asset can also be changed, making it possible to locally modify only the instance of the visual asset without changing the external code asset itself. Moreover, according to the embodiments presented herein, the design and coding of visual assets can be more closely linked, making it possible to create visual assets with code that directly reproduces the design of the visual asset, rather than code that approximates the visual design created in the design space. [Brief explanation of the drawing]

[0009] The above is a brief summary of the concept of the present invention. In order to understand the features of each of the various embodiments described above in detail, the concept of the present invention will be explained more specifically below, with reference to various embodiments. Some of these embodiments are also illustrated in the accompanying drawings. It should be noted that the accompanying drawings are merely illustrative of representative embodiments of the concept of the present invention and should not be interpreted as limiting the scope of this disclosure in any sense, and that other embodiments with equivalent effects also exist. [Figure 1] A diagram showing a computer system configured to implement one or more embodiments of various embodiments of the present disclosure. [Figure 2] Diagram illustrating a design environment in which visual components are created and maintained as code-defined assets, according to several embodiments. [Figure 3] Diagram illustrating a design environment and the link structure between visual components within the design environment and the design files associated with these visual components, according to several embodiments. [Figure 4A] A message flow diagram illustrating messages exchanged between a code management engine and a graphic design engine when importing and modifying code-defined visual assets in a design environment, according to several embodiments. [Figure 4B] A message flow diagram illustrating messages exchanged between a code management engine and a graphic design engine when importing and modifying code-defined visual assets in a design environment, according to several embodiments. [Figure 5] Flowchart illustrating the steps of a method for modifying code-defined visual assets in a design environment based on external code assets, according to several embodiments. [Figure 6] Flowchart illustrating steps for interacting with code-defined visual assets within a design environment, according to several embodiments. [Figure 7]Diagram showing a network computing system for implementing an interactive graphics application platform according to several embodiments. [Modes for carrying out the invention]

[0010] The following description includes many specific details to provide a deeper understanding of the various embodiments. However, it will be apparent to those skilled in the art that the concept of the present invention can be implemented even if one or more of these specific details are omitted.

[0011] Figure 1 shows a computing device 100 configured to implement one or more embodiments of various embodiments of the present invention. In one embodiment, the computing device 100 may be a desktop computer, a notebook computer, a smartphone, a personal digital assistant (PDA), or a tablet computer. Alternatively, the computing device 100 may be any other type of computing device suitable for implementing one or more embodiments, configured to receive input, process data, and, if necessary, display images. The computing device 100 is configured to run a code management engine 122 and a graphic design engine 124 that reside in memory 116.

[0012] It should be noted that the computing devices described herein are merely illustrative, and other technically feasible configurations are also included within the scope of this disclosure. For example, the functionality of computing device 100 may be implemented by running multiple instances of the code management engine 122 or the graphic design engine 124 on a group of nodes within a distributed computing system and / or a cloud computing system. Alternatively, the code management engine 122 or the graphic design engine 124 can be adapted to a variety of use cases and applications by running them on various hardware configurations, devices, or environments. Furthermore, the code management engine 122 or the graphic design engine 124 may be configured to run on other computing devices and / or groups of other computing devices.

[0013] In one embodiment, the computing device 100 comprises, but is not limited to, an interconnection bus 112 connecting one or more processors 102, an input / output (I / O) device interface 104 coupled to one or more input / output (I / O) devices 108, memory 116, storage device 114, and a network interface 106. The processors 102 can be any type of processing unit, such as a central processing unit (CPU), an image processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), an artificial intelligence (AI) accelerator, or any suitable processor implemented as a combination of different types of processing units (for example, a configuration in which a CPU and a GPU work together). Generally, the processor 102 can be any technically feasible hardware unit capable of processing data and / or executing software applications. Furthermore, in the context of this disclosure, each element illustrated as a computing element within the computing device 100 may correspond to a physical computing system (for example, a system in a data center) or a virtual computing instance running in a computing cloud.

[0014] The I / O devices 108 include devices that provide input such as keyboards, mice, touch-sensitive screens, and microphones, and devices that provide output such as display devices and speakers. In addition, the I / O devices 108 may also include devices that have both input-receiving and output-providing functions, such as touch panel screens and Universal Serial Bus (USB) ports. The I / O devices 108 can be configured to receive various inputs from the end user of the computing device 100 (e.g., a designer) and to provide various outputs (e.g., displays of digital images, digital video, or text) to the end user of the computing device 100. Furthermore, in some embodiments, one or more I / O devices 108 are configured to connect the computing device 100 to the network 110.

[0015] Network 110 is any technically feasible type of communication network that enables data exchange between computing device 100 and external entities or devices (e.g., network-attached computing devices such as web servers). For example, network 110 may include, in particular, a wide area network (WAN), a local area network (LAN), a wireless (Wi-Fi) network, and / or the internet.

[0016] The storage device 114 includes a non-volatile storage area for applications and data, and can consist of a fixed or removable disk drive, a flash memory device, or a magnetic, optical, or solid-state storage device such as a CD-ROM, DVD-ROM, Blu-ray®, or HD-DVD. The code management engine 122 and the graphic design engine 124 can be stored in the storage device 114 and loaded into memory 116 at runtime.

[0017] Memory 116 includes memory units of various types, such as random access memory (RAM) modules or flash memory units, or combinations thereof. The processor 102, I / O device interface 104, and network interface 106 are configured to read data from and write data to memory 116. Memory 116 contains various software programs executable by the processor 102 (including the code management engine 122 or the graphic design engine 124) and application data associated with these software programs.

[0018] Examples of code-defined visual assets in a design environment Graphic design software, such as that implemented by the graphic design engine 124 shown in Figure 1, can render and design visual assets. Examples of such visual assets include user interface components that constitute the user interface being designed in the graphic design engine 124. In this specification, "visual asset" can refer to a group of visual components rendered in the user interface. Visual assets may be predefined (for example, as code templates in the graphic design engine 124, i.e., pre-designed code-defined visual components), or they may be designed by the user of the graphic design engine 124 and implemented in code.

[0019] In the embodiments described in this specification, a graphical interface can be designed using visual assets closely associated with implementation code, and these visual assets can be easily changed. Therefore, the visual assets are configured to be defined, used, and reused as code-defined assets. Generally, a visual asset composed of one or more visual components can be defined within an external code asset (e.g., a code file) that can be imported into the design environment. Also, when importing a visual asset into the design environment, usually, the visual asset can be imported into a layer on the canvas within the design environment. The layer on the canvas within the design environment enables the spatial configuration of various visual assets within the canvas, and the layer can isolate the visual asset from other visual assets and the external code asset from which the visual asset originated. Within the layer, by changing various properties, actions, triggers, etc., the appearance and functionality of the instance of the visual asset within the design can be customized without modifying the external code asset that is the source of the visual asset. Furthermore, the instance of the visual asset and the code defining such a visual asset can also be exported to an external code asset so that they can be reused in other designs. Thereby, the embodiments described in this specification enhance the reusability of the code associated with the visual asset and enable the rapid development of a functional user interface based on code-defined visual assets. Additionally, according to the embodiments presented in this specification, since the design of the visual appearance of the visual asset and the implementation of the function of the visual asset are closely linked, the process of constructing an actually operating user interface can be accelerated, and such a user interface can be designed not simply to approximate the desired visual appearance but to be the desired visual appearance itself.

[0020] FIG. 2 is a diagram showing a design environment 200 according to some embodiments. In this design environment 200, visual components are created and maintained as code-defined assets. The design environment 200 can be rendered by one or both of the code management engine 122 and the graphic design engine 124 shown in FIG. 1.

[0021] In the embodiments presented in this specification, to define visual assets as code components and realize a configuration that closely associates the appearance of the visual assets in the design environment 200 with the implementation code that defines the visual assets, within the design environment 200, the implementation code of the visual assets is created and these code-defined visual assets can be used as modifiable objects. As shown, the design environment 200 includes a layout panel 210, a canvas 205, a control panel 230, and a code panel 240. The layout panel 210 shows one or more layers associated with visual components in the design. Each instance of the code-defined visual asset is displayed on the canvas 205.

[0022] Visual assets 222, 224, and 226 typically represent instances of different code-defined visual assets (note that other visual assets may exist besides those shown in Figure 2). For example, visual assets 222 and 224 show various examples of visual assets for rendering non-interactive text on a graphical display. Visual asset 226, on the other hand, shows an example of an interactive visual asset (in this example, visual asset 226 is shown as a calculator, but this is just one example, and it should be noted that any type of interactive visual asset is conceivable). In interactive visual assets, interaction with buttons can change the text displayed in a text field or trigger the execution of an operation that changes the text displayed in a text field (for example, interaction with a button can trigger the execution of a mathematical operation, and the result is displayed in the text field). To allow the properties and functions of each visual asset 222, 224, and 226 to be changed independently, these visual assets can be stored in separate code layers within the canvas 205 and listed in the layout panel 210.

[0023] Generally, external code assets can define the appearance and / or functionality of visual assets (for example, any of the visual assets 222, 224, or 226 shown in Figure 2). The appearance of a visual asset can be set, for example, via code-defined properties. Code-defined properties (for example, size, foreground and background colors, transparency, rotation, border thickness and type, etc.) are configured to be modifiable by the user in the design environment. The functionality of a visual asset can be set, for example, via triggers, actions, etc. Generally, a trigger corresponds to an operation that causes a code-defined visual asset to issue an event notification. This event notification triggers other code-defined visual assets to execute other code. On the other hand, an action corresponds to an operation that causes code to be executed based on it (for example, an interaction with a button in the user interface that triggers code execution).

[0024] For example, an external code asset associated with a visual asset 222 that implements a static text block may include code that, at code execution time, returns a command to render the static text block within the graphical user interface. Since the visual asset 222 does not have interactive functionality, the code associated with the visual asset 222 shown in the code panel 240 may be configured not to include code such as action handlers or triggers that are executed by user interaction with the visual asset 222. Visual design properties associated with the visual asset are configured to be modifiable. Visual design properties can be changed by modifying various design parameter values ​​displayed in the control panel 230, or by directly manipulating the visual asset on the canvas 205 (for example, using in-canvas control elements rendered on the visual asset). As shown in the figure, the parameters of a static text block may include position-related parameters (e.g., position, text orientation, element (text block itself) orientation, inner padding, etc.), size-related parameters, etc. In some embodiments, the external code asset associated with the visual asset 222 may include predefined values ​​for one or more parameters, and these parameter values ​​may be configured to be displayed in the control panel 230. These parameter values ​​can be changed by the user of the design environment 200. Parameter values ​​can be changed by directly modifying them on the control panel 230 or by changing the appearance of the visual asset 222 on the canvas 205. Similarly, an external code asset associated with the visual asset 224 may contain code that returns instructions to render multiple static text blocks with different properties within the graphical user interface.

[0025] On the other hand, since Visual Asset 226 is an asset with various interactive functional properties, the external code assets associated with it define both the appearance and functionality of Visual Asset 226. The code that defines Visual Asset 226 can include code for rendering Visual Asset 226 in a graphical user interface and code for executing various functions based on interactions with the visual components contained in Visual Asset 226. For example, the code that defines Visual Asset 226 can include code that issues triggers to change the text displayed in a text box or triggers to change the state of Visual Asset 226. Furthermore, the code that defines Visual Asset 226 can also include action processing code that executes specified actions within the graphical user interface. For example, action processing code such as code that defines the operation to be performed, code that clears memory such as buffers related to the calculator, or code that performs a previously specified operation can be assigned to a particular button.

[0026] Typically, creating a code layer for an instance of a visual asset can be done by importing an external code asset into a layer on the canvas 205. Importing an external code asset can be done directly from the external code asset itself, or from an in-canvas container (also called a code instance) that stores an external code asset already imported, used, or modified within the canvas 205. To achieve this, the code management engine 122 can maintain a collection of external code assets that can be used when generating code layers. This collection of external code assets can typically include code-defined visual assets designed and implemented within the design environment, as well as code-defined visual assets implemented outside the design environment. When an external code asset associated with a code-defined visual asset is imported into a layer on the canvas within the design environment, the code management engine 122 can copy the code to the container associated with the canvas and rewrite the code according to various rules specific to the code asset's writing language. For example, in the case of an external code asset written in a programming language that supports static determination of exported code assets, the code management engine 122 can change which parts of the exported code are exported and which parts are kept private by rewriting the code in accordance with namespace rules and other rules defined by the programming language.

[0027] For example, the code asset description language might allow defining only a single default export (or public code export pointer) component (e.g., a non-anonymous component that can be referenced by any name within the imported code) within a layer on canvas 205 in design environment 200. If the code associated with a code instance includes a default export component, and this default export component is exported to a layer on canvas 205, then that code can be copied from the code instance to the layer on canvas 205. Another example is when the code associated with a code-defined visual asset does not include a default export (public code export pointer) component. In this case, static analysis or a code compiler can be used to identify the function associated with the code-defined visual asset that should be designated as the default export (public code export pointer) component. For example, by performing a semantic match between the name of the code instance imported into design environment 200 (i.e., the external code asset) and the name of the visual component that is converted from the code instance to a code layer in the canvas, the function defined as the default export (public code export pointer) component within the layer can be identified. Another example is when the code associated with a code instance includes a default export (public code export pointer) component, but that component is different from the component that is converted from the code instance to a code layer in the canvas. In this case, the code management engine 122 can rewrite the code to change the default export (public code export pointer) component from the component specified in the external code asset to the component that is converted to a code layer in the canvas.

[0028] In some embodiments, the source code associated with a code-defined visual asset can be analyzed for errors in the code instance before it is converted from the code instance to a code layer on the canvas. To achieve this, the source code can be compiled, and the presence or absence of compile-time errors can be determined. Generally, errors may occur if the code does not conform to the syntax rules defined for the description language of the code-defined visual asset. If a syntax error is found in the source code associated with the code-defined visual asset, the code management engine 122 can copy the source code to the code layer on the canvas without making any changes to it. On the other hand, if the code is found to be syntactically correct through the analysis of the source code, it should be noted that the "correct" source code according to the syntax rules of the description language may be subject to modification during the conversion from a code instance to a code component. However, syntactically "correct" code does not necessarily produce the desired result at runtime. Therefore, changes to the source code made by the code management engine 122 when converting a visual asset from a code instance to a code layer may result in functionally incorrect code being imported into the code layer.

[0029] Visual assets 222, 224, and 226, which represent instances of code-defined visual assets rendered on layers within canvas 205, can be modified in various ways within the design environment 200. Visual assets can be imported into the design environment 200 as code instances or code layers. Changes made to a code instance can be configured to be global changes, i.e., changes that affect the appearance and / or functionality of all instances of that visual asset within the design environment 200. Here, if a code-defined visual asset is imported as a code instance, and further (for example, by conversion from the code instance as described above) one or more instances of that code-defined visual asset are generated within a code layer, then when a change is made to an external code asset associated with the code-defined visual asset, that change can be configured to propagate to each instance of that code-defined visual asset in the code layer within canvas 205. On the other hand, changes made to a code-defined visual asset within a layer on canvas 205 can be configured to be local changes that apply only to the specific layer in which the change was made.

[0030] Changes to code-defined visual assets can be made by modifying parameter values ​​in the control panel 230, modifying code in the code panel 240, or manipulating control elements displayed on the canvas 205 and associated with the visual asset. When a code-defined visual asset is selected in the canvas 205, the control panel 230 displays one or more control elements for changing the visual appearance of the code-defined visual asset. Simultaneously, the code panel 240 can be configured to display the code that defines the code-defined visual asset. When changes are made to the properties of a code-defined visual asset in the control panel 230, or when changes are made to the code that defines the code-defined visual asset in the code panel 240, the appearance of the code-defined visual asset in the canvas 205 can be updated. In some embodiments, when the visual appearance of a code-defined visual asset is changed, previously generated still images (i.e., snapshots) of the code-defined visual asset can be replaced with updated still images of the updated code-defined visual asset. Furthermore, in some embodiments, when the visual appearance of a code-defined visual asset is changed, the code associated with the code-defined visual asset is superimposed on the canvas 205, and this superimposed code is updated with code that reflects the updated code-defined visual asset.

[0031] Furthermore, code-defined visual assets within layers on canvas 205 can be converted and exported to external code assets (code instances). When exporting code-defined visual assets from layers in canvas 205 to external code assets, the code management engine 122 can first generate a new code file in a file system (for example, the file system of the computing device on which the design environment 200 is running, or a virtual file system provided by the design environment 200). Then, the code contained in the layer in canvas 205 can be copied into this new code file. In some embodiments, the step of converting a code-defined visual asset from a code layer to a code instance may include the step of converting the layer to a code instance node. This step allows for further modifications to the code-defined visual asset within the design environment 200. As described above, instances of code-defined visual assets copied to layers in canvas 205 can be linked to code instance nodes. This allows for a configuration where, when a change is made to a visual asset in the code instance node, that change is propagated to the layer in canvas 205 containing each instance of the visual asset.

[0032] Generally, layers within canvas 205 and their corresponding code-defined visual assets can be duplicated within design environment 200. When a layer is duplicated, design environment 200 creates a new layer (and the code container associated with that layer), and copies the contents of the source layer to this new layer. When copying the contents of the source layer to the new layer, the code associated with the code-defined visual assets in the source layer is copied to the code container associated with the new layer, and the snapshots associated with the code-defined visual assets in the source layer are also copied to the new layer and displayed within design environment 200. By making layers duplicable within design environment 200, it becomes possible to duplicate code-defined visual assets within design environment 200 and perform iterative changes and branching. Furthermore, since the code-defined visual assets stored in each layer are associated with an external code asset, when changes are made to the external code asset, those changes can be propagated to the code-defined visual assets associated with that external code asset stored in each layer within canvas 205.

[0033] Figure 3 shows Example 300, which illustrates a design environment and a link structure between visual components within the design environment and the design files associated with these visual components, according to several embodiments.

[0034] As shown in Figure 2, the design environment can include a canvas 310, on which various code-defined visual assets can be imported and edited. In the design environment, code-defined visual assets can be imported and edited as instances. These instances can be represented as object instances 312, 314, 316, or layers 322, 324, 326 (note that object instances and layers other than those shown in Figure 3 may also exist).

[0035] Code instances 312, 314, and 316 are associated with the same external code asset (for example, code asset 302 named ObjectInstance.tsx in this example). When a new code instance is added to canvas 310 (for example, by duplicating one of code instances 312, 314, or 316), this new code instance can also be configured to be associated with code asset 302. When an edit is made to any of code instances 312, 314, or 316 in the design environment, that edit can be propagated to code asset 302. Since code instances 312, 314, and 316 are references to code asset 302, changes made to code asset 302 are usually reflected in each associated code instance in canvas 310.

[0036] In some embodiments, one or more of the code instances 312, 314, and 316 can be forked from other code instances imported into the canvas 310. To achieve this, for example, a new code asset can be created by duplicating code asset 302. This new code asset can then be associated in some way with one of the code instances 312, 314, or 316 forked from the code instance associated with code asset 302. This allows editing and maintenance of the code instance associated with code asset 302, separate from the code instance associated with the new code asset. When editing is made to the code instance associated with code asset 302, the edits can be configured to propagate to other code instances associated with code asset 302, but not to the code instance associated with the new code asset. Similarly, edits made to the code instance associated with the new code asset can be configured not to propagate to the code instance associated with code asset 302.

[0037] On the other hand, code layers 322, 324, and 326 can localize changes to code-defined visual assets within their respective layers (note that there may be code layers other than those shown in Figure 3). As illustrated, each code layer 322, 324, and 326 can be associated with corresponding code assets 304, 306, and 308, respectively. Each code layer 322, 324, and 326 can be configured as a self-contained space where changes can be made to the design and functionality of the stored code-defined visual assets. Therefore, by using code layers 322, 324, and 326, designs on canvas 310 can be iterated without changing the content associated with the visual assets stored therein or the code that defines those visual assets. That is, when a change is made to a code-defined visual asset in a code layer, that change is propagated to the corresponding code asset 304, but not to the code-defined visual assets corresponding to code layers 324 and 326. Furthermore, it can be configured so that when a new code layer is created, a corresponding new code file is created. By creating a new code file in this way, the new code layer and the visual assets stored within it can be reliably separated from other code layers on canvas 310.

[0038] In some embodiments, each code layer 322, 324, and 326 can be associated with a layer-specific change history, as well as with information describing the various objects stored in that layer (note that there may be code layers other than those shown in Figure 3). Within each code layer 322, 324, and 326, modifications can be made to the code-defined visual assets that constitute that code layer using various techniques. For example, design changes can be made directly to the visual assets in the design environment (for example, modifications can be made using various design tools that modify objects through interaction with objects loaded into the design environment). Alternatively, design or functional changes can be made to an asset by modifying the source code associated with that visual asset in the design environment (for example, modifications can be made by modifying the code file associated with the layer that stores the visual asset or component to be changed). These changes can be made manually (for example, using the code editor in the design environment), or they can be made using various automated code generation techniques that take input prompts describing the functionality of the code-defined visual assets and generate corresponding code to implement that functionality.

[0039] Figures 4A and 4B are message flow diagrams showing messages exchanged between the code management engine and the graphic design engine when importing and modifying code-defined visual assets in a design environment, according to several embodiments.

[0040] Message flow diagram 400A shows messages exchanged between the code management engine 122 and the graphic design engine 124 when importing an external code asset associated with a code-defined visual asset (e.g., a code instance) into a layer on a canvas in the design environment, and further modifying the code-defined visual asset stored in that layer, according to several embodiments. As shown, when importing an external code asset into a layer in the canvas, the code management engine 122 may receive a code layer import request 402. Generally, the code layer import request 402 specifies the external code asset from which the instance of the code-defined visual asset is created, and the layer in the canvas into which the instance of the code-defined visual asset is inserted.

[0041] In response to this code layer import request 402, the code management engine 122 rewrites the source code obtained from the specified external code asset in block 404 to conform to various coding rules defined within the design environment. For example, as described above, the source code can be rewritten to conform to limitations on the number of non-anonymous default export components allowed in the code files associated with layers in the design environment, or to other constraints imposed by the design environment itself. In some embodiments, the code can be rewritten only if the source code obtained from the external code asset is successfully compiled. On the other hand, if the compilation of the source code obtained from the external code asset fails (a compilation failure indicates that the source code does not conform to the syntax rules defined for the description language of the external code asset), the code management engine 122 can usually omit the code rewriting process in block 404.

[0042] The rewritten code 406 is provided to the graphic design engine 124, which can then generate instances of the code-defined visual asset within the canvas layers it renders. To achieve this, a code layer is first created in block 408, and the rewritten code 406 is copied to this code layer. Generally, as described above, a code layer functions as a container for storing code-defined visual assets, and local edits can be made to those code-defined visual assets within this container (where "local" editing refers to edits made to a code-defined visual asset in a particular layer on the canvas that do not affect other instances of that code-defined visual asset stored in other layers on the canvas).

[0043] Next, the graphic design engine 124 can receive changes 410 to the code layer. As described above, these changes 410 may include changes to the visual properties of a code-defined visual asset (e.g., resizing the code-defined visual asset or its internal components, changing the color of the code-defined visual asset or its internal components, adding or removing components within the visual asset, etc.) or changes to the underlying code associated with the code-defined visual asset. Then, in block 412, the graphic design engine 124 modifies the code layer based on the received changes 410. As described above, since the changes 410 are applied locally to the code layer, the changes in block 412 can be configured to affect instances of the code-defined visual asset within that layer but not other instances of the code-defined visual asset within the design environment.

[0044] Figure 4B shows messages exchanged between the code management engine 122 and the graphic design engine 124 when, in some embodiments, an external code asset associated with a code-defined visual asset (e.g., a code instance) is imported into a layer on the canvas in the design environment, and when the external code asset associated with the code-defined visual asset stored in that layer is modified.

[0045] As illustrated, after generating a code layer in block 408 using code 406 rewritten in block 404 in response to a code layer import request 402, the system may receive a change 420 to an external code asset. Note that in Figure 4B, this change 420 is shown as being received by the code management engine 122, but it should be noted that the change 420 can also be made by modifying a global instance, which is a special instance of a code-defined visual asset rendered within the design environment run by the graphic design engine 124. As mentioned above, the change 420 may include changes to the visual properties of the code-defined visual asset (e.g., resizing the code-defined visual asset or its internal components, changing the color of the code-defined visual asset or its internal components, adding or deleting components within the visual asset, etc.) or changes to the underlying code associated with the code-defined visual asset. Then, in block 422, the graphic design engine 124 modifies each code layer into which each instance of the code-defined visual asset has been imported, based on the received change 420. Since this change 420 is applied to the underlying external code asset (or code instance), it may be applied as a comprehensive change to each instance of that code-defined visual asset contained within the canvas rendered by the graphic design engine 124.

[0046] Figure 5 is a flowchart illustrating an exemplary process 500 that modifies a code-defined visual asset in a design environment based on an external code asset, according to several embodiments. This process 500 can be performed by a computing system having one or more processors (for example, the processor 102 of the computing device 100 shown in Figure 1).

[0047] As shown in the diagram, process 500 begins with block 510, which generates a layer containing a code-defined visual asset on a canvas within the design environment based on an external code asset. Generally, a layer within the design environment functions as a container for storing the code that defines the code-defined visual asset, and also as a container for changing the relative position and layout of the code-defined visual asset relative to other visual assets displayed within the design environment.

[0048] In some embodiments, when generating a layer containing a code-defined visual asset, the source code of an external code asset can be copied as the source code associated with that layer in the canvas. Furthermore, the source code associated with this layer in the canvas can be modified based on the matching relationship between the default export name in the source code and the name of the code-defined visual asset within that layer.

[0049] In some embodiments, the process of modifying the source code associated with a layer in a canvas typically includes determining whether the default export name is undefined in the source code of an external code asset. If it is determined that the default export name is undefined, the source code associated with the layer can be modified to define the function associated with the name of the code-defined visual asset as the default export for that layer in the canvas. By modifying the source code in this way to define the function associated with the name of the code-defined visual asset as the default export for that layer in the canvas, the correct graphic asset can be identified when a graphical interface, such as a design environment, renders the graphic asset.

[0050] Furthermore, in some embodiments, the source code of an external code asset may be configured to export only a single component. In such cases, the source code associated with the layer can be modified to define the function associated with that component as the default export for that layer in the canvas.

[0051] In some embodiments, the step of modifying the source code associated with a layer in the canvas may include determining whether the source code of an external code asset contains a different default export name than the exported code associated with that layer. If it is determined that it contains a different default export name, the source code associated with the layer in the canvas can be modified so that the default export name defined in the modified source code is associated with the exported code instead of the default export name of the external code asset.

[0052] In some embodiments, the step of generating a layer containing a code-defined visual asset may include a step of determining whether or not to modify the source code associated with the code-defined visual asset. The determination of whether or not to modify the source code can be made based on the result of determining whether or not the code-defined visual asset in the layer was generated using the default export specified in the external code asset. If the code-defined visual asset was generated using the default export, it can be copied directly to the layer on the canvas without making any changes to the source code of the external code asset. In some embodiments, the determination of whether or not to modify the source code can also be made based on the result of compiling the source code of the code-defined visual asset and determining whether or not it is syntactically correct. If the compilation of the source code of the code-defined visual asset fails, or if the source code contains syntactic errors, it can be copied directly to the layer without making any changes to the source code.

[0053] In some embodiments, the step of generating a layer containing a code-defined visual asset may include the step of inserting a still image of the visual asset into the layer on the canvas. The still image of the visual asset may be an image captured from the design environment when the code-defined visual asset was created. Generally, using a still image of the visual asset when rendering it in the design environment eliminates the need to execute the code associated with the visual asset, thereby reducing the computational cost required to render the visual asset in the design environment.

[0054] Next, process 500 proceeds to block 520, which receives input associated with the changes to be applied to the code-defined visual assets within that layer.

[0055] Next, process 500 proceeds to block 530, where it modifies the code-defined visual asset in the layer based on the received input. In some embodiments, the code-defined visual asset in the layer can be modified without modifying the external code asset. Also, in some embodiments, the code-defined visual asset in the layer can be modified without modifying instances of the code-defined visual asset that exist in other layers on the canvas in the design environment.

[0056] In some embodiments, the input received as an input associated with changes applied to a code-defined visual asset within the layer may consist of changes made in one or more control panels within the design environment to one or more properties associated with one or more visual components within the code-defined visual asset. In this case, the step of making changes to the code-defined visual asset within the layer may include editing the code that defines the code-defined visual asset within the layer based on the received input.

[0057] Furthermore, in some embodiments, the input received as an input associated with a change applied to a code-defined visual asset within the layer may consist of changes made to the code associated with the code-defined visual asset.

[0058] Next, process 500 proceeds to block 540, which renders the modified code-defined visual asset onto the canvas.

[0059] In some embodiments, the process 500 further includes a step of detecting changes made to the external code asset from which the layer was generated. Furthermore, the configuration can be such that each instance of the code-defined visual asset is modified based on the detected changes. This allows for comprehensive changes to the code-defined visual asset by propagating changes to the external code asset to other instances of the code asset, eliminating the need to repeatedly and manually perform such changes on each instance of the code-defined visual asset.

[0060] Figure 6 is a flowchart illustrating an exemplary process 600 for interacting with code-defined visual assets in a design environment, according to several embodiments. This process 600 can be executed by a computing system having one or more processors (for example, the processor 102 of the computing device 100 shown in Figure 1).

[0061] As shown in the diagram, process 600 begins with block 610, which imports a code-defined visual asset into the canvas within the design environment for editing. As described above, code-defined visual assets can be imported into the canvas as instances or layers. Generally, a layer functions as a self-contained space within the canvas, and changes made to the visual asset within the layer are localized to that layer. An instance, on the other hand, functions as a reference to an underlying external code asset or code file, and changes made to that underlying external code asset or code file are reflected in each instance on the canvas that references that external code asset or code file.

[0062] Next, process 600 proceeds to block 620, where it receives a request to modify an imported code-defined visual asset. As described above, a request to modify an imported code-defined visual asset may include a request to modify the visual properties of the code-defined visual asset. Such a request to modify visual properties can be made by operations on the canvas or by operations on the control panel that specify the properties of the code-defined visual asset. In addition to or instead of this, in some embodiments, a request to modify an imported code-defined visual asset may include a request to modify the functionality of the code-defined visual asset. Such a request to modify functionality can be made by operations on the canvas or by operations on the code panel for viewing and editing the code associated with the code-defined visual asset.

[0063] Next, process 600 proceeds to block 630, where it modifies the code file associated with the code-defined visual asset based on the received request.

[0064] Furthermore, process 600 proceeds to block 640, which updates the canvas to reflect the modified code file based on how the code-defined visual asset is imported into the canvas.

[0065] In some embodiments, code-defined visual assets can be imported into a new layer on the canvas within the design environment. In some embodiments, this new layer can be a duplicate of another layer on the canvas within the design environment. Since each layer functions as a self-contained space on the canvas, each layer can be associated with its own code asset that defines the code-defined visual assets stored within it.

[0066] In some embodiments, the process of importing a code-defined visual asset into a new layer on the canvas may include the process of creating a code file associated with the new layer. The code associated with the code-defined visual asset can then be added to the created code file.

[0067] In some embodiments, when a code-defined visual asset in a new layer on the canvas within the design environment is modified, the modification can be configured to be decoupled from the code files associated with other layers on the canvas within the design environment. Furthermore, this modification can also be decoupled from the external code files associated with the imported code-defined visual asset.

[0068] In some embodiments, code-defined visual assets can be imported as instances onto a canvas in the design environment. As described above, instances generally function as references (e.g., pointers) to external code files associated with the code-defined visual asset. When a change request is received for a code-defined visual asset (for example, when such a request is made for an instance rendered on the canvas), the underlying external code file associated with that instance of the code-defined visual asset can be modified. Therefore, when the canvas is updated to reflect changes in the code file associated with the code-defined visual asset, each instance of the code-defined visual asset that references that external code file can be re-rendered on the canvas in the design environment.

[0069] In some embodiments, a request can be received to detach the instance from other instances that reference the external code file. In response to this request, a new external code file can be generated based on the external code file associated with the code-defined visual asset, and the instance can be associated with this new external code file. In some embodiments, when a change request is received for the instance of the code-defined visual asset, the system can be configured to modify the new code file based on the received request without making any changes to the code file associated with the code-defined visual asset. The instance on the canvas can then be updated to reflect the modified new code file.

[0070] Figure 7 shows a network computing system for implementing an interactive application platform on a user computing device, according to several embodiments. A network computing system like the one shown in Figure 7 can be implemented using one or more servers that communicate with the user computing device over one or more networks. The network computer system 750 shown in Figure 7 can correspond, for example, to the computing device 100 shown in Figure 1, and can be used to generate and / or modify visual content based on a generating artificial intelligence model and input prompts specifying design policies for the visual content.

[0071] In some embodiments, the network computer system 750 can be configured to implement an interactive application platform ("IAP700") on the user computing device 10 through operations performed by the network computer system 750. In some embodiments, the IAP700 can be implemented when a user initiates a session (for example, when a user accesses a website) and receives program resources for the IAP700. The browser component implements the IAP700 by executing the received program resources and has the function of receiving user input and rendering content based on or in response to said input. As described above, by implementing the IAP700, users can create various types of content such as interactive graphic designs, art, whiteboard content, program code renderings, presentations, and / or text content. Also as described above, the IAP700 may include logic ("application service logic (ASL) 716") for implementing one or more application services. Each application service is executed via the IAP700 and provides the functionality and user experience corresponding to that application service. Furthermore, IAP700 is implemented to share resources such as canvases, workspace files, and design element libraries across multiple application services. In addition, IAP700 allows for the simultaneous use of multiple application services during a given online session or in relation to a specific application service.

[0072] In some embodiments, a user of computing device 10 implements IAP700 by operating a web-based application 80 to access a network site, obtaining program resources from the network site, and executing them. The web-based application 80 can implement the functions of IAP700 by executing logic such as scripts and code ("programmatic components"). In some embodiments, the web-based application 80 can be compatible with commercially available browsers such as GOOGLE CHROME (developed by GOOGLE, INC.) and SAFARI (developed by APPLE, INC.). In some embodiments, each process of IAP700 can be implemented as embedded code, such as scripts, downloaded by the web-based application 80 from the network site. For example, the web-based application 80 can implement each process of IAP700 by executing code embedded in a web page. Furthermore, the web-based application 80 can also obtain other scripts and program resources (such as libraries) from network sites or other local or remote locations by executing scripts. For example, a web-based application 80 can execute JavaScript® embedded in an HTML resource (for example, a web page compliant with an HTML version such as HTML5.0, structured according to a standard specification published by the W3C Consortium or the WHATWG Consortium). In some embodiments, the rendering engine 720 can utilize GPU (graphics processing unit) acceleration logic. This GPU acceleration logic is provided, for example, through a Web GL (Web Graphics Library) program that executes a Graphics Library Shader Language (GLSL) program on the GPU.

[0073] IAP700 can be implemented as part of a network service, and a web-based application 80 can execute each process of IAP700 by communicating with one or more remote computers (e.g., a server for network services). The web-based application 80 obtains some or all of the program resources for implementing IAP700 from the network site. In providing IAP700, the web-based application 80 can also access various types of datasets. These datasets correspond to files and design libraries (e.g., pre-designed design elements), and these datasets may be held remotely (e.g., stored on a server or associated with an account) or locally. Furthermore, in some embodiments, a shared design library is provided by a network computer system 750, and the user computing device 10 can utilize this shared design library along with any application services provided via IAP700. This allows the user to initiate a session implementing IAP700 to create or edit workspace files rendered on the canvas 722 according to one of the IAP700's multiple collaboration application services.

[0074] In some embodiments, the IAP 700 comprises a program interface 502, an input interface 718, and a rendering engine 720. The program interface 72 may include one or more processes performed to access and obtain program resources from local and / or remote sources. In one embodiment, the program interface 502 can generate, for example, a canvas 722 using program resources associated with a web-based application 80 (e.g., an HTML5.0 canvas). In addition to or instead of this, the program interface 502 may generate the canvas 722 by triggering, etc., using program resources and datasets (e.g., canvas parameters) obtained from a local source (e.g., memory) or a remote source (e.g., a network service).

[0075] Furthermore, the program interface 502 can obtain program resources, including an application framework for use with the canvas 722. This application framework may include, for example, a dataset that defines or configures a set of interactive graphics tools integrated into the canvas 722 and constituting the input interface 718. This allows the user to provide input for generating or updating content to be rendered on the canvas 722.

[0076] According to some embodiments, the input interface 718 can be implemented as a functional layer integrated into the canvas 722, responsible for detecting and decoding user input. The input interface 718 can, for example, process interactions between the user and the input mechanisms of the user computing device (e.g., pointer device, keyboard), and can detect inputs such as cursor position / movement on the canvas 722, hover input (e.g., preliminary selection input), selection input (e.g., click or double-click), and shortcuts (e.g., keyboard input). When processing interactions between the user and the pointer device, the input interface 718 can identify the user's cursor position on the screen when the user interacts with the pointer device, such as moving the pointer device, by utilizing a reference to the canvas 722. Furthermore, the input interface 718 can decode the user's input actions. Such decoding can be performed based on the location of the detected input (e.g., whether the input location indicates a tool selection, a selection of an object rendered on the canvas, or a selection of an area within the canvas), the frequency of the detected input over a predetermined period (e.g., double-click), and / or the start and end positions of the input or a series of inputs (e.g., the start and end positions of a click or drag), as well as various other input types that the user can specify via one or more input devices (e.g., right-click, screen tap, etc.). In some embodiments, for example, the input interface 718 may interpret a series of inputs as a selection of a design tool (e.g., a selection of a shape based on the input location) or as an input defining the attributes of the selected shape (e.g., dimensions). Also in some embodiments, the input interface 718 may interpret a series of inputs (inputs corresponding to the continuous movement of the user's pointer device) as an input for selecting a tool (e.g., a shape) and specifying the location on the canvas where the output of the selected tool should be displayed.

[0077] In some embodiments, the IAP700 includes application service logic 716. This application service logic 716 enables the use of multiple application services during a given user session, providing the user with features and user experiences specific to each application service. As described in some embodiments, each application service is implemented on the IAP700 using the corresponding application service logic 716. This constitutes components of the IAP700, such as the interface component 718 and the rendering engine 720, and provides the features and user experience of the corresponding application service. In this way, the IAP700 enables the operation of multiple application services during a user's online session. Furthermore, resources such as program resources of the IAP700, including the canvas 722, can be shared among different application services. This allows each application service to provide content to the canvas 722 and / or utilize features and content provided to the canvas 722 during a given session. Application services can also be implemented as selectable modes of the IAP700. In this case, each mode provides specific features and user experiences, and the user can switch between these modes. In some embodiments, each application service can utilize a common workspace file associated with a user. By default, a computing device that opens this common workspace file can access and update it using the default application service. Alternatively, a user can access, use, and / or update the workspace file using different application services by switching the operating mode of the IAP700.

[0078] The network computing system 750 may include a site manager 758. The site manager 758 manages a website that provides a set of web resources 755 (e.g., web pages) for a web-based application 80 on a user computing device 10. The web resources 755 may include instructions ("IAP instructions 757") such as scripts or other logic that can be executed by the browser or web component of the user computing device. The web resources 755 may also include (i) inter-application service shared resources provided to user computing devices using any of the application services, and (ii) application-specific resources that run on the user computing device as resources dedicated to a particular one of the available application services. Furthermore, the web resources 755 may include a design library of design elements, some or all of which are shared among the application services. This design element library allows the user to select a predetermined design element to use on the canvas 722 when using any of the application services.

[0079] In some variations, when the computing device 10 accesses and downloads the web resource 755, the web-based application 80 executes an IAP instruction 757, thereby realizing the functions described above. For example, the execution of the IAP instruction 757 by the web-based application 80 can launch the program interface 502 on the user computing device 10. Alternatively, the launch of the program interface 502 may occur simultaneously with the establishment of a WebSocket connection between the program interface 502 and the service component 760 of the network computer system 750.

[0080] In some embodiments, the logic contained in the web resource 755 is configured such that when the web-based application 80 executes the logic and initiates one or more processes of the program interface 502, the IAP 700 acquires additional program resources and datasets necessary to implement the various functions exemplified. For example, the web resource 755 may be configured to embed logic (e.g., "JAVASCRIPT" code) including GPU acceleration logic within an HTML page downloaded by the user's computing device. Furthermore, when implementing any of the multiple application services of the IAP 700, each application service may be configured to trigger the program interface 502 to acquire additional program resources and datasets (e.g., from the network service 752 and / or local resources of the computing device 10). For example, some components of the IAP 700 may be implemented via a web page downloadable to the computing device 10 after authentication is complete, or in addition to or instead of this, after the user performs an additional action (e.g., downloading one or more pages of a workspace associated with an account identifier). Therefore, in the example described above, the network computing system 750 can be configured to transmit an IAP instruction 757 to the computing device 10 by combining multiple network communications, including a download process by the web-based application 80, and the web-based application 80 can receive and execute this IAP instruction 757.

[0081] The computing device 10 can access the website of the network service 752 using a web-based application 80 and download web pages or web resources. When the web-based application 80 accesses the website of the network service 752, it can transmit an account identifier to the service component 760 automatically (for example, using stored authentication information) or by manual input. In some embodiments, the web-based application 80 can also transmit one or more additional identifiers correlated with the user identifier.

[0082] Furthermore, in some embodiments, the service component 760 can retrieve profile information 509 from the user profile store 766 using the user identifier or its account identifier. Alternatively, the user's profile information 509 can be identified and stored locally on the user's computing device 10.

[0083] Furthermore, the service component 760 can retrieve files of the active workspace associated with a user account or identifier ("active workspace files" 763) from the file store 765. At this time, the workspace associated with the account and / or user can be identified by the profile store 766, and the datasets that constitute the workspace can be stored in the file store 765. The datasets stored in the file store 765 may include, for example, each page of the workspace and one or more data structure representations 761 of the design being edited that can be rendered from each active workspace file.

[0084] In addition to or instead of this, each file may be associated with metadata that identifies the application service used to create it. In some embodiments, this metadata identifies the default application service for viewing, using, or updating the file.

[0085] Furthermore, in some embodiments, the service component 760 provides the web-based application 80 with a workspace representation 759 representing a workspace associated with a user. The workspace representation 759 identifies, for example, individual files associated with the user and / or user account. The workspace representation 759 can also identify a set of files consisting of multiple files. Each file contains one or more pages, and each page contains multiple objects that constitute a design interface.

[0086] Meanwhile, on the user device 10, the user can view workspace representations via the web-based application 80 and choose to open workspace files via the web-based application 80. In some embodiments, when the user chooses to open one of the active workspace files 763, the web-based application 80 launches a canvas 722. For example, the IAP 700 can launch an HTML5.0 canvas as a component of the web-based application 80, and the rendering engine 720 can access one or more data structure representations 711 of the content rendered on the canvas 722.

[0087] The IAP700 implements multiple operating modes corresponding to each application service using the application service logic 716. As described above, the application service logic 716 associated with each application service can include instructions and data for configuring each component that makes up the IAP700, so that the functions and characteristics of that application service are incorporated into those components. Therefore, the application service logic 716 can configure the application framework and / or input interface 718 such that, for example, the appearance, functions, and configuration of the IAP700 change depending on the operating mode of the IAP700. Furthermore, the types of actions and interactions that a user can perform to register user input can also be configured differently depending on the operating mode. In addition, the input functions or user interface functions that a user can select or use as functions provided on the canvas 722 can also be configured differently depending on the operating mode. For example, when IAP700 is operating in whiteboard application service mode, the program interface 502 provides an input function that allows the user to select design elements in the form of "sticky notes." However, when IAP700 is switched to interactive graphic design application service mode, this "sticky note function" can be disabled. Instead, in this switched mode, the user can select from a number of shape options or previously designed objects, write a text message within it, and display it on the canvas 722.

[0088] Furthermore, the application service logic 716 can configure the operation of the rendering engine 720 so that its functions and behavior change in response to changes in the application service being used. In this way, the rendering engine 720 functions to switch its behavior depending on which of the different operating modes corresponds to the specific application service that is currently active. For example, by changing the settings of the rendering engine 720, it is possible to change the appearance of the canvas 722, the appearance of content elements rendered on the canvas 722 (e.g., visual attributes), the behavior or representation of user interactions (e.g., whether the user's cursor or pointer device is displayed on the canvas 722), the type or details of the rendered content, the physics engine used by the rendering engine 720 when representing dynamic events (e.g., the process of moving an object), and what user operations are possible (e.g., whether the size of a selected object can be changed).

[0089] Furthermore, each application service can utilize a shared library and core functions. The shared library contains multiple content elements (e.g., graphic design elements), and the core functions enable the sharing and updating of design elements among various application services available via IAP700. Additionally, a workspace file created and edited using one application service can be used in another application service. Moreover, switching between application services can be seamless. For example, user computing device 10 can open a workspace file using a first application service (e.g., an interactive graphic design application service for UIX design), and then seamlessly switch to a second application service (e.g., a whiteboard application service) without closing the workspace file, while still using the same file. In some embodiments, each application service is configured to allow the user to update the workspace file even when other computing devices are simultaneously using the same workspace file (e.g., in a collaborative environment). In some embodiments, the user can switch between application services by switching the IAP700 mode. Even after switching modes, multiple application services can be configured to use the same workspace file.

[0090] Example of item division The various aspects of this disclosure are described below in numbered sections.

[0091] Section 1 A method implemented by a computer, The steps include generating a layer containing code-defined visual assets on a canvas within the design environment based on external code assets, The steps include receiving input associated with changes to be applied to the code-defined visual asset in the layer, The steps include making changes to the code-defined visual asset in the layer based on the input received, The steps include rendering the modified code-defined visual asset onto the canvas, A method that includes this.

[0092] Section 2 The method according to the first paragraph, wherein the code-defined visual asset within the layer is modified without making any changes to the external code asset.

[0093] Section 3 The method according to paragraph 1 or 2, wherein the code-defined visual asset in a layer is modified without modifying instances of the code-defined visual asset existing in other layers on the canvas within the design environment.

[0094] Section 4 The step of generating the layer that includes the code-defined visual asset is, The steps include: copying the source code of the external code asset as the source code associated with the layer in the canvas; The steps include modifying the source code associated with the layer in the canvas based on the matching relationship between the default export name in the source code and the name of the code-defined visual asset in the layer, The method described in any of paragraphs 1 to 3, including the method described in paragraph 1 to 3.

[0095] Section 5 The step of modifying the source code associated with the layer in the canvas is, The steps include identifying that the default export name is undefined in the source code of the external code asset, The steps include modifying the source code associated with the layer in the canvas so that the function associated with the name of the code-defined visual asset is defined as the default export of the layer in the canvas, The method described in paragraph 4, including the method described in paragraph 4.

[0096] Section 6 The step of modifying the source code associated with the layer in the canvas is, The steps include identifying that the source code of the external code asset contains a default export name different from the exported code associated with the layer, The steps include modifying the source code associated with the layer in the canvas so that the default export name defined in the modified source code is associated with the code to be exported, instead of the default export name of the external code asset; The method described in paragraph 4 or 5, including the method described in paragraph 4 or 5.

[0097] Section 7 The step of generating the layer that includes the code-defined visual asset is, Steps to insert a still image of the visual asset into the layer on the canvas. The method described in any of paragraphs 1 to 6, including the method described in paragraph 1 to 6.

[0098] Section 8 The method according to any one of paragraphs 1 to 7, further comprising the step of exporting the modified code-defined visual asset to a new external code asset.

[0099] Section 9 The input received as an input associated with the changes applied to the code-defined visual asset in the layer consists of changes made in one or more control panels in the design environment to one or more properties associated with one or more visual components in the code-defined visual asset. The step of making changes to the code-defined visual asset in the layer is, Steps to edit the code that defines the code-defined visual asset in the layer based on the input received. The method described in any of paragraphs 1 to 8, including the method described in paragraph 1 to 8.

[0100] Section 10 The method according to any one of paragraphs 1 to 9, wherein the input received as an input associated with the change applied to the code-defined visual asset in the layer consists of a change made to the code associated with the code-defined visual asset.

[0101] Section 11 The steps include detecting changes made to the external code asset from which the layer was generated, The steps include making changes to the code-defined visual asset on the canvas based on the detected changes, The method described in any of paragraphs 1 to 10, further including the method described in any of paragraphs 1 to 10.

[0102] Section 12 A method implemented by a computer, To edit a code-defined visual asset, the steps include importing the code-defined visual asset into a canvas within the design environment, The steps include receiving a change request for the imported code-defined visual asset, The steps include modifying the code file associated with the code-defined visual asset based on the received request, The steps include updating the canvas to reflect the modified code file based on how the code-defined visual asset is imported into the canvas, A method that includes this.

[0103] Section 13 The step of importing the aforementioned code-defined visual asset into the canvas is: Steps to import the code-defined visual asset into a new layer on the canvas within the design environment. The method described in paragraph 12, including the method described in paragraph 12.

[0104] Section 14 The method according to paragraph 13, wherein the new layer on the canvas in the design environment consists of a copy of another layer on the canvas in the design environment.

[0105] Section 15 The step of importing the aforementioned code-defined visual asset into a new layer on the canvas is: The steps include creating a code file associated with the aforementioned new layer, The steps include adding the code associated with the code-defined visual asset to the created code file, The method described in paragraph 12 or 13, including the method described in paragraph 12 or 13.

[0106] Section 16 The method according to any of paragraphs 12 to 15, wherein the changes to the code file are made separately from the code files associated with other layers on the canvas in the design environment.

[0107] Section 17 The method described in any of paragraphs 12 to 16, wherein the changes to the code file are made separately from the external code file associated with the imported code-defined visual asset.

[0108] Section 18 The step of importing the aforementioned code-defined visual asset into the canvas is: Steps to import the code-defined visual asset as an instance onto the canvas in the design environment. Includes, The method described in any of paragraphs 12 to 17, wherein the instance references an external code file associated with the code-defined visual asset.

[0109] Section 19 The step of modifying the code file associated with the code-defined visual asset is, Steps to modify the aforementioned external code file The method described in paragraph 18, including the method described in paragraph 18.

[0110] Section 20 The step of updating the canvas to reflect the changes to the code file associated with the code-defined visual asset is: The step of re-rendering each instance of the code-defined visual asset that references the external code file on the canvas in the design environment. The method described in paragraph 18 or 19, including the method described in paragraph 18 or 19.

[0111] Section 21 The steps include receiving a request to separate the instance from other instances that reference the external code file, In response to the request, The steps include generating a new external code file based on the external code file associated with the aforementioned code-defined visual asset, The steps include associating the instance with the new external code file, The method described in any of paragraphs 18 to 20, including the method described in any of paragraphs 18 to 20.

[0112] Section 22 The steps include receiving a change request for the instance of the code-defined visual asset, The steps include modifying the new code file based on the received request without making any changes to the code file associated with the code-defined visual asset, The steps include updating the instance on the canvas to reflect the new code file after the changes, The method described in paragraph 21, further including the method described in paragraph 21.

[0113] Section 23 At least one memory containing executable instructions, One or more processors, A processing system comprising, A processing system in which one or more processors are configured to cause the processing system to perform the method described in any of paragraphs 1 to 22 by executing the executable instructions.

[0114] Section 24 A processing system comprising means for performing the method described in any of paragraphs 1 to 22.

[0115] Section 25 A non-temporary computer-readable medium containing executable instructions that, when processed by one or more processors, cause one or more processors to perform the method described in any of paragraphs 1 to 22.

[0116] Any component described in any claim of the patent claims and / or any combination of any element described in this specification, regardless of the form of the combination, shall be included in the scope intended and protected by the present invention.

[0117] While various embodiments are described herein, these are presented for illustrative purposes only and are not intended to exhaust or limit all embodiments of the disclosure. Those skilled in the art will see that many modifications and variations are possible without departing from the scope and spirit of the embodiments described herein.

[0118] Aspects of this embodiment can be embodied as a system, method, or computer program product. Accordingly, embodiments of this disclosure can take the form of hardware-only embodiments, software-only embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, all of which may be collectively referred to as “modules,” “systems,” or “computers” in this specification. Furthermore, any hardware and / or software technology, process, function, component, engine, module, or system described herein can be implemented as a single circuit or a set of circuits. In addition, embodiments of this disclosure can take the form of a computer program product embodied as computer-readable program code recorded on one or more computer-readable media.

[0119] One or more computer-readable media can be used in any combination. The computer-readable media may be computer-readable signal media or computer-readable storage media. The computer-readable storage media may be, for example, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, devices, or any suitable combination thereof. More specific examples of computer-readable storage media (but not exhaustively listed) include electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this specification, the computer-readable storage medium may be any tangible medium capable of storing (storing) programs for use by, or in association with, a system, apparatus, or device that executes instructions.

[0120] The embodiments of this disclosure have been described above with reference to flowcharts and / or block diagrams illustrating methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a programmable data processing device, such as a general-purpose computer or a dedicated computer, to generate a machine. The instructions, when executed by the processor of the computer or programmable data processing device, realize the implementation of a function or operation defined in one or more blocks of the flowchart and / or block diagram. Such processors may include, but are not limited to, general-purpose processors, dedicated processors, application-specific processors, or field-programmable gate arrays.

[0121] The flowcharts and block diagrams shown in the drawings illustrate architectures, functions, and operations that may constitute embodiments of systems, methods, and computer program products according to various embodiments of the present disclosure. In this sense, each block in a flowchart or block diagram can be considered to represent a module, segment, or portion of code containing one or more executable instructions for implementing the one or more logical functions defined in the block. It should also be noted that in some alternative embodiments, the functions shown in the blocks may be executed in an order different from that shown in the drawings. For example, two blocks shown as consecutive blocks may actually be executed substantially simultaneously, or in some cases, these blocks may be executed in reverse order depending on the related functions. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, may also be implemented by a dedicated hardware-based system or a combination of dedicated hardware and computer instructions that executes the functions or operations defined in the blocks.

[0122] While the above description is directed towards embodiments of the present disclosure, it is possible to devise other embodiments and further embodiments of the present disclosure, provided that they do not deviate from the basic scope of the present disclosure, and such scope is defined by the claims set forth below. [Explanation of symbols]

[0123] 10 User Computing Devices 80 Web-based applications 100 Computing Devices 102 processors 104 Input / Output (I / O) Device Interfaces 106 Network Interfaces 108 Input / Output (I / O) Devices 110 Network 112 Interconnection wiring, bus 114 Storage device 116 memory 122 Code Management Engine 124 Graphic Design Engines 200 Design Environments 205 Canvas 210 Layout Panel 222, 224, 226 Visual Assets 230 Control Panel 240 Code Panel Code Assets 302, 304, 306, 308 310 Canvas 312, 314, 316 Code Instances, Object Instances 322, 324, 326 Code Layers 402 Code Layer Import Request 406 Rewritten code 410 Changes to the Code Layer 420 Changes to external code assets 700 Interactive Application Platform (IAP) 711, 761 Data Structure Representation (DSR) 716 Application Service Logic (ASL) 718 Input Interfaces, Interface Components 720 Rendering Engine 722 Canvas 755 Web Resources 758 Site Manager 759 Workspace representation 760 Service Components 763 Active Workspace File 765 File Store 766 User Profile Store

Claims

1. A method implemented by a computer, The steps include generating a layer containing code-defined visual assets on a canvas within the design environment based on external code assets, The steps include receiving input associated with changes to be applied to the code-defined visual asset in the layer, The steps include making changes to the code-defined visual asset in the layer based on the input received, The steps include rendering the modified code-defined visual asset onto the canvas, A method that includes this.

2. The method according to claim 1, wherein the code-defined visual asset in the layer is modified without making any changes to the external code asset.

3. The method according to claim 1, wherein the code-defined visual asset in a layer is modified without modifying instances of the code-defined visual asset existing in other layers on the canvas within the design environment.

4. The step of generating the layer that includes the code-defined visual asset is, The steps include: copying the source code of the external code asset as the source code associated with the layer in the canvas; The steps include modifying the source code associated with the layer in the canvas based on the matching relationship between the default export name in the source code and the name of the code-defined visual asset in the layer, The method according to claim 1, including the method described in claim 1.

5. The step of modifying the source code associated with the layer in the canvas is, The steps include identifying that the default export name is undefined in the source code of the external code asset, The steps include modifying the source code associated with the layer in the canvas so that the function associated with the name of the code-defined visual asset is defined as the default export of the layer in the canvas, The method according to claim 4, including the method described in claim 4.

6. The step of modifying the source code associated with the layer in the canvas is, The steps include identifying that the source code of the external code asset contains a default export name different from the exported code associated with the layer, The steps include modifying the source code associated with the layer in the canvas so that the default export name defined in the modified source code is associated with the code to be exported, instead of the default export name of the external code asset; The method according to claim 4, including the method described in claim 4.

7. The step of generating the layer that includes the code-defined visual asset is, Steps to insert a still image of the visual asset into the layer on the canvas. The method according to claim 1, including the method described in claim 1.

8. The method according to claim 1, further comprising the step of exporting the modified code-defined visual asset to a new external code asset.

9. The input received as an input associated with the changes applied to the code-defined visual asset in the layer consists of changes made in one or more control panels in the design environment to one or more properties associated with one or more visual components in the code-defined visual asset. The step of making changes to the code-defined visual asset in the layer is, Steps to edit the code that defines the code-defined visual asset in the layer based on the input received. The method according to claim 1, including the method described in claim 1.

10. The method according to claim 1, wherein the input received as an input associated with the change applied to the code-defined visual asset in the layer consists of a change made to the code associated with the code-defined visual asset.

11. The steps include detecting changes made to the external code asset from which the layer was generated, The steps include making changes to the code-defined visual asset on the canvas based on the detected changes, The method according to claim 1, further comprising:

12. At least one memory containing executable instructions, One or more processors, A processing system comprising, The one or more processors execute the executable instructions to the processing system, The steps include generating a layer containing code-defined visual assets on a canvas within the design environment based on external code assets, The steps include receiving input associated with changes to be applied to the code-defined visual asset in the layer, The steps include making changes to the code-defined visual asset in the layer based on the input received, The steps include rendering the modified code-defined visual asset onto the canvas, A processing system configured to perform the following.

13. The processing system according to claim 12, wherein the code-defined visual asset in the layer is modified without modifying the external code asset and without modifying instances of the code-defined visual asset existing in other layers on the canvas within the design environment.

14. When generating the layer which includes the code-defined visual asset, one or more processors provide the processing system with the following: The steps include: copying the source code of the external code asset as the source code associated with the layer in the canvas; The steps include modifying the source code associated with the layer in the canvas based on the matching relationship between the default export name in the source code and the name of the code-defined visual asset in the layer, The processing system according to claim 12, configured to perform the following.

15. When modifying the source code associated with the layer in the canvas, one or more processors instruct the processing system to: The steps include identifying that the default export name is undefined in the source code of the external code asset, The steps include modifying the source code associated with the layer in the canvas so that the function associated with the name of the code-defined visual asset is defined as the default export of the layer in the canvas, The processing system according to claim 14, configured to perform the following:

16. When modifying the source code associated with the layer in the canvas, one or more processors instruct the processing system to: The steps include identifying that the source code of the external code asset contains a default export name different from the exported code associated with the layer, The steps include modifying the source code associated with the layer in the canvas so that the default export name defined in the modified source code is associated with the code to be exported, instead of the default export name of the external code asset; The processing system according to claim 14, configured to perform the following:

17. When generating the layer which includes the code-defined visual asset, one or more processors provide the processing system with the following: Steps to insert a still image of the visual asset into the layer on the canvas. The processing system according to claim 12, configured to perform the following.

18. The one or more processors, in relation to the processing system, Step to export the modified code-defined visual asset to a new external code asset. The processing system according to claim 12, configured to further perform the following.

19. The input received as an input associated with the changes applied to the code-defined visual asset in the layer consists of changes made in one or more control panels in the design environment to one or more properties associated with one or more visual components in the code-defined visual asset. When making changes to the code-defined visual asset in the layer, one or more processors instruct the processing system to: Steps to edit the code that defines the code-defined visual asset in the layer based on the input received. The processing system according to claim 12, configured to perform the following.

20. The one or more processors, in relation to the processing system, The steps include detecting changes made to the external code asset from which the layer was generated, The steps include making changes to the code-defined visual asset on the canvas based on the detected changes, The processing system according to claim 12, configured to further perform the following.

Images