Taking snapshots of code-defined visual content in computing systems

Abstract

Embodiments of this disclosure provide a technology for rendering visual content in a design environment. [Solution] An exemplary method includes step 410 detecting that a code-defined visual asset loaded onto a canvas in a design environment has been modified. The code-defined visual asset is copied to an invisible rendering environment in the design environment (420). The code-defined visual asset is rendered in the invisible rendering environment based on the execution of code associated with the code-defined visual asset in the invisible rendering environment (430). An image of the rendered code-defined visual asset is retrieved (440), and the retrieved image of the code-defined visual asset is displayed in the design environment (450).

Description

[[Technical Field]]

[0001] Embodiments of the present disclosure generally relate generally to graphic design tools, and in particular, to a technique for efficiently rendering visual content in a graphic design tool. [[Background Art]]

[0002] In general, a graphic design tool is a tool for creating visual assets in a design environment. These visual assets can be defined, for example, as vector assets defined based on mathematical relationships between different components, or raster assets defined based on the absolute position of pixels. In some cases, a visual asset includes one or more containers defined in a vector coordinate space, and a raster asset (e.g., an image) can be inserted into the container. In addition, a visual asset can include any number of visual components, and the visual asset can be defined based on spatial relationships such as the relative positional relationship between the visual components contained therein.

[0003] In many workflows, the process of designing a visual asset is separated from the process of generating code used to implement the visual asset in an executable environment outside the design environment. Due to this separation, the design of visual assets is sometimes performed in the design environment, while coding is performed in a coding environment. However, since the supported functions of the design environment and the coding environment do not necessarily match, designs created in the design environment may not be reproduced on the coding environment side. In other words, the design of a visual asset created in the design environment only serves as a guide for creating the visual asset in the coding environment, and there have been cases where the appearance of the visual asset created in the coding environment does not match the appearance of the visual asset in the design environment.

[0004] Furthermore, even if the process of designing a visual asset and the process of generating the code used to implement that visual asset are separate, the code generated to implement the visual asset in the executable environment may be closely tied to the appearance of that visual asset. Because the appearance and the implementation code are thus tied, when implementing multiple instances of a visual asset, there was a possibility of code duplication within the codebase implementing that visual asset. Consequently, when making changes to each instance of the visual asset, it may have been necessary to repeatedly modify the same code. [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] As explained above, there is a need for techniques to implement visual content more effectively with executable code. [Means for solving the problem]

[0006] One embodiment of the present disclosure provides a technique for rendering visual content in a design environment. An exemplary method includes the step of detecting that a code-defined visual asset loaded onto a canvas in the design environment has been modified. The code-defined visual asset is copied to an invisible rendering environment in the design environment. The code-defined visual asset is rendered in the invisible rendering environment based on the execution of code associated with the code-defined visual asset in the invisible rendering environment. An image of the rendered code-defined visual asset is retrieved and the retrieved image of the code-defined visual asset is displayed in the design environment. [Effects of the Invention]

[0007] The technical advantage of the technology of this disclosure over the prior art is that visual assets can be rendered efficiently and securely in a design environment. When importing a code-defined visual asset into a design environment, a snapshot (e.g., a still image taken from the code-defined visual asset) may be rendered in the design environment for display to the user of the design environment. When rendering a snapshot in the design environment, embodiments of the disclosure may bypass the execution of the source code associated with the code-defined visual asset. By bypassing the execution of the source code associated with the code-defined visual asset, the use of computing resources can be reduced and security within the design environment can be enhanced. Generally, rendering still images uses less processing and memory resources than executing code, and therefore may enable a reduction in power and computing resource usage compared to rendering a code-defined visual asset in the design environment based on the execution of code associated with the code-defined visual asset. Furthermore, since no code is executed when loading a design containing a code-defined visual asset into the design environment, embodiments of the disclosure may prevent, or at least reduce the likelihood of, malicious code associated with the code-defined visual asset being executed in the design environment.

[0008] The above is a brief summary of the concept of the present invention. In order to understand the features of each of the various embodiments described above in detail, the concept of the present invention will be explained more specifically below, with reference to various embodiments. Some of these embodiments are also illustrated in the accompanying drawings. It should be noted that the accompanying drawings are merely illustrative of representative embodiments of the concept of the present invention and should not be interpreted as limiting the scope of this disclosure in any sense, and that other embodiments with equivalent effects also exist. [Brief explanation of the drawing]

[0009] [Figure 1]This figure shows a computer system configured to implement one or more embodiments of various embodiments of the present disclosure. [Figure 2] This figure shows a design environment in which visual components, according to several embodiments, are created and maintained as code-defined assets. [Figure 3] This message flow diagram shows the messages exchanged between the snapshot engine and the graphic design engine for the generation of snapshots of code-defined visual assets according to several embodiments. [Figure 4] This is a flowchart illustrating the process of generating snapshots of code-defined visual assets in a design environment according to several embodiments. [Figure 5] This figure shows a network computing system for implementing an interactive graphics application platform according to several embodiments. [Modes for carrying out the invention]

[0010] The following description includes many specific details to provide a deeper understanding of the various embodiments. However, it will be apparent to those skilled in the art that the concept of the present invention can be implemented even if one or more of these specific details are omitted.

[0011] Figure 1 shows a computing device 100 configured to implement one or more embodiments of various embodiments of the present invention. In one embodiment, the computing device 100 may be a desktop computer, a notebook computer, a smartphone, a personal digital assistant (PDA), or a tablet computer, or the computing device 100 may be any other type of computing device suitable for implementing one or more embodiments, configured to receive input, process data, and, if necessary, display images. The computing device 100 is configured to run a snapshot engine 122 (including a watchdog 123 and a snapshot generation unit 124) and a graphic design engine 126, which reside in memory 116.

[0012] It should be noted that the computing devices described herein are merely illustrative, and other technically feasible configurations are also included in the scope of this disclosure. For example, the functionality of computing device 100 may be implemented by running multiple instances of the snapshot engine 122 or graphic design engine 126 on a group of nodes within a distributed computing system and / or a cloud computing system. Alternatively, the snapshot engine 122 or graphic design engine 126 can be adapted to a variety of use cases and applications by running it on various hardware configurations, devices, or environments. In a third example, the snapshot engine 122 or graphic design engine 126 may be configured to run on other computing devices and / or other groups of computing devices.

[0013] In one embodiment, the computing device 100 comprises, but is not limited to, an interconnection bus 112 connecting one or more processors 102, an input / output (I / O) device interface 104 coupled to one or more input / output (I / O) devices 108, memory 116, storage device 114, and a network interface 106. The processors 102 can be any type of processing unit, such as a central processing unit (CPU), an image processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), an artificial intelligence (AI) accelerator, or any suitable processor implemented as a combination of different types of processing units (e.g., a configuration in which a CPU and a GPU work together). Generally, the processors 102 can be any technically feasible hardware unit capable of processing data and / or executing software applications. Furthermore, in the context of this disclosure, each element illustrated as a computing element within the computing device 100 may correspond to a physical computing system (e.g., a system in a data center) or a virtual computing instance running in a computing cloud.

[0014] The I / O devices 108 include devices that provide input, such as keyboards, mice, touch-sensitive screens, and microphones, and devices that provide output, such as display devices and speakers. In addition, the I / O devices 108 may also include devices that have both input-receiving and output-providing capabilities, such as touch panel screens and Universal Serial Bus (USB) ports. The I / O devices 108 can be configured to receive various inputs from the end user of the computing device 100 (e.g., a designer) and to provide various outputs (e.g., displays of digital images, digital video, or text) to the end user of the computing device 100. Furthermore, in some embodiments, one or more I / O devices 108 are configured to connect the computing device 100 to the network 110.

[0015] Network 110 is any technically feasible type of communication network that enables data exchange between computing device 100 and external entities or devices (e.g., network-attached computing devices such as web servers). For example, network 110 may include, in particular, a wide area network (WAN), a local area network (LAN), a wireless (Wi-Fi) network, and / or the internet.

[0016] The storage device 114 includes a non-volatile storage area for applications and data, and can consist of fixed or removable disk drives, flash memory devices, magnetic, optical, or solid-state storage devices such as CD-ROMs, DVD-ROMs, Blu-ray®, and HD-DVDs. The snapshot engine 122 and the graphic design engine 126 can be configured to be stored in the storage device 114 and loaded into memory 116 at runtime.

[0017] Memory 116 includes memory units of various types, such as random access memory (RAM) modules or flash memory units, or combinations thereof. The processor 102, I / O device interface 104, and network interface 106 are configured to read data from and write data to memory 116. Memory 116 contains various software programs executable by the processor 102 (including the snapshot engine 122 or the graphic design engine 126) and application data associated with these software programs.

[0018] Example of taking a snapshot of a code-defined visual asset in a design environment. Graphic design software, such as that implemented by the graphic design engine 126 shown in Figure 1, can render and design visual assets. Examples of such visual assets include user interface components that constitute the user interface being designed in the graphic design engine 126. In this specification, "visual asset" can refer to a group of visual components rendered in the user interface. Visual assets may be predefined (for example, as code templates in the graphic design engine 126, i.e., pre-designed code-defined visual components), or they may be designed and implemented in code by the user of the graphic design engine 126.

[0019] In the embodiments described herein, to enable the design of graphical interfaces using visual assets closely tied to implementation code, and to allow for easy modification of these visual assets, visual assets are defined, used, and reused as code-defined assets, and configured to be rendered as static snapshots in the design environment. Generally, a visual asset consisting of one or more visual components can be defined within an external code asset (e.g., a code file) that can be imported into the design environment. When importing a visual asset into the design environment, it can usually be imported into a layer on a canvas within the design environment. Layers on a canvas within the design environment allow for the spatial configuration of various visual assets within the canvas, and also allow for the separation of a visual asset from other visual assets and the external code assets from which it originates. Within the layer, the appearance and functionality of instances of visual assets in the design can be customized by changing various properties, actions, triggers, etc., without modifying the external code assets from which the visual assets originated. Furthermore, instances of visual assets and the code that defines such visual assets can also be exported to an external code asset so that they can be reused in other designs.

[0020] When a code-defined visual asset is modified in the design environment, the embodiments described herein detect that the appearance and / or functionality of the code-defined visual asset has been changed. As described herein, changes to the appearance and / or functionality of a code-defined visual asset may include, but are not limited to, changes within the layer on the canvas in which the code-defined visual asset resides. Changes may include changes to the source code associated with the code-defined visual asset, changes to the properties of the code-defined visual asset, changes to the size of the code-defined visual asset, etc. The modified code-defined visual asset may be rendered in the invisible rendering environment of the design environment, and a snapshot of the code-defined visual asset may be taken after rendering the code-defined visual asset in the invisible rendering environment. As described further herein, the snapshot may include a rasterized image of the code-defined visual asset, size information defining how the code-defined visual asset shifts the placement of other assets on the layer, and overflow information defining the amount of space the snapshot overflows onto other assets on the canvas. A snapshot of a code-defined visual asset is associated with the code-defined visual asset, and the snapshot is rendered in the design environment when the code-defined visual asset is imported into the design environment. By doing so, the embodiments described herein can reduce the amount of computing resources used to render the code-defined visual asset in the design environment and improve security in the design environment where the code-defined visual asset is rendered. As described in more detail herein, instead of rendering the code-defined visual asset based on the execution of code associated with the code-defined visual asset, a static image of the code-defined visual asset may be rendered in the design environment.Therefore, the embodiments described in the present specification perform operations with low computational cost for rendering code-defined visual assets in a design environment, and can prevent, or at least reduce the likelihood of, malicious code from being executed in the local environment when the code-defined visual assets contain arbitrary code.

[0021] Figure 2 is a diagram showing a design environment 200 according to some embodiments. In this design environment 200, visual components are created and maintained as code-defined assets. The design environment 200 can be rendered by one or both of the snapshot engine 122 and the graphic design engine 126 shown in Figure 1.

[0022] In the embodiments presented herein, in order to achieve a configuration that defines visual assets as code components and closely links the appearance of visual assets in the design environment 200 to the implementation code that defines the visual assets, implementation code for visual assets is created within the design environment 200, and these code-defined visual assets are enabled to be used as modifiable objects. As illustrated, the design environment 200 includes a layout panel 210, a canvas 205, a control panel 230, and a code panel 240. The layout panel 210 displays one or more layers associated with visual components in the design. The canvas 205 displays each instance of a code-defined visual asset.

[0023] Visual assets 222, 224, and 226 generally represent instances of mutually different code-defined visual assets (note that visual assets may exist other than those illustrated in FIG. 2). For example, visual assets 222 and 224 show various examples of visual assets for rendering non-interactive text on a graphical display. In contrast, visual asset 226 shows an example of an interactive visual asset (in the present example, visual asset 226 is illustrated as a calculator; however, this is merely an example, and it should be understood that any type of interactive visual asset is contemplated). In an interactive visual asset, interaction with a button changes text displayed in a text field, or triggers execution of an operation that changes the text displayed in the text field (for example, interaction with a button triggers execution of a mathematical operation, and the result thereof is displayed in the text field). To allow the properties and functions of each of visual assets 222, 224, and 226 to be modified independently of each other, these visual assets may be configured to be stored in respectively different code layers within canvas 205 and displayed in a list in layout panel 210.

[0024] Generally, external code assets can define the appearance and / or functionality of visual assets (for example, any of the visual assets 222, 224, or 226 shown in Figure 2). The appearance of a visual asset can be set, for example, via code-defined properties. Code-defined properties (for example, size, foreground and background colors, transparency, rotation, border thickness and type, etc.) are configured to be modifiable by the user in the design environment. The functionality of a visual asset can be set, for example, via triggers, actions, etc. Generally, a trigger corresponds to an operation that causes a code-defined visual asset to issue an event notification. This event notification triggers other code-defined visual assets to execute other code. An action, on the other hand, corresponds to an operation that causes code to be executed based on it (for example, an interaction with a button in the user interface that triggers code execution).

[0025] For example, an external code asset associated with a visual asset 222 that implements a static text block may include code that returns a command to render the static text block within the graphical user interface when the code is executed. Since the visual asset 222 does not have interactive functionality, the code associated with the visual asset 222 shown in the code panel 240 may be configured not to include code such as action handlers or triggers that are executed by user interaction with the visual asset 222. Visual design properties associated with the visual asset are configured to be modifiable. Changes to visual design properties can be made by changing various design parameter values ​​displayed in the control panel 230. As shown in the figure, the parameters of a static text block may include position-related parameters (e.g., position, text orientation, element (text block itself) orientation, inner padding, etc.), size-related parameters, etc. In some embodiments, the external code asset associated with the visual asset 222 may include predefined values ​​for one or more parameters, and these parameter values ​​may be displayed in the control panel 230. These parameter values ​​can be changed by the user of the design environment 200. Parameter values ​​can be changed by directly modifying the parameter values ​​on the control panel 230 or by changing the appearance of the visual asset 222 on the canvas 205. Similarly, an external code asset associated with the visual asset 224 may contain code that returns instructions to render multiple static text blocks with different properties within the graphical user interface.

[0026] On the other hand, since Visual Asset 226 is an asset with various interactive functional properties, the external code assets associated with it define both the appearance and functionality of Visual Asset 226. The code that defines Visual Asset 226 can include code for rendering Visual Asset 226 in a graphical user interface and code for executing various functions based on interactions with the visual components contained in Visual Asset 226. For example, the code that defines Visual Asset 226 can include code that issues triggers to change the text displayed in a text box or triggers to change the state of Visual Asset 226. Furthermore, the code that defines Visual Asset 226 can also include action processing code that executes specified actions within the graphical user interface. For example, action processing code such as code that defines the operation to be performed, code that clears memory such as buffers related to the calculator, or code that performs a previously specified operation can be assigned to a particular button.

[0027] Visual assets 222, 224, and 226, which represent instances of code-defined visual assets rendered on layers within canvas 205, can be modified in various ways within the design environment 200. Visual assets can be imported into the design environment 200 as code instances or code layers. Changes made to a code instance can be configured to be global changes (e.g., changes to external code assets associated with the visual asset), i.e., changes affecting the appearance and / or functionality of all instances of that visual asset within the design environment 200. Here, if a code-defined visual asset is imported as a code instance, and further (e.g., by conversion from the code instance as described above) one or more instances of that code-defined visual asset are generated within a code layer, then when a change is made to the version of the code instance of the code-defined visual asset, that change can be configured to propagate to each instance of that code-defined visual asset in the code layer within canvas 205. In some embodiments, changes to a visual asset may be recursive, and changes to a first code-defined visual asset used by a second code-defined visual asset may constitute changes to both the first and second code-defined visual assets (which may trigger the creation of updated snapshots of both the first and second code-defined visual assets, as described later). On the other hand, changes made to a code-defined visual asset within a layer on canvas 205 may be configured to be local changes that apply only to the specific layer in which the change was made.

[0028] Changes to code-defined visual assets can be made based on changes to parameter values ​​in the control panel 230 or changes to the code in the code panel 240. When a code-defined visual asset in the canvas 205 is selected, the control panel 230 displays one or more control elements for changing the visual appearance of the code-defined visual asset. At the same time, the code panel 240 can be configured to display the code that defines the code-defined visual asset. When changes are made to the properties of the code-defined visual asset in the control panel 230, or when changes are made to the code that defines the code-defined visual asset in the code panel 240, the appearance and / or layout of the code-defined visual asset in the canvas 205 can be updated. In some embodiments, when the visual appearance of a code-defined visual asset is changed, previously generated still images (i.e., snapshots) of the code-defined visual asset can be replaced with updated still images of the updated code-defined visual asset.

[0029] To generate snapshots of code-defined visual assets rendered and modified in the design environment 200, a watchdog (e.g., the watchdog 123 of the snapshot engine 122 shown in Figure 1) monitors the design environment for changes to the code-defined visual assets. Generally, the watchdog 123 may monitor inputs to one or more control panels (e.g., control panel 230 and / or code panel 240) that change the appearance and / or functionality of the code-defined visual assets 222, 224, and 226. When the watchdog 123 detects a change to a code-defined visual asset, it may mark the code-defined visual asset as a modified visual asset and instruct the snapshot generation unit (e.g., the snapshot generation unit 124 shown in Figure 1) to generate an updated snapshot of the code-defined visual asset.

[0030] In some embodiments, the watchdog 123 may determine that a snapshot of a code-defined visual asset should be generated based on a delta (or other amount of change) between the pre-modified and post-modified code-defined visual assets. The delta may include, for example, a defined minimum change in the appearance of the visual asset that causes the watchdog 123 to designate the code-defined visual asset as a modified visual asset for which a snapshot should be generated. This defined minimum change may be, for example, the number of pixels by which the size or position of the visual components of the visual asset has changed, or the number of color steps between the color of the pre-modified and post-modified visual assets. If the watchdog 123 determines that the amount of change between the pre-modified and post-modified visual assets is less than the defined delta, then the watchdog 123 may determine that the visual asset has not changed sufficiently and there is no need to generate a new snapshot. In another scenario, the watchdog 123 determines that the amount of change between the pre-modification visual asset and the post-modification visual asset is greater than a defined delta, and instructs the snapshot generation unit 124 to generate a snapshot of the modified visual asset.

[0031] In some embodiments, the watchdog 123 may monitor the code associated with a code-defined visual asset to determine whether the visual asset has been modified sufficiently to justify creating a new snapshot of the visual asset. For code that changes the functionality of a visual asset or alters its appearance when a specific event occurs, the watchdog 123 may determine that the current snapshot associated with the visual asset still adequately represents the visual asset. Therefore, in such cases, the watchdog 123 may determine that it is not necessary to generate a new snapshot. However, for code that alters the appearance of a visual asset and is not associated with the occurrence of a specific event, the watchdog 123 may compare parameter values ​​that define the appearance of the visual asset between the pre-modified code and the post-modified code. If the delta between the pre-modified code and the post-modified code exceeds the defined delta, the watchdog 123 then determines that the amount of change between the pre-modified and post-modified visual assets is greater than the defined delta and causes the snapshot generation unit 124 to generate a snapshot of the modified visual asset.

[0032] In some embodiments, while updating the snapshot associated with the modified visual asset, the watchdog 123 may instruct the graphical design engine 126 to change the appearance of the snapshot associated with the modified visual asset on the canvas 205. For example, the watchdog 123 may instruct the graphical design engine 126 to add defined visual effects to the modified visual asset in the design environment 200 (e.g., adding a shimmer effect, adding a glow effect, highlighting the modified visual asset, etc.).

[0033] Generally, when the watchdog 123 determines that the appearance of a visual asset has changed sufficiently, the snapshot generation unit 124 generates a snapshot of the code-defined visual asset in the invisible rendering environment of the design environment 200. Generally, the invisible rendering environment can be a canvas or other rendering environment that logically exists within the design environment 200 but is not visible to the user. For example, the invisible rendering environment may be "rendered" using a reference pixel position outside the pixel grid of the display on which the design environment 200 is rendered. When a new snapshot is generated, the current snapshot associated with the visual asset no longer represents the appearance of the visual asset. To generate a snapshot of a code-defined visual asset, the snapshot generation unit 124 copies the source code associated with the code-defined visual asset, including changes to the code-defined visual asset, from the canvas 205 to the invisible rendering environment of the design environment 200. The source code is executed in the invisible rendering environment to cause the visual asset to be rendered in the invisible rendering environment, but may be invisible to the user performing tasks in the design environment 200. After the snapshot generation unit 124 renders the visual asset in an invisible rendering environment, the snapshot generation unit 124 acquires a new image of the visual asset and replaces any previously acquired images associated with the visual asset with the new image of the visual asset.

[0034] In some embodiments, the snapshot generation unit 124 may generate a snapshot of the modified visual asset immediately after the rendering operation begins in the invisible rendering environment. Generally, the snapshot generation unit 124 may do so when the visual asset includes statically defined visual content. In some embodiments, the modified visual asset may include dynamically generated or loaded visual content from an external resource. In such cases, the snapshot generation unit 124 may operate with a defined time delay between the start of the rendering operation in the invisible rendering environment and the acquisition of a snapshot of the modified visual asset in the invisible rendering environment. In some embodiments, the delay may be defined based on the type of dynamically generated or loaded visual content, defined a priori, or otherwise configured to load dynamically generated content or content from an external resource into the invisible rendering environment. After the defined time length has elapsed from the start of the rendering operation in the invisible rendering environment, the snapshot generation unit 124 may acquire a snapshot of the modified visual asset.

[0035] In some embodiments, the snapshot generation unit 124 may inspect code-defined visual assets for components that include reference information to external resources. Generally, this reference information to external resources may be contained in various visual components that function as containers into which other visual content is loaded. Before generating snapshots of the code-defined visual assets, placeholder visual content may be generated for these components and loaded into them in an invisible rendering environment. Placeholder visual content may include, for example, still images associated with the external resources, keyframes or defined thumbnails associated with the external resources that are video or animated content. The size of the placeholder visual content may be based on the size of the component in which the placeholder visual content is placed in an invisible rendering environment.

[0036] In some embodiments, the snapshot generation unit 124 may identify a bounding box surrounding a code-defined visual asset, and thus the extent to which the snapshot captures the code-defined visual asset in an invisible rendering environment. To this end, the snapshot generation unit 124 may identify the size of the code-defined visual asset and generate a bounding box based on the identified size and bleed amount of the code-defined visual asset. Generally, the bleed amount corresponds to the additional space in the invisible rendering environment beyond the boundaries of the code-defined visual asset and may reflect dynamically generated content that overflows beyond the boundaries of the code-defined visual asset. When the snapshot generation unit 124 captures a snapshot of the code-defined visual asset, the snapshot generation unit 124 may superimpose the bounding box onto the code-defined visual asset in the invisible rendering environment (for example, using the center of the code-defined visual asset as the center of the bounding box), and then capture an image of the content of the bounding box.

[0037] In some embodiments, the snapshot generation unit 124 may acquire an image of a code-defined visual asset based on a bounding box identified based on the size of the code-defined visual asset and the position of the visual components of the code-defined visual asset. Generally, the positions of these code-defined visual assets may include inner margins or other additional space that allows for the acquisition of overflowing content (e.g., dropdown boxes or other content controls extending beyond the boundaries of the code-defined visual asset, or shadow effects applied to the components of the code-defined visual asset). As described above, the snapshot generation unit 124 may acquire an image of the bounding box content by superimposing the bounding box onto the code-defined visual asset in an invisible rendering environment (e.g., using the center of the code-defined visual asset as the center of the bounding box).

[0038] In some embodiments, the snapshot generation unit 124 may generate snapshots of visual components as high-resolution images. The high-resolution image may be, for example, an image defined based on the resolution and multiplier of the screen on which the design environment 200 is rendered. The high-resolution image may be obtained at a resolution calculated as the product of the screen resolution (e.g., pixels per unit of measurement) and the multiplier. For example, for a screen where the image is rendered at 144 pixels per inch, the multiplier may be defined as 2, and the snapshot generation unit 124 may generate a snapshot at a resolution of 144 × 2 = 288 pixels per inch.

[0039] Figure 3 is a message flow diagram 300 showing messages exchanged between a snapshot engine 122 (including a watchdog 123 and a snapshot generation unit 124) and a graphical design engine 126 in several embodiments to generate snapshots of code-defined visual assets in the design environment.

[0040] To generate a snapshot of code-defined visual assets, a canvas containing one or more layers may have various code-defined visual assets placed on at least one of those layers. As already mentioned, each layer may correspond to a canvas container in which the code-defined visual assets associated with that layer are separated from the code-defined visual assets associated with other layers of the canvas. Code-defined visual assets may be modified while operating within a layer (for example, through changes to the design of the code-defined visual assets on the canvas, or through changes to the code-defined visual assets through changes to parameters in the control panel in the design environment).

[0041] As illustrated, the visual asset design change 302 can be input to the graphical design engine 126. In various examples, the visual asset design change 302 changes the appearance of the visual asset to such an extent that the current snapshot associated with the code-defined visual asset no longer accurately reflects the visual appearance of the code-defined visual asset.

[0042] In block 304, the watchdog 123 detects design changes to a visual asset. Generally, when detecting a design change, the watchdog 123 may compare the pre-change design of the visual asset with the post-change visual asset to determine if the appearance of the visual asset has changed by a threshold amount, and if the current snapshot associated with the visual asset no longer accurately reflects the visual appearance of the code-defined visual asset. For example, a threshold size difference or threshold color difference may be used to determine if a design change has been made to the visual asset. In another example, an enumeration of components associated with a code-defined visual asset may be used to determine if a visual component has been added to or removed from the code-defined visual asset. Generally, if a visual component has been added or removed, in block 304, the watchdog 123 may determine that there is a sufficient difference between the pre-change and post-change designs of the visual asset, and therefore a design change has been made to the visual asset.

[0043] Based on the detection of design changes to the visual asset in block 304, the watchdog 123 outputs a snapshot generation trigger 306 to the snapshot generation unit 124. Generally, the snapshot generation trigger 306 indicates to the snapshot generation unit 124 that (1) the code-defined visual asset has changed sufficiently, and the current snapshot associated with the visual asset no longer accurately reflects the visual appearance of the code-defined visual asset, and (2) a new snapshot should be taken for the visual asset. In response to receiving the snapshot generation trigger 306, in block 308, the snapshot generation unit 124 renders the updated visual asset in an invisible rendering environment. Generally, in order to render the updated visual asset in an invisible rendering environment, the canvas of the design environment may be established outside the coordinates of the screen on which the design environment is rendered. The updated visual asset can be rendered by copying the code associated with the updated visual asset to the invisible rendering environment for execution and executing the code associated with the updated visual asset.

[0044] In block 308, after rendering the updated visual asset in an invisible rendering environment, in block 310, the snapshot generation unit 124 takes a snapshot of the updated visual asset. In some embodiments, the snapshot may be taken after instructing the graphical design engine 126 to render the updated visual asset in an invisible rendering environment. In some embodiments, if the updated visual asset includes an external asset to be rendered, or an asset that takes a certain amount of time to render, block 310 may be executed after a defined length of time has elapsed after rendering the updated visual asset in an invisible rendering environment.

[0045] After taking a snapshot in block 310, the snapshot generation unit 124 provides the snapshot 312 to the graphical design engine 126. In block 314, the graphical design engine 126 renders the updated visual assets in the design environment using the snapshot 312. When rendering the updated visual assets in the design environment, the graphical design engine 126 renders the updated visual assets in the appropriate position and reflows other elements on the canvas based on the size of the updated visual assets to match the new layout of the design on the canvas in the design environment.

[0046] Figure 4 is a flowchart illustrating an exemplary operation 400 for generating a snapshot of a code-defined visual asset according to several embodiments. Operation 400 may be performed, for example, by having a snapshot generation unit (e.g., the snapshot generation unit 122 shown in Figure 1) run on a computing system including one or more processors, such as the processor 102 of the computing device 100 shown in Figure 1.

[0047] As illustrated, operation 400 begins in block 410, and the snapshot generation unit 122 detects that a code-defined visual asset loaded onto the design environment canvas has been modified.

[0048] In some embodiments, detecting changes to a code-defined visual asset may include detecting when the amount of change exceeds a threshold. The threshold amount may be, for example, a threshold size difference (e.g., in pixels), a threshold color difference, or a threshold number of visual components being added or removed.

[0049] In block 420, operation 400 proceeds with the snapshot generation unit 122 copying the code-defined visual asset to the invisible rendering environment of the design environment.

[0050] In block 430, operation 400 proceeds with the snapshot generation unit 122 rendering the code-defined visual asset in the invisible rendering environment based on the execution of the code associated with the code-defined visual asset in the invisible rendering environment.

[0051] In some embodiments, when rendering of a code-defined visual asset is initiated in an invisible rendering environment, an image of the rendered code-defined visual asset can be obtained.

[0052] In some embodiments, an image of the rendered code-defined visual asset is obtained based on the elapsed time since rendering of the code-defined visual asset began in an invisible rendering environment.

[0053] In some embodiments, an image of the rendered code-defined visual asset is obtained based on an indication received from the invisible rendering environment that the rendering of the code-defined visual asset is complete. In some embodiments, an image of the rendered code-defined visual asset may be obtained based on the completion of rendering of the code-defined visual asset in the invisible rendering environment or a defined timeout period, whichever comes first.

[0054] In block 440, operation 400 proceeds with the snapshot generation unit 122 acquiring an image of the rendered code-defined visual asset.

[0055] In some embodiments, obtaining an image of a rendered code-defined visual asset may involve identifying one or more visual components of the code-defined visual asset from which an external resource has been loaded. Placeholder visual content may be generated for the identified one or more visual components. The identified one or more visual components in the image of the rendered code-defined visual asset may be replaced with the placeholder visual content. In some embodiments, the external resource may include a video, and the still images may be selected from the video or be keyframes or thumbnails otherwise associated with the video.

[0056] In some embodiments, obtaining an image of a rendered code-defined visual asset may involve determining the size of the code-defined visual asset. A bounding box may be generated based on the size and bleed of the code-defined visual asset. The content within the bounding box may then be obtained. Generally, the bounding box is superimposed on the code-defined visual asset in the invisible rendering environment of the design environment (for example, such that the center point of the code-defined visual asset is also the center point of the bounding box).

[0057] In some embodiments, obtaining an image of a rendered code-defined visual asset may involve generating a bounding box based on the size of the code-defined visual asset and the position of each visual component of the code-defined visual asset. The content within the bounding box may then be obtained. Generally, the bounding box is superimposed on the code-defined visual asset in the invisible rendering environment of the design environment.

[0058] In block 450, operation 400 proceeds with the snapshot generation unit 122 storing the acquired image of the code-defined visual asset rendered for rendering in the design environment.

[0059] In some embodiments, operation 400 further includes replacing previously generated images of a code-defined visual asset with images obtained on the canvas in the design environment without executing any code associated with the code-defined visual asset in the design environment.

[0060] In some embodiments, the resolution of the acquired image is higher than the resolution at which the code-defined visual asset is rendered on the screen where the design environment is displayed.

[0061] Figure 5 shows a network computing system for implementing an interactive application platform on a user computing device, according to several embodiments. A network computing system like the one shown in Figure 5 can be implemented using one or more servers that communicate with the user computing device over one or more networks. The network computer system 550 shown in Figure 5 can correspond, for example, to the computing device 100 shown in Figure 1, and can be used to generate and / or modify visual content based on a generating artificial intelligence model and input prompts specifying design policies for the visual content.

[0062] In some embodiments, an interactive application platform ("IAP500") can be implemented on the user computing device 10 through operations performed by the network computer system 550. In some embodiments, the IAP500 can be implemented when a user initiates a session (for example, when a user accesses a website) and receives program resources for the IAP500. The browser component implements the IAP500 by executing the received program resources and also has the function of receiving user input and rendering content based on or in response to that input. As described above, by implementing the IAP500, users can create various types of content such as interactive graphic designs, art, whiteboard content, program code renderings, presentations, and / or text content. Also as described above, the IAP500 may include logic ("application service logic (ASL) 516") for implementing one or more application services. Each application service is executed via the IAP500 and provides the functionality and user experience corresponding to that application service. Furthermore, IAP500 is implemented to share resources such as canvases, workspace files, and design element libraries across multiple application services. In addition, IAP500 allows for the simultaneous use of multiple application services during a given online session or in relation to a specific application service.

[0063] In some embodiments, a user of computing device 10 implements IAP500 by operating a web-based application 80 to access a network site, obtaining program resources from the network site, and executing them. The web-based application 80 can implement the functions of IAP500 by executing logic such as scripts and code ("programmatic components"). In some embodiments, the web-based application 80 can be compatible with commercially available browsers such as GOOGLE CHROME (developed by GOOGLE, INC.) and SAFARI (developed by APPLE, INC.). In some embodiments, each process of IAP500 can be implemented as embedded code, such as scripts, downloaded by the web-based application 80 from the network site. For example, the web-based application 80 can implement each process of IAP500 by executing code embedded in a web page. Furthermore, the web-based application 80 can also obtain other scripts and program resources (such as libraries) from network sites or other local or remote locations by executing scripts. For example, a web-based application 80 can execute JavaScript® embedded in an HTML resource (for example, a web page compliant with an HTML version such as HTML5.0, structured according to a standard specification published by the W3C Consortium or the WHATWG Consortium). In some embodiments, the rendering engine 520 can utilize GPU (graphics processing unit) acceleration logic. This GPU acceleration logic is provided, for example, through a Web GL (Web Graphics Library) program that executes a Graphics Library Shader Language (GLSL) program on the GPU.

[0064] IAP500 can be implemented as part of a network service, and a web-based application 80 can execute each process of IAP500 by communicating with one or more remote computers (e.g., a server for network services). The web-based application 80 obtains some or all of the program resources for implementing IAP500 from the network site. In providing IAP500, the web-based application 80 can also access various types of datasets. These datasets correspond to files and design libraries (e.g., pre-designed design elements), and these datasets may be held remotely (e.g., stored on a server or associated with an account) or locally. Furthermore, in some embodiments, a shared design library is provided by a network computer system 550, and the user computing device 10 can utilize this shared design library along with any application services provided via IAP500. This allows the user to initiate a session to implement IAP500 to create or edit workspace files rendered on the canvas 522, according to one of the IAP500's multiple collaboration application services.

[0065] In some embodiments, the IAP 500 comprises a program interface 502, an input interface 518, and a rendering engine 520. The program interface 502 may include one or more processes performed to access and obtain program resources from local and / or remote sources. In one embodiment, the program interface 502 can generate, for example, a canvas 522 using program resources associated with a web-based application 80 (e.g., an HTML5.0 canvas). In addition to or instead of this, the program interface 502 may generate the canvas 522 by triggering, etc., using program resources and datasets (e.g., canvas parameters) obtained from a local source (e.g., memory) or a remote source (e.g., a network service).

[0066] Furthermore, the program interface 502 can obtain program resources, including an application framework for use with the canvas 522. This application framework may include, for example, a dataset that defines or configures a set of interactive graphics tools that are integrated into the canvas 522 and constitute the input interface 518. This allows the user to provide input for generating or updating content to be rendered on the canvas 522.

[0067] According to some embodiments, the input interface 518 can be implemented as a functional layer integrated into the canvas 522, responsible for detecting and decoding user input. The input interface 518 can, for example, process interactions between the user and the input mechanisms of the user computing device (e.g., pointer device, keyboard), and can detect inputs such as cursor position / movement on the canvas 522, hover input (e.g., preliminary selection input), selection input (e.g., click or double-click), and shortcuts (e.g., keyboard input). When processing interactions between the user and the pointer device, the input interface 518 can identify the user's cursor position on the screen when the user performs an interaction with the pointer device, such as moving the pointer device, by utilizing reference information from the canvas 522. Furthermore, the input interface 518 can decode the user's input actions. Such decoding can be performed based on the location of the detected input (e.g., whether the input location indicates a tool selection, a selection of an object rendered on the canvas, or a selection of an area within the canvas), the frequency of the detected input over a predetermined period (e.g., double-click), and / or the start and end positions of the input or a series of inputs (e.g., the start and end positions of a click or drag), as well as various other input types that the user can specify via one or more input devices (e.g., right-click, screen tap, etc.). In some embodiments, for example, the input interface 518 may interpret a series of inputs as a selection of a design tool (e.g., a selection of a shape based on the input location) or as an input defining the attributes of the selected shape (e.g., dimensions). Also in some embodiments, the input interface 518 may interpret a series of inputs (inputs corresponding to a continuous movement of the user's pointer device) as an input for selecting a tool (e.g., a shape) and specifying the location on the canvas where the output of the selected tool should be displayed.

[0068] In some embodiments, the IAP500 includes application service logic 516. This application service logic 516 enables the use of multiple application services during a given user session, providing the user with features and user experiences specific to each application service. As described in some embodiments, each application service is implemented on the IAP500 using the corresponding application service logic 516. This constitutes components of the IAP500, such as interface components 518 and rendering engines 520, providing the features and user experiences of the corresponding application services. In this way, the IAP500 enables the operation of multiple application services during a single user online session. Furthermore, resources such as program resources of the IAP500, including the canvas 522, can be shared among different application services. This allows each application service to provide content to the canvas 522 and / or utilize the features and content provided to the canvas 522 during a given session. Application services can also be implemented as selectable modes of the IAP500. In this case, each mode provides specific features and user experiences, and the user can switch between these modes. In some embodiments, each application service can utilize a common workspace file associated with a user. By default, a computing device that opens this common workspace file can access and update it using the default application service. Alternatively, a user can access, use, and / or update the workspace file using different application services by switching the operating mode of the IAP500.

[0069] The network computing system 550 may include a site manager 558. The site manager 558 manages a website that provides a set of web resources 555 (e.g., web pages) for a web-based application 80 on a user computing device 10. The web resources 555 may include instructions ("IAP instructions 557") such as scripts or other logic that can be executed by the browser or web component of the user computing device. The web resources 555 may also include (i) inter-application service shared resources provided to user computing devices using any of the application services, and (ii) application-specific resources that run on the user computing device as resources dedicated to a particular one of the available application services. Furthermore, the web resources 555 may include a design library of design elements, some or all of which are shared among the application services. This design element library allows the user to select a predetermined design element to use on the canvas 522 when using any of the application services.

[0070] In some variations, when the computing device 10 accesses and downloads the web resource 555, the web-based application 80 executes an IAP instruction 557, thereby realizing the functions described above. For example, the execution of the IAP instruction 557 by the web-based application 80 can launch the program interface 502 on the user computing device 10. Alternatively, the launch of the program interface 502 may occur simultaneously with the establishment of a WebSocket connection between the program interface 502 and the service component 560 of the network computer system 550.

[0071] In some embodiments, the logic contained in the web resource 555 is configured such that when the web-based application 80 executes the logic and initiates one or more processes of the program interface 502, the IAP 500 acquires additional program resources and datasets necessary to implement the various functions exemplified. For example, the web resource 555 may be configured to embed logic (e.g., "JAVASCRIPT" code) including GPU acceleration logic within an HTML page downloaded by the user's computing device. Furthermore, when implementing any of the multiple application services of the IAP 500, each application service may be configured to trigger the program interface 502 to acquire additional program resources and datasets (e.g., from the network service 552 and / or local resources of the computing device 10). For example, some components of the IAP 500 may be implemented via a web page downloadable to the computing device 10 after authentication is complete, or in addition to or instead of this, after the user performs an additional action (e.g., downloading one or more pages of a workspace associated with an account identifier). Therefore, in the above example, the network computing system 550 can be configured to transmit an IAP instruction 557 to the computing device 10 by combining multiple network communications, including a download process by the web-based application 80, and the web-based application 80 can receive and execute this IAP instruction 557.

[0072] The computing device 10 can access the website of the network service 552 using a web-based application 80 and download web pages or web resources. When the web-based application 80 accesses the website of the network service 552, it can transmit an account identifier to the service component 560 automatically (for example, using stored authentication information) or by manual input. In some embodiments, the web-based application 80 can also transmit one or more additional identifiers correlated with the user identifier.

[0073] Furthermore, in some embodiments, the service component 560 may use the user identifier or account identifier to retrieve profile information 509 from the user profile store 566. Alternatively, the user's profile information 509 may be identified and stored locally on the user's computing device 10.

[0074] Furthermore, the service component 560 can retrieve files of the active workspace associated with a user account or identifier ("active workspace files" 563) from the file store 565. At this time, the workspace associated with the account and / or user can be identified by the profile store 566, and the datasets that make up the workspace can be stored in the file store 565. The datasets stored in the file store 565 may include, for example, each page of the workspace and one or more data structure representations 561 of the design being edited that can be rendered from each active workspace file.

[0075] In addition to or instead of this, each file may be associated with metadata that identifies the application service used to create it. In some embodiments, this metadata identifies the default application service for viewing, using, or updating the file.

[0076] Furthermore, in some embodiments, the service component 560 provides the web-based application 80 with a workspace representation 559 representing a workspace associated with a user. The workspace representation 559 identifies, for example, individual files associated with the user and / or user account. The workspace representation 559 can also identify a set of files consisting of multiple files. Each file contains one or more pages, and each page contains multiple objects that constitute a design interface.

[0077] Meanwhile, on the user device 10, the user can view workspace representations via the web-based application 80 and choose to open workspace files via the web-based application 80. In some embodiments, when the user chooses to open one of the active workspace files 563, the web-based application 80 launches a canvas 522. For example, the IAP 500 can launch an HTML5.0 canvas as a component of the web-based application 80, and the rendering engine 520 can access one or more data structure representations 511 of the content rendered on the canvas 522.

[0078] The IAP500 implements multiple operating modes corresponding to each application service using the application service logic 516. As described above, the application service logic 516 associated with each application service can include instructions and data for configuring each component that makes up the IAP500, so that the functions and characteristics of that application service are incorporated into those components. Therefore, the application service logic 516 can configure the application framework and / or input interface 518 such that, for example, the appearance, functions, and configuration of the IAP500 change depending on the operating mode of the IAP500. The types of actions and interactions that a user can perform to register user input can also be configured differently depending on the operating mode. Furthermore, the input functions or user interface functions that a user can select or use as functions provided on the canvas 522 can also be configured differently depending on the operating mode. For example, when IAP500 is operating in whiteboard application service mode, the program interface 502 provides an input function that allows the user to select design elements in the form of "sticky notes." However, when IAP500 is switched to interactive graphic design application service mode, this "sticky note function" can be disabled. Instead, in this switched mode, the user can select from a number of shape options or previously designed objects, write a text message within it, and display it on the canvas 522.

[0079] Furthermore, the application service logic 516 can configure the operation of the rendering engine 520 so that its functions and behavior change in response to changes in the application service being used. In this way, the rendering engine 520 functions to switch its behavior depending on which of the different operating modes corresponds to the specific application service that is currently active. For example, by changing the settings of the rendering engine 520, it is possible to change the appearance of the canvas 522, the appearance of content elements rendered on the canvas 522 (e.g., visual attributes), the behavior or representation of user interactions (e.g., whether the user's cursor or pointer device is displayed on the canvas 522), the type or details of the rendered content, the physics engine used by the rendering engine 520 when representing dynamic events (e.g., the process of moving an object), and what user operations are possible (e.g., whether the size of a selected object can be changed).

[0080] Furthermore, each application service can utilize a shared library and core functions. The shared library contains multiple content elements (e.g., graphic design elements), and the core functions enable the sharing and updating of design elements among various application services available via IAP500. Additionally, a workspace file created and edited using one application service can be used in another application service. Moreover, switching between application services can be seamless. For example, user computing device 10 can open a workspace file using a first application service (e.g., an interactive graphic design application service for UIX design), and then seamlessly switch to a second application service (e.g., a whiteboard application service) without closing the workspace file, while continuing to use the same file. In some embodiments, each application service is configured to allow the user to update the workspace file even when other computing devices are simultaneously using the same workspace file (e.g., in a collaborative environment). In some embodiments, the user can switch between application services by switching the IAP500 mode. Even after switching modes, multiple application services can be configured to use the same workspace file.

[0081] Example of item division The various aspects of this disclosure are described below in numbered sections.

[0082] Example of item division The various aspects of this disclosure are described below in numbered sections.

[0083] 1. In some embodiments, a method implemented on a processor, comprising: detecting that a code-defined visual asset loaded onto a canvas in a design environment has been modified; copying the code-defined visual asset to an invisible rendering environment in the design environment; rendering the code-defined visual asset in the invisible rendering environment based on the execution of code associated with the code-defined visual asset in the invisible rendering environment; obtaining an image of the rendered code-defined visual asset; and displaying the obtained image of the rendered code-defined visual asset in the design environment.

[0084] 2. The method according to paragraph 1, wherein the step of detecting that a code-defined visual asset has been modified includes the step of detecting that the amount of change exceeds a threshold.

[0085] 3. The method described in paragraph 1 or 2, wherein the rendered image of the code-defined visual asset is obtained when rendering of the code-defined visual asset in an invisible rendering environment begins.

[0086] 4. The method according to any one of paragraphs 1 through 3, wherein the rendered image of the code-defined visual asset is obtained based on the elapsed time since rendering of the code-defined visual asset in an invisible rendering environment was initiated.

[0087] 5. The method according to any one of paragraphs 1 to 4, wherein the step of obtaining an image of a rendered code-defined visual asset includes the steps of identifying one or more visual components of a code-defined visual asset on which an external resource is loaded, generating placeholder visual content for the identified one or more visual components, and replacing the identified one or more visual components of the image of the rendered code-defined visual asset with the placeholder visual content.

[0088] 6. The method described in Section 5, wherein the process of generating placeholder visual content includes the process of obtaining a still image from an external resource.

[0089] 7. The method described in Section 6, wherein external resources include video, and still images include keyframes selected from the video.

[0090] 8. The process of obtaining an image of a rendered code-defined visual asset includes the steps of determining the size of the code-defined visual asset, generating a bounding box based on the size and bleed amount of the code-defined visual asset, and obtaining the content within the bounding box. The method according to any one of claims 1 to 7, wherein the bounding box is superimposed on a code-defined visual asset on the invisible rendering environment of the design environment.

[0091] 9. The method according to any one of claims 1 to 8, wherein the step of obtaining an image of a rendered code-defined visual asset includes the steps of generating a bounding box based on the size of the code-defined visual asset and the position of each visual component of the code-defined visual asset, and obtaining the content within the bounding box, the bounding box being superimposed on the code-defined visual asset in an invisible rendering environment of the design environment.

[0092] 10. The method according to any one of claims 1 to 9, further comprising the step of replacing a previously generated image of a code-defined visual asset with a retrieved image on the canvas of the design environment without executing any code associated with the code-defined visual asset in the design environment.

[0093] 11. The method according to any one of claims 1 to 10, wherein the resolution of the acquired image is higher than the resolution at which the code-defined visual asset is rendered on the screen displaying the design environment.

[0094] 12. A processing system comprising at least one memory in which executable instructions are stored, and one or more processors configured to execute executable instructions and cause the processing system to perform the method described in any one of paragraphs 1 to 11.

[0095] 13. A processing system comprising means for performing the method described in any one of paragraphs 1 to 11.

[0096] 14. A non-temporary computer-readable medium containing executable instructions that, when processed by one or more processors, cause one or more processors to perform the method described in any one of paragraphs 1 through 11.

[0097] Any component described in any claim of the patent claims and / or any combination of any element described in this specification, regardless of the form of the combination, shall be included in the scope intended and protected by the present invention.

[0098] While various embodiments are described herein, these are presented for illustrative purposes only and are not intended to exhaust or limit all embodiments of the disclosure. Those skilled in the art will see that many modifications and variations are possible without departing from the scope and spirit of the embodiments described herein.

[0099] Aspects of this embodiment can be embodied as a system, method, or computer program product. Accordingly, embodiments of this disclosure can take the form of hardware-only embodiments, software-only embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, all of which may be collectively referred to as “modules,” “systems,” or “computers” in this specification. Furthermore, any hardware and / or software technology, process, function, component, engine, module, or system described herein can be implemented as a single circuit or a set of circuits. In addition, embodiments of this disclosure can take the form of a computer program product embodied as computer-readable program code recorded on one or more computer-readable media.

[0100] One or more computer-readable media can be used in any combination. A computer-readable media may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or any suitable combination thereof. More specific examples (but not exhaustive) of computer-readable storage media include electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this specification, a computer-readable storage medium may be any tangible medium capable of storing (storing) programs for use by, or in association with, a system, apparatus, or device that executes instructions.

[0101] The embodiments of this disclosure have been described above with reference to flowcharts and / or block diagrams illustrating methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a programmable data processing device, such as a general-purpose computer or a dedicated computer, to generate a machine. The instructions, when executed by the processor of the computer or programmable data processing device, realize the implementation of a function or operation defined in one or more blocks of the flowcharts and / or block diagrams. Such processors may include, but are not limited to, general-purpose processors, dedicated processors, application-specific processors, or field-programmable gate arrays.

[0102] The flowcharts and block diagrams shown in the drawings illustrate architectures, functions, and operations that may constitute embodiments of systems, methods, and computer program products according to various embodiments of this disclosure. In this sense, each block in a flowchart or block diagram can be considered to represent a module, segment, or portion of code containing one or more executable instructions for implementing the one or more logical functions defined in that block. It should also be noted that in some alternative embodiments, the functions shown in the blocks may be executed in an order different from that shown in the drawings. For example, two blocks shown as consecutive blocks may actually be executed substantially simultaneously, or in some cases, these blocks may be executed in reverse order depending on the related functions. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, may also be implemented by a dedicated hardware-based system or a combination of dedicated hardware and computer instructions that performs the functions or operations defined in that block.

[0103] While the above description is directed towards embodiments of the present disclosure, it is possible to devise other embodiments and further embodiments of the present disclosure, provided that they do not deviate from the basic scope of the present disclosure, and such scope is defined by the claims set forth below. [Explanation of Symbols]

[0104] 102 processors 116 memory 122 Snapshot Engine 123 Watchdog 124 Snapshot generation unit 126 Graphical Design Engines

Claims

1. In the way implemented by the processor, The process of detecting when a code-defined visual asset loaded onto the canvas of the design environment has been modified, The steps include copying the code-defined visual asset to the invisible rendering environment of the design environment, The process of rendering the code-defined visual asset in the invisible rendering environment based on the execution of code associated with the code-defined visual asset in the invisible rendering environment, The process of obtaining the image of the rendered code-defined visual asset, The steps include displaying the acquired image of the rendered code-defined visual asset in the design environment. A method that includes this.

2. The method according to claim 1, wherein the step of detecting that the code-defined visual asset has been modified includes the step of detecting that the amount of change exceeds a threshold.

3. The method according to claim 1, wherein the image of the rendered code-defined visual asset is obtained when rendering of the code-defined visual asset in the invisible rendering environment begins.

4. The method according to claim 1, wherein the image of the rendered code-defined visual asset is obtained based on the elapsed time since rendering of the code-defined visual asset in the invisible rendering environment was initiated.

5. The step of obtaining the image of the rendered code-defined visual asset is: The steps include identifying one or more visual components of the code-defined visual asset on which external resources are loaded, A step of generating placeholder visual content for one or more of the identified visual components, The steps include replacing one or more identified visual components of the rendered code-defined visual asset image with the placeholder visual content. The method according to claim 1, comprising:

6. The method according to claim 5, wherein the step of generating the placeholder visual content includes the step of acquiring a still image from the external resource.

7. The above external resources include videos, The method according to claim 6, wherein the still image includes keyframes selected from the video.

8. The step of obtaining the image of the rendered code-defined visual asset is: The process of determining the size of the aforementioned code-defined visual asset, A process of generating a bounding box based on the size and bleed amount of the code-defined visual asset, The process of obtaining the content within the bounding box and Includes, The method according to claim 1, wherein the bounding box is superimposed on the code-defined visual asset on the invisible rendering environment of the design environment.

9. The step of obtaining the image of the rendered code-defined visual asset is: A process of generating a bounding box based on the size of the code-defined visual asset and the position of each visual component of the code-defined visual asset, The process of obtaining the content within the bounding box and Includes, The method according to claim 1, wherein the bounding box is superimposed on the code-defined visual asset on the invisible rendering environment of the design environment.

10. The process of replacing previously generated images of the code-defined visual asset with the acquired images on the canvas of the design environment without executing the code associated with the code-defined visual asset in the design environment, The method according to claim 1, further comprising:

11. The method according to claim 1, wherein the resolution of the acquired image is higher than the resolution at which the code-defined visual asset is rendered on the screen displaying the design environment.

12. In a processing system, At least one memory containing executable instructions, One or more processors configured to execute the aforementioned executable instructions Includes, The instruction is given to the processing system: To detect when a code-defined visual asset loaded onto the design environment canvas has been modified, The code-defined visual asset is copied to the invisible rendering environment of the design environment. The code-defined visual asset is rendered in the invisible rendering environment based on the execution of the code associated with the code-defined visual asset in the invisible rendering environment. The rendered image of the code-defined visual asset is retrieved, A processing system that displays the acquired image of the rendered code-defined visual asset in the design environment.

13. The processing system according to claim 12, wherein, in order to detect that the code-defined visual asset has been modified, one or more processors are configured to cause the processing system to detect that the amount of change of the modification exceeds a threshold.

14. The processing system according to claim 12, wherein the image of the rendered code-defined visual asset is obtained when rendering of the code-defined visual asset is started in the invisible rendering environment.

15. The processing system according to claim 12, wherein the image of the rendered code-defined visual asset is obtained based on the fact that a period of time has elapsed since rendering of the code-defined visual asset in the invisible rendering environment was started.

16. To obtain the image of the rendered code-defined visual asset, one or more processors provide the processing system with: Identify one or more visual components of the code-defined visual asset on which external resources are loaded, For one or more of the identified visual components, generate placeholder visual content. The processing system according to claim 12, configured to replace one or more identified visual components of the image of the rendered code-defined visual asset with the placeholder visual content.

17. To obtain the image of the rendered code-defined visual asset, one or more processors provide the processing system with: Determine the size of the aforementioned code-defined visual asset, A bounding box is generated based on the size and bleed amount of the code-defined visual asset. It is configured to retrieve the content within the bounding box, The processing system according to claim 12, wherein the bounding box is superimposed on the code-defined visual asset in the invisible rendering environment of the design environment.

18. To obtain the image of the rendered code-defined visual asset, one or more processors provide the processing system with: A bounding box is generated based on the size of the code-defined visual asset and the position of each visual component of the code-defined visual asset. It is configured to retrieve the content within the bounding box, The processing system according to claim 12, wherein the bounding box is superimposed on the code-defined visual asset in the invisible rendering environment of the design environment.

19. The one or more processors further include in the processing system: The processing system according to claim 12, configured to replace previously generated images of the code-defined visual asset with the acquired images on the canvas of the design environment without executing any code associated with the code-defined visual asset in the design environment.

20. In one or more non-temporary computer-readable media storing instructions, When the aforementioned instruction is executed by one or more processors, the one or more processors shall The process of detecting when a code-defined visual asset loaded onto the canvas of the design environment has been modified, The steps include copying the code-defined visual asset to the invisible rendering environment of the design environment, The process of rendering the code-defined visual asset in the invisible rendering environment based on executing the code associated with the code-defined visual asset in the invisible rendering environment, The steps include obtaining the rendered image of the code-defined visual asset, The steps include displaying the acquired image of the rendered code-defined visual asset in the design environment. A non-temporary computer-readable medium that enables the following action.

Images