Systems and methods for generating simulations using section groupings

Abstract

The computer system allows a user to specify multiple cards comprising a design interface or presentation, each card being renderable in a simulation or production environment separately from other cards of the multiple cards. The user can specify one or more sections or groupings of cards, each section or grouping including various cards. The user can specify multiple flow connections, including at least a first flow connection, from one of the multiple cards to a first section, where the card of the first flow connection is not part of the first section. During simulation rendering of the design interface, the individual cards are rendered in a sequence based at least in part on one or more of the flow connections, including at least the first flow connection.

Description

Related Applications

[0001] This application claims the benefit of priority to (i) U.S. Patent Application No. 18 / 382,999, filed October 23, 2023, and (ii) U.S. Provisional Patent Application No. 63 / 418,953, filed October 24, 2022, the entire contents of each of which are incorporated herein by reference. [Technical Field]

[0002] The examples described herein relate to interactive graphic design systems, and more particularly to systems and methods for generating simulations using section groupings. [Background technology]

[0003] Software design tools come in many forms and uses. For example, in the field of application user interfaces, software design tools require designers to blend the functional aspects of a program with aesthetic and even legal requirements, resulting in a collection of pages that form the application's user interface. For a given application, designers often have many objectives and requirements that are difficult to keep track of. To assist designers in their work, some design tools allow for production environment simulations of cards (or arrangements of other design elements). For example, a production environment simulation can be implemented by rendering a sequence of cards to reflect state changes that may occur in the production environment. The use of such simulations allows designers to see how their design interfaces will be implemented in the production environment, enabling them to develop design interfaces with the production environment in mind. [Brief explanation of the drawings]

[0004] [Figure 1A]FIG. 1 illustrates an interactive graphic design system for a user's computing device, according to one or more examples. [Figure 1B] FIG. 1 illustrates a network computing system for implementing an interactive graphic design system on a user computing device, according to one or more examples. [Figure 1C] FIG. 1 illustrates a network computing system for implementing an interactive graphic design system for multiple users in a collaborative network platform, according to one or more examples. [Figure 2] FIG. 1 illustrates components of a simulation engine for utilizing section grouping, according to one or more embodiments. [Figure 3A] FIG. 1 illustrates an exemplary method for implementing logic for providing section groupings (or sections) in an integrated graphic design system (IGDS), according to one or more embodiments. [Figure 3B] FIG. 1 illustrates an exemplary method for generating a production environment rendering for a simulation in which conditional sequencing of individual cards is based at least in part on state information associated with sections, according to one or more embodiments. [Figure 4A] FIG. 10 illustrates a design interface in which a collection of cards is sectioned, according to one or more embodiments. [Figure 4B] FIG. 10 illustrates a sequence of renderings generated in a simulated environment for a collection of sectioned cards, in accordance with one or more embodiments. [Figure 4C] FIG. 10 illustrates a sequence of renderings generated in a simulated environment for a collection of sectioned cards, in accordance with one or more embodiments. [Figure 4D]FIG. 10 illustrates a sequence of renderings generated in a simulated environment for a collection of sectioned cards, in accordance with one or more embodiments. [Figure 4E] FIG. 10 illustrates a sequence of renderings generated in a simulated environment for a collection of sectioned cards, in accordance with one or more embodiments. [Figure 4F] FIG. 10 illustrates a sequence of renderings generated in a simulated environment for a collection of sectioned cards, in accordance with one or more embodiments. [Figure 5] FIG. 1 illustrates a networked computer system in which one or more embodiments may be implemented. [Figure 6] FIG. 1 illustrates a user computing device for use with one or more examples as described. DETAILED DESCRIPTION OF THE INVENTION

[0005] In embodiments, an integrated graphic design system (IGDS) allows users to create sections, which are logical elements that represent groupings of various cards. A card may correspond to a frame containing design elements that can be rendered during production to display a screen, presentation (e.g., a slide), or page. In embodiments, a section may be associated with cards of a design to be rendered in a simulation environment and designated as a target for a flow connection. Sections may also be associated with state information that can identify, for example, which cards in each section were most recently rendered. The IGDS can use the sections and their associated state information to determine the sequence in which cards in a design or presentation are rendered.

[0006] Additionally, embodiments provide a networked computer system that enables one or more users to create cards for a design interface or presentation, each of a plurality of cards being renderable separately from other cards of the plurality of cards in a simulation or production environment. The networked computer system enables the user to specify one or more sections of cards (alternatively referred to as section groupings) from a plurality of groupings, each section including a variety of cards. The user can further specify a plurality of flow connections, including at least a first flow connection, from one of the plurality of cards to a first section of the one or more sections. During simulation rendering of the design interface or presentation, the system renders the cards of the plurality of cards in a sequence based at least in part on the one or more flow connections, including at least the first flow connection.

[0007] In an example, a flow connection that specifies a section (or section grouping) as a target can cause a computer system to select which cards of that section to render at a particular moment during a simulation. The computer system can select which cards of a section to render for a simulation based on state information recorded during the simulation for the section. In an example, the state information can identify the most recently rendered card. Thus, in an example, when a card of a section is rendered during a simulation rendering, the card can reflect or correspond to the card of the section that was most recently rendered during the simulation.

[0008] One or more embodiments described herein provide that the methods, techniques, and actions performed by a computing device are performed programmatically or as computer-implemented methods. Programmatically, as used herein, means through the use of code or computer-executable instructions. These instructions may be stored in one or more memory resources of the computing device. Programmatically performed steps may or may not be automatic.

[0009] One or more embodiments described herein can be implemented using program modules, engines, or components. A program module, engine, or component can include a program, a subroutine, a portion of a program, or a software or hardware component that can perform one or more specified tasks or functions. As used herein, a module or component can exist on a hardware component independent of other modules or components. Alternatively, a module or component can be a shared element or process of other modules, programs, or machines.

[0010] Some embodiments described herein may generally require the use of computing devices that include processing and memory resources. For example, one or more embodiments described herein may be implemented, in whole or in part, on computing devices such as servers, desktop computers, mobile phones or smartphones, tablets, wearable electronic devices, laptop computers, printers, digital photo frames, network equipment (e.g., routers), and tablet devices. Memory, processing, and network resources may all be used in connection with establishing, using, or executing any embodiment described herein (including performing any method or implementing any system).

[0011] Furthermore, one or more embodiments described herein may be implemented through the use of instructions executable by one or more processors. These instructions may be stored on a computer-readable medium. The machines illustrated or described using the following figures provide examples of processing resources and computer-readable media capable of carrying and / or executing instructions for implementing embodiments of the present invention. In particular, many machines illustrated with embodiments of the present invention include a processor and various forms of memory for storing data and instructions. Examples of computer-readable media include permanent memory storage devices such as hard drives on personal computers or servers. Other examples of computer storage media include portable storage units such as CD or DVD units, flash memory (such as found in smartphones, multifunction devices, or tablets), and magnetic memory. Computers, terminals, and network-enabled devices (e.g., mobile devices such as mobile phones) are all examples of machines and devices that utilize processors, memory, and instructions stored on a computer-readable medium. Furthermore, embodiments may be implemented in the form of a computer program or a computer-usable carrier medium capable of carrying such a program.

[0012] System Description 1A illustrates an interactive graphic design system for a user's computing device, according to one or more examples. The interactive graphic design system ("IGDS") 100 can be implemented in any one of several different computing environments. For example, in some variations, the IGDS 100 can be implemented as a client-side application running on the user computing device 10 to provide functionality as described in the various examples. In other examples, as described below, the IGDS 100 can be implemented through the use of a web-based application 80. Additionally or alternatively, the IGDS 100 can be implemented as a distributed system, such that the processes described in the various examples run on network computers (e.g., servers) and on the user device 10.

[0013] By way of example, the IGDS 100 may be implemented on a user computing device 10 to enable a corresponding user to design various types of interfaces using graphical elements. The IGDS 100 may include processes executed as or through a Web-based application 80 installed on the computing device 10. As illustrated by various examples, the Web-based application 80 may execute scripts, code, and / or other logic (“program components”) to implement the functionality of the IGDS 100. Furthermore, in some variations, the IGDS 100 may be implemented as part of a network service, and the Web-based application 80 may communicate with one or more remote computers (e.g., servers used for the network service) to execute the processes of the IGDS 100.

[0014] In some examples, the web-based application 80 obtains some or all of the program resources for implementing the IGDS 100 from a network site. Additionally or alternatively, the web-based application 80 can obtain some or all of the program resources from a local source (e.g., local memory resident on the computing device 10). The web-based application 80 can also access various types of datasets in providing the IGDS 100. The datasets can correspond to files and libraries, which can be stored remotely (e.g., on a server, associated with an account) or locally.

[0015] In an example, the web-based application 80 may support commercially available browsers such as GOOGLE CHROME (developed by GOOGLE, INC.), SAFARI (developed by APPLE, INC.), and INTERNET EXPLORER (developed by MICROSOFT CORPORATION). In such an example, the IGDS 100 processes may be implemented as scripts and / or other embedded code that the web-based application 80 downloads from a network site. For example, the web-based application 80 may execute code embedded within a web page to implement the IGDS 100 processes. The web-based application 80 may also execute scripts to retrieve other scripts and program resources (e.g., libraries) from the network site and / or other local or remote locations. As an example, the web-based application 80 may execute JAVASCRIPT® embedded in HTML resources (e.g., web pages structured according to HTML 5.0 or other versions provided under standards published by the W3C or WHATWG consortium). In some examples, the rendering engine 120 and / or other components may utilize graphics processing unit (GPU) acceleration logic, such as that provided through WebGL (Web Graphics Library) programs that execute Graphics Library Shader Language (GLSL) programs that run on the GPU.

[0016] According to an example, a user of computing device 10 operates web-based application 80 to access network sites to obtain and execute program resources to implement IGDS 100. In this manner, a user may initiate a session to implement IGDS 100 for purposes of creating and / or editing a design interface. In an example, IGDS 100 includes program interface 102, input interface 118, and rendering engine 120. Program interface 102 may include one or more processes executed to access and obtain program resources from local and / or remote sources.

[0017] In one implementation, the programmatic interface 102 can use programmatic resources associated with the web-based application 80 (e.g., an HTML 5.0 canvas), for example, to generate the canvas 122. Additionally or alternatively, the programmatic interface 102 can trigger or otherwise generate the canvas 122 using programmatic resources and data sets (e.g., canvas parameters) obtained from a local source (e.g., memory) or a remote source (e.g., a network service).

[0018] The programmatic interface 102 may also obtain programmatic resources including an application framework for use with the canvas 122. The application framework may include, for example, a data set that defines or configures a set of interactive graphic tools that are integrated with the canvas 122 and that configure the input interface 118, allowing a user to provide input for creating and / or editing the design interface.

[0019] According to some examples, the input interface 118 can be integrated with the canvas 122 and implemented as a functional layer that detects and interprets user input. The input interface 118 can, for example, use a reference to the canvas 122 to identify the on-screen location of a user input (e.g., a “click”). Furthermore, the input interface 118 can interpret a user's input action based on the location of the detected input (e.g., whether the location of the input indicates a tool selection, an object rendered on the canvas, or an area of ​​the canvas), the frequency of the detected input in a given period of time (e.g., a double-click), and / or the start and end positions of an input or series of inputs (e.g., the start and end positions of a click and drag), as well as various other input types (e.g., a right-click, a screen tap, etc.) that a user can specify through one or more input devices. In this way, the input interface 118 can, for example, interpret a series of inputs as a design tool selection (e.g., a shape selection based on the input location) or as input to define attributes (e.g., dimensions) of the selected shape.

[0020] Additionally, the programmatic interface 102 can be used to retrieve program resources and data sets, including files 101, that comprise a user's active workspace from local or remote sources. In an example, the files 101 can include a collection of cards, where the cards in the collection provide design elements for a user interface or presentation when rendered in a production environment. In an example, individual cards can represent, for example, an application screen or an application state. When rendered in production or through a simulation, the cards can be rendered sequentially or consecutively, with one card replacing another. The retrieved data sets can include one or more cards that include design elements that collectively form a design interface, or a design interface in progress. Each file 101 can include one or more data structure representations 111 (denoted "DSR 111") that collectively define the design interface. As described in more detail in some examples, the data structure representations 111 can be in the form of a Document Object Model (DOM). The files 101 can also include additional data sets associated with the active workspace. For example, as will be described in some examples, a workspace file can store animation data sets that define animation behavior, such as between objects or states, in a rendering of canvas 122.

[0021] In the example, the rendering engine 120 uses the DOM representation 111 to render a corresponding design 125 (or presentation) on the canvas 122, where the design reflects the graphic elements and their respective attributes provided with the individual pages of the file 101. A user can edit the design using the input interface 118. Alternatively, the rendering engine 120 can generate a blank page of the canvas 122, and a user can generate a design using the input interface 118. When rendered, the design can include graphic elements, such as a background and / or a set of objects (e.g., shapes, text, images, program elements), as well as attributes for the individual graphic elements. Each attribute of a graphic element can include an attribute type and an attribute value. For objects, the attribute type includes shape, dimension (or size), layer, type, color, line weight, text size, text color, font, and / or other visual characteristics. Depending on the implementation, the attributes reflect properties of the two-dimensional or three-dimensional design. In this way, the attribute values ​​of individual objects can define, for example, the visual characteristics of the size, color, position, layering, and content of elements rendered as part of the design.

[0022] Network Computing System Implementing IGDS 1B illustrates a network computing system for implementing an interactive graphic design system on a user computing device, according to one or more examples. A network computing system such as that illustrated in the example of FIG. 1B can be implemented using one or more servers that communicate with the user computing device over one or more networks.

[0023] 1B example, network computing system 150 performs operations to enable IGDS 100 to be implemented on user computing device 10. In a variation, network computing system 150 provides network services 152 to support use of IGDS 100 by user computing devices utilizing a browser or other web-based application. Network computing system 150 may include a site manager 158 that manages a website where a set of web resources 155 (e.g., web pages) are made available for site visitors. Web resources 155 may include instructions such as scripts or other logic (“IGDS instructions 157”) that are executable by a browser or web component of the user computing device.

[0024] In some variations, when computing device 10 accesses and downloads web resource 155, web-based application 80 executes IGDS instructions 157 to implement functionality as described in some examples of FIG. 1A. For example, IGDS instructions 157 may be executed by web-based application 80 to launch program interface 102 on user computing device 10. The launch of program interface 102 may occur simultaneously with the establishment of a web socket connection between program interface 102 and service component 160 of network computing system 150, for example.

[0025] In some examples, web resources 155 include logic that web-based application 80 executes to launch one or more processes of program interface 102, thereby causing IGDS 100 to obtain additional program resources and data sets for implementing functionality as described by the examples. Web resources 155 may, for example, embed logic (e.g., JAVASCRIPT code), including GPU acceleration logic, in an HTML page for download by a user's computing device. Program interface 102 may be triggered to obtain additional program resources and data sets, for example, from network services 152 and / or from local resources of computing device 10, to implement IGDS 100. For example, some of the components of IGDS 100 may be implemented through web pages that can be downloaded to computing device 10 after authentication is performed and / or when a user performs additional actions (e.g., downloading one or more pages of a workspace associated with an account identifier). Thus, in the illustrated example, the network computing system 150 can communicate the IGDS instructions 157 to the computing device 10 through a combination of network communications, including through an activity download of the web-based application 80, where the IGDS instructions 157 are received and executed by the web-based application 80.

[0026] Computing device 10 may use web-based application 80 to access a website of network service 152 and download web pages or web resources. Upon accessing the website, web-based application 80 may communicate an account identifier to service component 160 automatically (e.g., through stored credentials) or through manual entry. In some examples, web-based application 80 may also communicate one or more additional identifiers that correlate with the user identifier.

[0027] Additionally, in some examples, the service component 160 may retrieve profile information from a user profile store using a user or account identifier of the user identifier. Additionally or alternatively, the user's profile information may be determined and stored locally on the user's computing device 10.

[0028] The service component 160 can also retrieve from the file store 164 a file of an active workspace linked to a user account or identifier ("active workspace file 163"). The profile store can also identify a workspace identified with the account and / or user, and the file store 164 can store the datasets that make up the workspace. The datasets stored in the file store 164 can include, for example, the pages of the workspace, a dataset identifying constraints for the active set of workspace files, and one or more data structure representations 161 of the design being edited that can be rendered from each active workspace file.

[0029] Further, in the example, the service component 160 provides the web-based application 80 with a representation 159 of a workspace associated with the user, which representation identifies, for example, individual files associated with the user and / or user account. The workspace representation 159 can also identify a set of files, each file containing one or more pages, and each page containing objects that are part of a design interface.

[0030] On user device 10, a user can view the workspace representation through a web-based application 80, and the user can select to open a file of the workspace through web-based application 80. In an example, when a user selects to open one of the active workspace files 163, web-based application 80 launches a canvas 122. For example, IGDS 100 can launch an HTML 5.0 canvas as a component of web-based application 80, and rendering engine 120 can access one or more data structure representations 111 of the design interface being edited and render the corresponding design on canvas 122.

[0031] The service component 160 can also determine a user's permission setting or role associated with the account identifier based on the user credentials. The user's permission setting or role can determine, for example, the files the user can access. In some examples, the implementation of the rendering engine 120 on the computing device 10 can be configured based at least in part on the user's role or setting. For example, a user's ability to specify design constraints can be determined by the user's permission setting, and a user can be authorized or revoked from creating design constraints 145 based on their permission setting. Furthermore, in some variations, the response actions a user can take to resolve a conflict can be limited by the user's permission setting. For example, a user's ability to override constraints 145 can be based on the user's permission setting.

[0032] In examples, changes implemented by the rendering engine 120 to the design may also be recorded with the respective DOM representation 111 for storage on the computing device 10. The program interface 102 may repeatedly or continuously stream change data 121 to the service component 160, with updates reflecting edits made to the design 125. The service component 160 may receive the change data 121 and then use this change data 121 to implement changes in the network-side data structure representation 161. In this manner, the network-side data structure representation 161 in the active workspace file 163 may mirror (or synchronize) the local DOM representation 111 on the user computing device 10. As the rendering engine 120 implements changes to the design on the user device 10, the changes may be recorded or otherwise implemented with the local DOM representation 111, and the program interface 102 may stream the changes as change data 121 to the service component 160 to synchronize the local representation 111 and the network-side representation 161 of the design. This process can be performed repeatedly or continuously so that the local representation 111 and the network-side representation 161 of the design remain synchronized.

[0033] Collaborative Network Platform 1C illustrates a network computing system for implementing an interactive graphic design system for multiple users in a collaborative network platform, according to one or more examples. In the example of FIG. 1C, the collaborative network platform is implemented by a network computing system 150 that communicates with multiple user computing devices 10, 12 over one or more networks (e.g., the World Wide Web) and implements an IGDS 100 on each computing device. While FIG. 1C illustrates an example in which two users utilize the collaborative network platform, in the illustrated example, the network computing system 150 enables collaboration on design interfaces among a larger group of users.

[0034] 1C, the user computing devices 10, 12 can be assumed to be operated by multiple users associated with a common account, each implementing a corresponding IGDS 100 to access the same workspace during their respective overlapping sessions. Thus, each of the user computing devices 10, 12 can simultaneously access the same set of active workspace files 163, and the respective programmatic interfaces 102 of the IGDSs 100 on each user computing device 10, 12 operate to establish corresponding communication channels (e.g., web socket connections) with the service components 160.

[0035] In an example, the service component 160 can communicate a copy of the active workspace file 163 to each user computing device 10, 12 such that the computing devices 10, 12 simultaneously render the designs in the active workspace file 163. Additionally, each of the computing devices 10, 12 can maintain a local DOM representation 111 of the respective designs as determined from the active workspace file 163. The service component 160 can also maintain a network-side data structure representation 161 obtained from the file of the active workspace 163 and consistent with the local DOM representation 111 of each of the computing devices 10, 12.

[0036] The network computing system 150 can continuously synchronize the active workspace files 163 on each user computing device. In particular, changes made by a user to a design on one computing device 10, 12 can be immediately reflected in the rendered design on the other user computing device 10, 12. As an example, a user at a computing device 10 can make changes to their respective designs, and their respective rendering engines 120 can implement updates that are reflected in their local copies of the DOM representation 111. From the computing device 10, the programmatic interface 102 of the IGDS 100 can stream change data 121 reflecting the user-input changes to the service component 160. The service component 160 processes the change data 121 on the user computing device. The service component 160 can use the change data 121 to make corresponding changes to the network-side data structure representation 161. The service component 160 can also stream remotely generated change data 171 (which, in the provided example, corresponds to or reflects change data 121 received from the user device 10) to the computing device 12 to cause the corresponding IGDS 100 to update the design rendered on that device. The computing device 12 can also use the remotely generated change data 171 to update its local DOM representation 111. The program interface 102 of the computing device 12 can receive updates from the network computing system 150, and the rendering engine 120 can update the design and its respective local DOM representation 111 of the computing device 12.

[0037] The reverse process can also be implemented to update the data structure representation 161 of the network computing system 150 using change data 121 communicated from the second computing device 12 (e.g., corresponding to a user of the second computing device updating a design rendered on the second computing device 12). The network computing system 150 can then stream remotely generated change data 171 (which, in the provided example, corresponds to or reflects the change data 121 received from the user device 12) to update the local DOM representation 111 of the design on the first computing device 10. In this manner, the design of the first computing device 10 can be updated in response to a user of the second computing device 12 providing user input to modify the design.

[0038] To facilitate synchronization of the DOM representations 111, 111 on the computing devices 10, 12, the network computing system 150 may implement stream connectors that merge data streams exchanged between the first computing device 10 and the network computing system 150 and between the second computing device 12 and the network computing system 150. In some implementations, the stream connectors may be implemented to allow each computing device 10, 12 to make changes to the network-side data representation 161 without the additional data duplication that may be required to process the streams from each device individually.

[0039] Additionally, over time, one or both of the computing devices 10, 12 may become out of sync with the server-side data representation 161. In such an event, each computing device 10, 12 may re-download the active workspace file 163 to resume maintenance of the data structure representation of the designs rendered and edited on that device.

[0040] 1A-1C, by way of example, the IGDS 100 may implement a user-facing simulation engine 200. The IGDS 100 may implement alternative modes, including a design mode and a simulation mode. In the simulation mode, the simulation engine 200 generates simulated renderings of individual cards in a collection. The simulation engine 200 may render a sequence of cards to provide the user with a production environment simulation of an ongoing or editing design interface or presentation. In an example, the simulation engine 200 may be implemented as part of the rendering engine 120. In a variation, the simulation engine 200 may be implemented through a separate component.

[0041] In design mode, the IGDS 100 may include section logic 129, which allows a user to specify one or more sections for each design 125. Each section may identify a set of cards. Once a section is created, the DOM representation 111 of the design 125 may include an additional root node representing the section, and nodes representing individual cards selected for the section may be subnodes of the root node. As described, design sectioning may include additional logic implemented specifically or automatically for the section. For example, a design element similarity search may be performed to determine other design elements in the section that are similar to the selected design element. Additionally, a user may provide additional input for creating or incorporating additional design elements based on such section-level similarity searches.

[0042] Users can also specify flow information that is specific to a section, rather than to a card or card design element. For example, as shown in Figure 4A, flow information can be represented by a line connector that can terminate a section, meaning that one of the cards in the section will be rendered after the event identified by the source of the line connector.

[0043] The IGDS 100 may further implement section logic 129 to maintain state information for each identified section. The IGDS 100 may implement section logic 129 to maintain state information as the sections are rendered during a simulated rendering of the design 125. This state information may contribute to determining the sequence in which cards are rendered during a simulation.

[0044] Simulation Engine 2 illustrates a simulation engine for utilizing section groupings, according to one or more embodiments. The simulation engine 200 is implemented or otherwise provided in the IGDS 100 to enable a user to simulate how a sequence of cards would be rendered in a production environment (a "production environment rendering" or "simulation rendering"), with each card including a top-level frame containing a set of design elements. Thus, the simulation engine 200 can generate a production environment rendering as output, often utilizing a collection 201 of various cards 200, where the design elements of each card 202 are combined to simulate a set of production elements for a user interface or presentation in the production environment.

[0045] In some examples, the simulation engine 200 can be implemented as part of the rendering engine 120 of the IGDS 100. For example, the IGDS 100 can implement alternative modes, including a design mode and a simulation mode, in which the rendering engine 120 executes the processes of the simulation engine 200 to render as output a production environment rendering 205 that simulates the design interface during production. The production environment rendering 205 can be provided to the user devices 10, 12 to allow designs and users of the IGDS 100 to see how their designs in progress will appear in a production environment. In variations, the simulation engine 200 can be implemented as a separate component or application.

[0046] In the example, simulation engine 200 includes processes represented by section logic 210 and simulation rendering logic 220. When launched, simulation engine 200 generates a production environment rendering 205 of a set of cards 202 comprising a particular design 201 or presentation. During or in the context of a simulation of the production environment rendering, section logic 210 may execute to identify which cards 202 of the design or presentation to load, and simulation rendering logic 220 generates the production environment for rendering.

[0047] Simulation rendering logic 220 generates a production environment rendering 205 from each card 202 processed by simulation engine 200, where the production environment rendering 205 includes production elements of a simulated user interface or presentation. Additionally, the production environment rendering 205 can respond interactively or dynamically to events, such as responding to user input that simulates end-user input in the production environment.

[0048] In an example, the simulation rendering 205 can be ordered based at least in part on conditions specified in information associated with individual cards 202 (e.g., line connectors indicating flow) and state information associated with each section. For example, a line connector (or flow connector) can identify the sequence in which a given card is rendered and the condition specified by the line connector is detected (e.g., a design element identified by the line connector receives input), and the section that will determine the next card or cards after that. Each time a card is rendered from one of the sections 212, the section logic 210 updates the state information 221 recorded in the state memory 222. Additionally, the simulation rendering logic 220 can process flow information (e.g., line connections) associated with the rendered card 202 in response to detecting an event (e.g., user interaction with a design element of the rendered card 202A). Based on this flow information, the simulation rendering logic 220 can identify a target for determining the next card in the flow or sequence. If the flow identifies, for example, another card, the simulation rendering logic 220 renders the next card. If a flow identifies a section as a target of flow information, the simulation rendering logic 220 checks the state information 221 for that section from the state memory 222. If state information 221 is identified, the simulation rendering logic 220 uses the state information to generate a rendering of the card identified from the state information 221 (e.g., the card most recently rendered in that section). If there is no state information for the identified section, default sequence rules may be used to identify which card in the section should be rendered. After rendering each card, the section logic 210 again updates the state information 221 stored in the state memory 222.

[0049] Among other examples, examples such as the one illustrated in Figure 2 allow design users to specify simplified flow information in the design interface so that simulated renderings of the production environment more accurately reflect end-user interactions with the production environment.

[0050] methodology

[0014] Figure 3A illustrates an example method for implementing logic for providing section groupings (or sections) in an integrated graphic design system (IGDS) according to one or more embodiments. Figure 3B illustrates an example method for generating a production environment rendering for a simulation in which conditional sequencing of individual cards is based at least in part on state information associated with the sections, according to one or more embodiments. In describing the examples of Figures 3A and 3B, reference will be made to elements of Figures 1 and 2 for ease of explanation.

[0051] Referring to FIG. 3A , in step 310, a user of the IGDS 100 defines one or more sections of a design interface or presentation. The IGDS 100 operates in design mode to enable individual or collaborative users to create and update designs or presentations. A design or presentation can include a collection of cards, with each card corresponding to application or presentation content, for example, a display screen, window, or page. The IGDS 100 can maintain a hierarchical node representation of the design or presentation on the canvas 122, with a node created as a top-level or root node representing the corresponding section of the design interface or presentation. In an example, the IGDS 100 maintains a document object model (DOM) of the designed interface or presentation, which includes a hierarchical arrangement of nodes. Furthermore, in some examples, a section can be defined as the root node (level 0). Within each root node, subnodes with different sublevels can be arranged. In some examples, each card in the collection is represented by a top-level subnode (level 1), and any design elements that do not have a parent (i.e., design elements that are not nested in any other design elements other than the card's container) may be represented by the next-highest subnode (level 2) of the top-level subnode (e.g., container). Similarly, any child design elements to one of the design elements represented by any of the parentless design elements may be represented by a third-level subnode (i.e., level 3 node), and so on.

[0052] In step 320, user input is received to identify cards for the sections. The input may be received over multiple time periods. For example, a user may initially select various cards that will comprise each of one or more sections. As described in other examples, a section corresponds to a grouping of cards, with each card being a container that displays, for example, a production environment screen (or a screen for a particular state) or a paginated presentation (e.g., a slide in a slide deck). For each of one or more sections, the IGDS 100 may enable a user to select cards from a larger collection of cards that form the designed interface or presentation. For example, a design user may select a section that contains cards for a given module or workflow of an application (e.g., a mobile app). In some implementations, a collection of cards for the designed interface or presentation may be rendered on the canvas 122 at one time. A user may utilize tools or otherwise interact with the canvas to select one or more cards to group as a given section. Additionally or as a variation, a user may select, delete, or modify the cards that comprise a section.

[0053] When a user selects a card for a section, the IGDS 100 can implement a process that updates the DOM of the design or presentation. As described, each section can correspond to a root node in the DOM representation. In some implementations, the creation of a section can result in a new root node corresponding to the newly created section. Furthermore, each card associated with a section can be hierarchically located under the section node in the DOM representation.

[0054] In step 330, the user specifies flow connections for the design interface or presentation. The flow connections can specify conditional flows that specify the sequence in which different cards in the designed interface or presentation are rendered in a production environment. In some examples, the flow connections are rendered as graphic elements on the canvas, for example, in the form of arrows or lines with terminal segments, to reflect the sequence or flow direction. The graphic elements can be rendered on the canvas when the IGDS 100 is in design mode. When a simulation is implemented, the graphic elements that display the flow connections can be hidden (or not rendered) because the graphic elements do not form part of the production rendering.

[0055] For each defined section, in step 334, the user can specify various types of flow information, including internal flow information that identifies other cards in the common section and external flow information that designates the section as a target for the flow. Various flow connectors can combine to specify the conditions under which a given sequence of cards can be rendered in a production environment. As shown in the example of FIGS. 4A-4F, the flow information can be specified as a line connector (or flow connector) that designates a source or origin and a target. In an example, the user can provide input to the flow information by designating a section as a target. As will be further described, in such cases, state information associated with the section determines which cards in the section are rendered in a given sequence for that flow.

[0056] 3B , at step 350, a production environment rendering of a design interface or presentation may be initiated by IGDS 100 operating in simulation mode. At step 352, simulation rendering logic 220 renders the first card of the design or interface. At step 360, section logic 210 and / or simulation rendering logic 220 processes information associated with the rendered or active card to record state information for the corresponding section. The state information may include, for example, (i) an identifier for the section that contains the card, (ii) a flow connector that defines a condition for identifying the next card in the design interface or presentation to be rendered, and (iii) a condition for selecting which of multiple flow connectors to use in determining which card in the design interface or presentation to render next by simulation engine 200. In an example, section logic 210 updates state information 221 stored in state memory 222 for the identified section, where the state information identifies which card in the section was most recently rendered.

[0057] In step 370, when section logic 210 detects one or more events, section logic 210 identifies flow information (e.g., lines or flow connections) for the rendered card. In step 380, if the identified flow information identifies another card, in step 382, ​​simulation rendering logic 220 renders the next card as part of the production environment rendering. In step 390, if the flow information identifies a section rather than a specific card, in step 392, rendering logic identifies which card of the identified section is the next card based on the identified section's state information. For example, the identified flow connector may specify a section identifier. If the identified flow connector identifies another section, section logic 210 retrieves state information for the identified section from state memory 212. The state information may identify a card of the section most recently rendered during the simulated rendering of the design interface or presentation, and the card identified by the state information may be rendered as the next card. In a variant, the card identified by the state information is used to determine which card to render next.

[0058] As the next card is rendered, the section logic 210 updates the state information for the particular section of the next card, step 394. This method repeats until the simulation engine has finished rendering the card.

[0059] Example FIG. 4A illustrates a canvas 402 on which elements of a design interface 410 are rendered to enable design input and modification. The example of FIG. 4A may be implemented by one or more users operating the IGDS 100 in design mode. As shown, the design interface 410 includes various cards 422, 424, 426, and 428, which may be grouped into sections 430 and 432. The grouping of cards 422 and 424 as section 430 and cards 426 and 428 as section 432 may be implemented by user input. For example, a user may designate each set of cards as sections 430 and 432 by drawing a box around each set of cards 422, 424, and 426, 428, respectively.

[0060] In design mode, a user can specify multiple flows that define the sequence in which individual cards 422, 424, 426, and 428 in the design interface 410 are rendered in a production environment. The user can operate the IGDS 100 to specify flows using visual line connectors 442 and 444. The line connectors 442 and 444 may extend from a card (source) to a section (target), signifying a production environment sequence in which one of the cards in the target section is rendered in the production environment after the source card is rendered. Each line connector 442 and 444 can indicate a condition or event related to the source. For example, a line connector originating from a particular function of a source card indicates that an event related to that particular function (e.g., received user input) triggers the flow (or sequence of rendering) indicated by the line connector.

[0061] Further, by way of example, the determination of which cards in a given section are rendered can be conditional based on state information recorded or otherwise developed during production environment rendering. Furthermore, line connectors can be extended between cards, such as between cards in a given section 430, 432, to define the sequence in which the cards in the section are rendered. By way of example, within each section 430, 432, the sequence in which cards are rendered in the simulated environment can be determined by default to correspond to the positioning of cards along a horizontal axis, e.g., with the leftmost card being rendered as the first card in the section. In the absence of other inputs or events, the next card to be rendered can correspond to the card positioned immediately adjacent to the rendered card and to the right. The implementation of such default sequence rules can vary based on the implementation.

[0062] 4B-4F illustrate simulated production environment renderings of design interface 410. These simulated production renderings can be generated by simulation engine 200 operating as part of or in conjunction with IGDS 100 implemented in simulation mode. In the example of FIG. 4B, an initial screen 452 of an interface (e.g., for a mobile device app) is rendered corresponding to card 422 in section 430. Section 430 includes cards 422 and 424 (based on user selection). The determination of initial screen 452 can be based on default rules or settings, user selection, and / or user preference. For example, section 430 can be selected for rendering in the simulated production environment based on user input or specification, and the selection of card 422 as the first card to be rendered can be based on default rules from left to right.

[0063] 4C shows that, by default, card 424 of section 430 is next rendered as screen 454. Upon generating the simulated production environment rendering of each card 422, 424, simulation engine 200 records state information reflecting which card 424 was most recently rendered. In examples, the state information may hold additional information such as the identity of each rendered card, the relative timing or sequence in which each card was rendered, and / or the duration during the simulation interval in which each card of the section was rendered.

[0064] Following screen 454, simulation engine 200 can detect an event defined by line connector 442. For example, the event can correspond to a user interacting with the design element that is the source of line connector 442. When line connector 442 terminates at section 432, simulation engine 200 can utilize state information associated with section 432 to determine which of cards 426, 428 in section 432 to render next during the simulation. At the start of the simulation, neither of cards 426, 428 in section 432 may have been rendered. Thus, in FIG. 4D , simulation engine 200 renders card 426 as the next screen 456 based on default sequencing rules (e.g., the left-most card in the section is rendered first, followed by the card immediately to the right, etc.). Similarly, in response to a specified event (e.g., user interaction with a tab, the passage of time), in FIG. 4E , card 428 is selected as the next screen 458 using default sequencing rules. Then, in response to an event specified by line connector 444 (e.g., a user interaction with the "Home" design element), simulation engine 200 determines the next panel to display as screen 460. Line connector 444 terminates at section 430. As described in the example, simulation engine 200 utilizes state information associated with section 430 to determine which of cards 422, 424 in section 430 to render as the display screen. Referring to FIG. 4F, in this example, the state information reflects that card 424 was most recently rendered. Based on this state information, card 424 is used to render screen 460.

[0065] Among other benefits and advantages, the described example eliminates the traditional practice of utilizing line connectors between individual cards to transition between cards (to simulate a production environment). In this traditional approach, the use of line connectors can clutter the screen display and complicate users' understanding of the flows implemented between various panels of a design interface. Also, in a collaborative environment, newly created flows by one user can be difficult for other users to discover or incorporate. In contrast, the described example allows design users to terminate line connectors representing card transitions for a given flow on the section of the target card. Furthermore, by utilizing state information to determine which card in a section to render, the example prevents the undesirable result of a flow returning to the first card in the section (due to default sequencing rules). This allows design users to better visualize the flow of a design interface or presentation.

[0066] Network Computer System FIG. 5 illustrates a computer system on which one or more embodiments may be implemented. Computer system 500 may be implemented, for example, on a server or combination of servers. For example, computer system 500 may be implemented as network computing system 150 of FIGS. 1A-1C. Additionally, in some examples, computer system 500 may provide instructions to user devices to enable the user devices to implement the functionality of IGDS 100. Additionally, computer system 500 may provide instructions to user devices or otherwise perform operations to implement an example method (or steps therein) such as those described in FIGS. 3A and 3B.

[0067] In one implementation, computer system 500 includes processing resources 510, memory resources 520 (e.g., read-only memory (ROM) or random access memory (RAM)), one or more instruction memory resources 540, and a communication interface 550. Computer system 500 includes at least one processor 510 for processing information stored with memory resources 520, such as provided by random access memory (RAM) or other dynamic storage device, to store information and instructions executable by processor 510. Memory resources 520 may also be used to store temporary variables or other intermediate information during execution of instructions executed by processor 510.

[0068] The communication interface 550 allows the computer system 500 to communicate with one or more user computing devices through one or more networks (e.g., cellular networks) through the use of a network link 580 (wireless or wired). Using the network link 580, the computer system 500 can communicate with one or more computing devices, special purpose devices and modules, and / or one or more servers.

[0069] In an example, the processor 510 may execute service instructions 522 stored with the memory resources 520 to enable the network computing system to implement the network service 152 and operate as the network computing system 150 in an example such as that described with reference to Figures 1A-1C.

[0070] Computer system 500 may also include additional memory resources ("instruction memory 540") for storing executable instruction sets ("IGDS instructions 545") embedded in web pages and other web resources to enable user computing devices to implement functionality as described in IGDS100.

[0071] As such, the examples described herein relate to the use of computer system 500 for performing the techniques described herein. According to one aspect, the techniques are performed by computer system 500 in response to processor 510 executing one or more sequences of one or more instructions contained in memory 520. Such instructions may be read into memory 520 from another machine-readable medium. Execution of the sequences of instructions contained in memory 520 causes processor 510 to perform the process steps described herein. In alternative implementations, hardwired circuitry may be used in place of or in combination with software instructions to implement the examples described herein. Thus, the described examples are not limited to any specific combination of hardware circuitry and software.

[0072] User Computing Device 6 illustrates a user computing device for use in one or more examples, as described. In examples, user computing device 600 may correspond to, for example, a workstation, desktop computer, laptop, or other computer system having graphics processing capabilities suitable for rendering design interfaces and enabling graphic design work. In variations, user computing device 600 may correspond to a mobile computing device, such as a smartphone, tablet computer, laptop computer, VR or AR headset device, etc.

[0073] In the example, computing device 600 includes a central or main processor 610, a graphics processing unit 612, memory resources 620, and one or more communication ports 630. Computing device 600 can use main processor 610 and memory resources 620 to store and launch a browser 625 or other web-based applications. A user can operate browser 625 to access a network site of network service 152 using communication port 630, where the user can download one or more web pages or other resources 605 (see FIGS. 1A-1C ) for network service 152. Web resources 605 can be stored in active memory 624 (cache).

[0074] As described in various examples, the processor 610 can detect and execute scripts and other logic embedded in web resources to implement the IGDS 100 (see FIGS. 1A-1C). Additionally, the processor 610 can execute scripts or instructions to perform exemplary methods, such as those described in the example of FIG. 3. In some examples, some of the scripts 615 embedded in the web resources 605 can include GPU-accelerated logic executed directly by the GPU 612. The main processor 610 and the GPU can combine to render an editing design interface (“DUE 611”) on the display component 640. The rendered design interface can include web content from the browser 625 and design interface content and functional elements generated by the scripts and other logic embedded in the web resources 605. By including scripts 615 that can be executed directly on the GPU 612, the logic embedded with the web resources 615 can better execute the IGDS 100, as described in various examples.

[0075] conclusion Although embodiments are described in detail herein with reference to the accompanying drawings, it should be understood that the concepts are not limited to those precise embodiments. Accordingly, it is intended that the scope of the concepts be defined by the following claims and their equivalents. Furthermore, it is contemplated that particular features described individually or as part of an embodiment can be combined with other individually described features or with parts of other embodiments, even if the other features and embodiments do not refer to the particular feature. Thus, the absence of a description of a combination should not exclude entitlement to such a combination.

Claims

1. 1. A computer system comprising: one or more processors; Memory that stores the instruction set and Equipped with The one or more processors execute instructions stored in the memory to allowing a user to specify multiple cards comprising a design interface or presentation, each of the multiple cards being renderable in a simulation or production environment separately from other cards of the multiple cards; allowing a user to designate one or more sections, each section including a grouping of various cards from the plurality of cards; allowing a user to specify a plurality of flow connections, the plurality of flow connections including at least a first flow connection from one of the plurality of cards to a first section of the one or more sections, the one of the plurality of cards of the first flow connection not being part of the first section; rendering cards of the plurality of cards in a sequence based at least in part on one or more of the flow connections, including at least the first flow connection, during a simulated rendering of the design interface or presentation; A computer system that performs operations including:

2. Rendering a card of the plurality of cards during the simulation rendering includes: determining state information for the first section; selecting and rendering one of the cards in the first section based on the state information; and 2. The computer system of claim 1, comprising:

3. 3. The computer system of claim 2, wherein determining state information for the first section includes identifying a most recent card of the section that was rendered during the simulation rendering.

4. 4. The computer system of claim 3, wherein rendering cards of the plurality of cards in the sequence comprises rendering at least one card based on the state information.

5. 3. The computer system of claim 2, wherein the state information identifies conditions for selecting which of the plurality of flow connections to use to determine the next card to render in the simulation rendering.

6. The computer system of claim 5 , wherein the operations further comprise updating the state information based on the next card rendered during the simulation rendering.

7. 2. The computer system of claim 1, wherein each of the plurality of flow connections is designated as a corresponding graphic element that is rendered on a canvas of the design interface or presentation during design mode.

8. 1. A computer-implemented method comprising: allowing a user to specify a plurality of cards comprising a design interface or presentation, each of the plurality of cards being renderable in a simulation or production environment separately from other cards of the plurality of cards; allowing a user to specify one or more sections, each section including a grouping of various cards from the plurality of cards; allowing a user to specify a plurality of flow connections, the plurality of flow connections including at least a first flow connection from one of the plurality of cards to a first section of the one or more sections, wherein the one of the plurality of cards of the first flow connection is not part of the first section; during a simulated rendering of the design interface or presentation, rendering cards of the plurality of cards in a sequence based at least in part on one or more of the flow connections, including at least the first flow connection; A method comprising:

9. Rendering a card of the plurality of cards during the simulation rendering includes: determining state information for the first section; selecting and rendering one of the cards in the first section based on the state information; 9. The method of claim 8, comprising:

10. The method of claim 9 , wherein determining state information for the first section comprises identifying a most recent card of the section that was rendered during the simulation rendering.

11. The method of claim 10 , wherein rendering cards of the plurality of cards in the sequence comprises rendering at least one card based on the state information.

12. The method of claim 9 , wherein the state information identifies conditions for selecting which of the plurality of flow connections to use to determine a next card to render in the simulation rendering.

13. The method of claim 12 , further comprising updating the state information based on the next card rendered during the simulation rendering.

14. The method of claim 8 , wherein each of the plurality of flow connections is designated as a corresponding graphic element that is rendered on a canvas of the design interface or presentation during design mode.

15. A non-transitory computer-readable medium storing instructions, comprising: When executed by one or more processors of a computer system, the computer system: allowing a user to specify multiple cards comprising a design interface or presentation, each of the multiple cards being renderable in a simulation or production environment separately from other cards of the multiple cards; allowing a user to designate one or more sections, each section including a grouping of various cards from the plurality of cards; allowing a user to specify a plurality of flow connections, the plurality of flow connections including at least a first flow connection from one of the plurality of cards to a first section of the one or more sections, the one of the plurality of cards of the first flow connection not being part of the first section; rendering cards of the plurality of cards in a sequence based at least in part on one or more of the flow connections, including at least the first flow connection, during a simulated rendering of the design interface or presentation; A non-transitory computer-readable medium for causing operations to be performed, including:

16. Rendering a card of the plurality of cards during the simulation rendering includes: determining state information for the first section; selecting and rendering one of the cards in the first section based on the state information; and 20. The non-transitory computer-readable medium of claim 15, comprising:

17. The non-transitory computer-readable medium of claim 16 , wherein determining state information for the first section includes identifying a most recent card of the section rendered during the simulation rendering.

18. 20. The non-transitory computer-readable medium of claim 17, wherein rendering cards of the plurality of cards in the sequence comprises rendering at least one card based on the state information.

19. 20. The non-transitory computer-readable medium of claim 17, wherein the state information identifies conditions for selecting which of the plurality of flow connections to use to determine a next card to render in the simulation rendering.

20. 20. The non-transitory computer-readable medium of claim 19, wherein the operations further comprise updating the state information based on the next card rendered during the simulation rendering.

Images