A constraint-based layout system for efficient placement and control of user interface elements
By specifying the layout relationships of user interface elements using a graphical method based on the Cassowary algorithm, the problems of complexity and performance limitations of traditional layout systems are solved, enabling more efficient user interface development and a better user experience.
Patent Information
- Application Number
- CN202111325406.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2017-04-21
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2037-04-21
AI Technical Summary
Traditional layout systems are complex and performance-limited in user interface development, requiring developers to write linear equations, resulting in time-consuming development processes and poor user experience.
The layout system, based on the Cassowary algorithm, specifies the relationships between widgets through a graphical method and uses a general linear equation solver to generate layout constraints, reducing the complex input requirements for developers and providing better performance and flexibility.
It simplifies the development process, improves layout performance and flexibility, reduces the complexity of the view hierarchy, and enhances the efficiency and predictability of user interface development.
Smart Images

Figure CN114217888B_ABST
Abstract
Description
[0001] Description of the case
[0002] This application is a divisional application of Chinese invention patent application No. 201780016736.X, filed on April 21, 2017.
[0003] CROSS-REFERENCE TO RELATED APPLICATIONS
[0004] This application claims priority under 35 U.S.C. §119 to U.S. Provisional Patent Application Serial No. 62 / 337,850, filed on May 17, 2016, entitled “CONSTRAINTS-BASEDLAYOUT SYSTEM FOR EFFICIENT LAYOUT AND CONTROL OF USER INTERFACE ELEMENTS,” and U.S. Provisional Patent Application Serial No. 62 / 337,852, filed on May 17, 2016, entitled “SYSTEMS AND METHODS FOR AUTOMATIC DETERMINATION OF LAYOUT CONSTRAINTS,” the entire contents of which are incorporated herein by reference. Technical Field
[0005] The present disclosure relates generally to computer-based layout systems for user interface development and deployment, and more particularly to constraint-based layout systems for efficient layout and control of user interface elements. Background Art
[0006] User interfaces are designed to adapt to various screen conditions, such as screen size, screen density, and the human language used. Traditional layout systems allow developers to specify how widgets are laid out, and how their layout positions change to adapt to changing screen conditions. However, traditional layout systems are complex and have performance limitations, which can make creating user interfaces an undesirable and time-consuming process for developers.
[0007] The description provided in the Background section should not be considered prior art merely because it is mentioned in or related to the Background section.The Background section may include information describing one or more aspects of the subject technology. Summary of the invention
[0008] The disclosed subject matter relates to a constraint-based layout system for efficient layout and control of user interface elements. More specifically, a constraint-based layout system and method are provided that allows developers to more easily and efficiently specify how widgets are laid out, and how their positions change to accommodate those changing and / or variable display conditions (e.g., various screen sizes, screen densities, and languages). This facilitates additional user interface complexity and design options.
[0009] According to aspects of the present disclosure, a computer-implemented method is provided. The method includes providing a layout application for developing a user interface (UI) on a computing device, wherein the layout application includes a representation of a page. The method includes receiving a first user input via the layout application, wherein the first user input indicates an instruction to position a first UI element on the page. The method includes receiving a second user input via the layout application, wherein the second user input indicates an instruction to position a second UI element on the page. The method includes receiving a third user input via the layout application, wherein the third user input indicates an instruction to create a connection from the second UI element to the first UI element. The method includes generating a layout constraint indicating a spatial relationship between the first UI element and the second UI element based on the connection. The method also includes providing the layout constraint in a layout data file associated with the UI.
[0010] According to aspects of the present disclosure, a system is provided, comprising one or more processors, and a computer-readable storage medium coupled to the one or more processors, the computer-readable storage medium comprising instructions. The instructions cause the one or more processors to provide a layout application for developing a UI on a computing device, wherein the layout application includes a representation of a page. The instructions also cause the one or more processors to receive a first user input via the layout application, wherein the first user input indicates an instruction to locate a first UI element on the page. The instructions also cause the one or more processors to receive a second user input via the layout application, wherein the second user input indicates an instruction to locate a second UI element on the page. The instructions also cause the one or more processors to receive a third user input via the layout application, wherein the third user input indicates an instruction to create a connection from the second UI element to the first UI element. The instructions also cause the one or more processors to generate a layout constraint indicating a spatial relationship between the first UI element and the second UI element based on the connection. The instructions also cause the one or more processors to provide the layout constraint in a layout data file associated with the UI.
[0011] According to aspects of the present disclosure, a non-transitory computer-readable storage medium including instructions is provided, which, when executed by a processor, causes the processor to perform a method. The method includes providing a layout application for developing a UI on a computing device, wherein the layout application includes a representation of a page. The method includes receiving a first user input via the layout application, wherein the first user input indicates an instruction to position a first UI element on the page. The method includes receiving a second user input via the layout application, wherein the second user input indicates an instruction to position a second UI element on the page. The method includes receiving a third user input via the layout application, wherein the third user input indicates an instruction to create a connection from the second UI element to the first UI element. The method includes generating a layout constraint indicating a spatial relationship between the first UI element and the second UI element based on the connection. The method also includes providing the layout constraint in a layout data file associated with the UI. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The accompanying drawings, which are included to provide a further understanding of the specification and are incorporated in and constitute a part of the specification, illustrate the disclosed embodiments and together with the description serve to explain the principles of the disclosed embodiments. In the drawings:
[0013] Figure 1 An exemplary architecture of a layout tool suitable for implementing some embodiments of the present disclosure is shown.
[0014] Figure 2 It is shown that according to certain aspects of the present disclosure, Figure 1 A block diagram of an exemplary client and server of an architecture.
[0015] Figure 3 Shows the use of Figure 2 An exemplary client of, an exemplary process for building a constraint-based layout system.
[0016] Figure 4 Shows the use of Figure 2 An exemplary process for developing and deploying an exemplary client and server user interface.
[0017] Figures 5 to 9 A schematic diagram showing exemplary layout constraints is shown.
[0018] Fig.10 A diagram illustrating an example of dimension-dependent layout constraints.
[0019] Fig.11A and 11B An example of a chain-related placement constraint is shown.
[0020] Fig.12An example of a virtual object used to set layout constraints is shown.
[0021] Fig.13A and 13B An example of a layout constraint indicating an object that disappears at runtime is shown.
[0022] Fig.14 An example of an editor interface for an application for developing and deploying user interfaces is shown.
[0023] Fig.15 It shows that it is possible to Figure 2 A block diagram of an exemplary computer system of a client and a server.
[0024] In one or more embodiments, not all components depicted in each figure may be required, and one or more embodiments may include additional components not shown in the figures. Without departing from the scope of the present disclosure, the arrangement and type of components may be changed. Within the scope of the present disclosure, additional components, different components or fewer components may be utilized. DETAILED DESCRIPTION
[0025] The detailed description set forth below is intended to describe various embodiments, and is not intended to represent the only embodiment in which the subject technology can be implemented. As will be appreciated by those skilled in the art, the embodiments described can be modified in various ways without departing from the scope of the present disclosure. Therefore, the drawings and descriptions should be viewed as exemplary rather than restrictive in nature.
[0026] As used herein, the term "widget" may refer to an application or a component of an application that performs a function that implements an application or provides access to services associated with an application. As used herein, the term "anchor" may refer to an element of a widget that is used to establish a connection with another object on a page, such as another widget.
[0027] General Overview
[0028] Traditional layout systems have been developed based on the Cassowary algorithm that discloses the use of constraint systems for user interfaces. Specifically, based on the algorithm, a linear system solver is used to solve the constraint system, thereby allowing iterative solutions for real-time use of the user interface. However, there are problems with traditional UI layout systems. The UI consists of multiple display objects commonly referred to as "widgets". Determining how to position these widgets on the display screen while taking into account complex considerations such as various device screen sizes, display screen densities, and / or various human languages included in the display is a complex task. Existing layout systems that express these layout considerations do not minimize this complexity for developers. In some cases, these traditional layout systems fully expose the linear system to developers, resulting in complex and cumbersome development time to predict the behavior of UI layout. This exposure can also cause performance issues that reduce the layout system, and due to the difficult input requirements of the layout system, it can also adversely affect the user experience of developers. For example, such a layout system requires developers to specify the layout by mainly writing linear equations, which is a cumbersome task for developers. Other traditional layout systems separate various functions into specific use cases (e.g., implementing content layout as a series of widgets, organizing the layout in a grid, etc.), thereby placing unnecessary burden on developers to use multiple components before arriving at the final layout.
[0029] The disclosed technology provides for expressing complex constraints on widgets using a small, general set of rules that allows developers to express arbitrary relationships between widgets (e.g., one-to-one, chained). The system is based on a general linear equation solver, facilitating internal flexibility and adaptability. The specific set of rules defined provides better performance and predictability, and allows widget positions to be expressed in a simpler manner than directly creating linear equations. In addition, by moving the entire layout process in a single location, better layout performance can be achieved (no longer requiring the complex view hierarchies typically found on mobile operating systems), as well as better flexibility and new animation capabilities.
[0030] The disclosed technology solves problems of traditional layout systems that occur particularly in the field of computer technology by providing solutions that also originate from computer technology, namely by defining a higher-level set of constraints that are used to express layout relationships. The layout system of the present disclosure is based on a specially created effective implementation of the Cassowary algorithm. One of the distinguishing features between the layout system of the present disclosure and traditional layout systems is that the Cassowary method used in traditional layout systems requires developers to specify layouts by mainly writing linear equations, while the present disclosure provides a graphical method for layout development and deployment. In this regard, the layout system of the present disclosure eliminates the burden of developers writing linear equations.
[0031] In addition to the basic layout capabilities of the layout system of the present disclosure, the disclosed technology allows for a complete separation between the widget hierarchy and the way the widgets are positioned. This has a wide range of implications in terms of performance and flexibility. For example, traditional layout systems provide a view hierarchy that tends to be deep and complex due to layout, while the layout system of the present disclosure eliminates this need. In addition, the layout system of the present disclosure provides conceptual virtual containers / helpers and animation elements, which create greater flexibility in the layout system because they share the same "scene".
[0032] The disclosed techniques further provide improvements to the functionality of the computer itself because: (1) a minimal set of different constraints is required, thereby requiring fewer computer operations and memory resources; (2) the solution is easier to communicate to the user, rather than requiring the user to provide input identifying the linear equation written; (3) critical points are predictable - both from the user's perspective and from a performance perspective; and (4) greater flexibility is allowed in the types of layouts supported.
[0033] Exemplary System Architecture
[0034] Figure 1 An exemplary architecture 100 for efficient layout and control of user interface elements based on a constraint-based layout system during development and deployment of user interfaces is shown. The architecture 100 includes a server 130 and a client 110 connected via a network 150.
[0035] One of the many servers 130 is configured to host an application server and / or a web server. The server 130 may be any device having an appropriate processor, memory, and communication capabilities for hosting an application server and / or a web server. The constraint-based layout system may be accessed on various clients 110. In some aspects, the constraint-based layout system may be downloaded from the server 130 via a network 150 and installed on the client 110. The client 110 may be, for example, a desktop computer, a mobile computer, a tablet computer (e.g., including an e-book reader), a mobile device (e.g., a smart phone or a PDA), or any other device having an appropriate processor, memory, and communication capabilities for accessing an image search engine on one of the servers 130. The network 150 may include, for example, any one or more of a local area network (LAN), a wide area network (WAN), the Internet, and the like. In addition, the network 150 may include, but is not limited to, any one or more of the following network topologies, including a bus network, a star network, a ring network, a mesh network, a star bus network, a tree or hierarchical network, and the like.
[0036] Exemplary constraint-based layout system
[0037] Figure 2According to certain aspects of the present disclosure, Figure 1 1 and 10. The server 130 includes a processor 236, a communication module 238, and a memory 232. The memory 232 includes resources 234. The client 110 includes a processor 212, a communication module 218, and a memory 220. The client 110 and the server 130 are connected through a network 150 via communication modules 218 and 238, respectively. The communication modules 218 and 238 are configured to connect to the network 150 to send and receive information such as data, requests, responses, and commands to other devices on the network. The communication modules 218 and 238 can be, for example, modems or Ethernet cards.
[0038] The memory 220 of the client 110 includes an application 221, a UI construction engine 222, a UI reasoning engine 223, and a linear equation engine 224. The memory 220 also includes constraint data 225 and a library 226. One or more of the application 221, the UI construction engine 222, the UI reasoning engine 223, and the linear equation engine 224 may be operated (e.g., using the constraint data 225 and / or the library 226) to provide a developer with a layout tool for visual layout of display objects of a user interface.
[0039] The processor 212 of the client 110 is configured to execute instructions, such as instructions physically encoded into the processor 212, instructions received from software in the memory 220, or a combination of both. For example, the processor 212 of the client 110 executes instructions to provide a layout tool for developing a user interface (UI) of an application for display on the output device 214. In some aspects, the layout tool includes a representation of a page. In some implementations, the page represents a web page or a login page.
[0040] In some aspects, the processor 212 of the client 110 is configured to receive user input from the user via an editor interface (e.g., a layout tool) of the application 221. The user input indicates an instruction to position the first UI element on the page. For example, the user may access the application 221 in the memory 220 on the user's client 110, and the user may submit the user input using the input device 216 of the client 110 to receive the user input. In another aspect, the processor 212 of the client 110 is configured to receive a second user input from the user via the layout tool. The second user input may indicate an instruction to position the second UI element on the page.
[0041] The user of the client 110 can submit a third user input using the input device 216, which indicates an instruction to create a connection from the second UI element to the first UI element via the layout tool. The layout tool may include an input portion, wherein the positioning of the displayed widgets can be manipulated by the user. The input portion may include a layout area and one or more controls to allow the user to connect the widgets together and align the widgets relative to each other within the layout area. For example, the first UI element may be a first widget, and the second UI element may be a second widget, wherein the second widget is positioned relative to the first widget. In this example, the first widget may be connected to one side of the second widget (or via an anchor), thereby creating a spatial relationship between the first widget and the second widget. In this regard, the spatial relationship can indicate how to position the first widget relative to the second widget on a page (or application interface layout) without specifying a specific position (or coordinates) for each of the first widget and the second widget, and vice versa.
[0042] In some aspects, the processor 212 uses the application 221 and the UI construction engine 222 to generate a layout constraint indicating the spatial relationship between the first UI element and the second UI element based on the connection. In some embodiments, the layout constraint indicates that the first UI element and the second UI element are co-located at the same position on the page, or the first UI element and the second UI element are separated from each other in space by a defined spacing. Depending on the implementation, the spacing can be based on density-independent pixels (dp or dip), scaling-independent pixels (sp), pixels (px) or points (pt). In some aspects, the processor 212 uses the UI construction engine 222 and the linear equation engine 224 to provide layout constraints in a layout data file associated with the UI. For example, the layout constraint may include a linear equation that represents the connection between the first UI element and the second UI element as a mathematical expression. In some aspects, the layout constraint is stored as constraint data 225. In some embodiments, the processor 212 uses the application 221 and the library 226 to provide the linear equation engine 224 with layout constraints to solve the linear equation from the layout constraints in the constraint data 225. For example, linear equation engine 224 may be configured to determine one or more errors in the connection between a first UI element and a second UI element, and minimize the number of errors in the connection by solving the linear equation. In some aspects, solving the linear equation may include modifying the spatial relationship between the first UI element and the second UI element, which minimizes the number of errors in the connection, and providing a new linear equation representing the modified spatial relationship. Library 226 may be a data structure that identifies how each layout constraint is defined.
[0043] As described above, the subject technology maps the two-dimensional placement of display objects such as widgets to a linear equation without requiring manual input from the end user, thereby enhancing the end-user experience of developing a UI for an application. The subject technology also reduces the complexity of hierarchical arrangements found in traditional layout tools by defining constraints based on positioning widgets relative to each other, wherein the hierarchy is defined in relation to a parent container.
[0044] In some embodiments, the constraints are defined by the interconnections between widgets at their anchor locations (see Figure 5 ). The layout constraints may include an anchor spacing that defines a distance between spatially related widgets (see FIG. 6 ). The layout constraints may include an interconnection between at least two widgets via relatively connected anchors, wherein the subject technology attempts to solve a linear equation representing the relatively connected anchors (see Fig. 7A and 7B In this regard, constraints may need to be adjusted to diffuse the amount of errors detected in connections between the widgets' anchor points. In some embodiments, the layout constraints include dimension-related constraints, where one or more dimensions of the widgets are modified to facilitate diffusion of errors found in connections between the widgets (see Fig. 9 and 10 ). The subject technology can automatically center a widget relative to other adjacent widgets (or hierarchically related widgets) using centering constraints determined by a specific rule set (see Fig. 9 In some embodiments, the layout constraints include chain-related constraints, where widgets are connected as a chain, indicating a sequence-based relationship between the widgets in the chain (see Fig.11A and 11B In some embodiments, the subject technology includes virtual containers and / or templates to help group display objects and guide the layout of display objects during the UI development process (see Fig.12 ). In this regard, the virtual guides and / or templates are removed from the display prior to the runtime process of the final layout. In one or more embodiments, the virtual guides and / or templates are not included in the layout constraints and are not part of the UI layout hierarchy.
[0045] Figure 3 Shows the use of Figure 2 The exemplary client 110 of FIG. 10A constructs an exemplary process 300 for identifying constraints for layout relationships. Although reference is made to FIG. Figure 2 describe Figure 3 , but it can be noted that Figure 3 The process steps may be performed by other systems.
[0046] Process 300 begins with start step 301. In step 301, a layout tool provided for display on a computing device may include a representation of a page, wherein the page may also include a first UI element positioned on the page. The first UI element may be a first widget. A layout tool may sometimes be referred to as a graphical development environment or a layout application.
[0047] Next, in step 302, processor 236 uses UI construction engine 222 to receive a first user input indicating an instruction to position a first UI element on the page. In step 303, processor 236 uses UI construction engine 222 to receive a second user input indicating an instruction to position a second UI element on the page. The second UI element can be a second widget. Next, in step 304, processor 236 uses UI construction engine 222 to receive a third user input indicating an instruction to create a connection from the second UI element to the first UI element. On the one hand, the connection can be a connection from the anchor of the first UI element to the anchor of the second UI element. On the other hand, the connection can be created by combining the widget with other widgets in a virtual container (see Fig.12 ).
[0048] Sequentially, in step 305, processor 236 uses UI construction engine 222 to generate a layout constraint indicating the spatial relationship between the first UI element and the second UI element based on the connection. In this regard, the layout constraint represents a specific rule indicating how to position the second UI element on the page relative to the first UI element. In one or more embodiments, the specific rule is generated as a linear equation by UI construction engine 222, which indicates the spatial relationship between UI elements as a mathematical expression. The two-dimensional placement of UI elements is automatically mapped to the linear equation without requiring user input indicating the linear equation. Then, in step 306, processor 236 uses UI construction engine 222 to provide layout constraints in a layout data file associated with the UI. In some embodiments, the layout data file is in an extensible markup language (XML) file format.
[0049] Processor 236 generates layout constraints using UI construction engine 222 and linear equation engine 224 by mapping connections to spatial positioning data and relationship data, the spatial positioning data indicating positions on the page relative to the first UI element and the second UI element, determining coordinates on the page for the first UI element and the second UI element respectively based on the spatial positioning data and the relationship data, and providing the first UI element and the second UI element for display at the determined coordinates. The relationship data indicates a directional relationship between the first UI element and the second UI element relative to one of the first UI element or the second UI element.
[0050] The processor 236 uses the UI construction engine 222 to process the spatial positioning data and the relationship data, and based on the processing, determines whether the relationship data and the spatial positioning data are ambiguous. When more than one potential coordinate set is calculated for at least one of the first UI element or the second UI element, the relationship data and the spatial positioning data may be ambiguous. For example, if the spatial positioning data and / or the relationship data indicate that at least one known dimension of the widget is known, a linear equation solver may not be required, thereby rendering the relationship data and the spatial positioning data as ambiguous. In this regard, the processor 236 uses the UI construction engine 222 independently of the linear equation solver (e.g., the linear equation engine 224) to parse the relationship data and the spatial positioning data to determine the coordinates when it is determined that the relationship data and the spatial positioning data are not ambiguous.
[0051] However, if no known variables satisfy the layout constraints, the relationship data and the spatial positioning data can be transformed into linear equations. In this regard, when it is determined that the relationship data and the spatial positioning data are fuzzy, the processor 236 uses the UI construction engine 222 and the linear equation engine 224 to transform the relationship data and the spatial positioning data into one or more linear equations, and solves the one or more linear equations through the linear equation solver to determine the coordinates when the relationship data and the spatial positioning data are transformed into one or more linear equations. In some aspects, the processor 236 uses the UI construction engine 222 and the linear equation engine 224 to solve the linear equations in the layout constraints. In one or more embodiments, the processor 236 uses the UI construction engine 222 to determine the coordinates on the page for the first UI element and the second UI element, respectively, based on the solved linear equations. In one or more embodiments, the processor 236 uses the UI construction engine 222 to provide the first UI element and the second UI element for display at the determined coordinates.
[0052] In one or more embodiments, the processor 236 uses the UI construction engine 222 to classify the relationship data and the spatial positioning data into a specific class of constraints, and determines whether the specific class of constraints is associated with a first class of constraints or a second class of constraints that is different from the first class of constraints. For example, the first class of constraints includes constraints that do not require a linear equation solver to resolve the constraints, while the second class of constraints require a linear equation solver. In this regard, the processor 236 uses the UI construction engine 222 to parse the relationship data and the spatial positioning data independently of the linear equation solver to determine the coordinates when it is determined that the specific class of constraints is associated with the first class of constraints. On the other hand, when it is determined that the specific constraint set is associated with the second class of constraints, the processor 236 uses the UI construction engine 222 to transform the relationship data and the spatial positioning data into one or more linear equations, and solves the one or more linear equations through the linear equation solver to determine the coordinates when the relationship data and the spatial positioning data are converted into one or more linear equations.
[0053] The first UI element includes a first anchor, and the second UI element includes a second anchor, wherein the second anchor of the second UI element is connected to the first anchor of the first UI element based on the instruction to create the connection from the second UI element to the first UI element. The instruction to create the connection from the second UI element to the first UI element may indicate that the first anchor and the second anchor occupy the same target location on the page. The instruction to create the connection from the second UI element to the first UI element may indicate a defined distance between the first anchor and the second anchor.
[0054] Processor 236 uses UI construction engine 222 to receive a fourth user input indicating an instruction to position a third UI element on a page, and then, via a layout application, receives a fifth user input indicating an instruction to create a first connection from the third UI element to the first UI element and a second connection from the third UI element to the second UI element. Processor 236 uses UI construction engine 222 to determine that the position of the third UI element relative to the first UI element and the second UI element includes an error, wherein the anchor (e.g., the left anchor) on the first UI element and the anchor (e.g., the right anchor) on the second UI element are restricted to be positioned at the same target position on the page due to the error. In one or more embodiments, the error found in the layout constraint (e.g., defining the connection between the first and second UI elements) indicates that the two UI elements cannot be located at the same target position of relative positioning (e.g., the first UI element is located to the left of the second UI element). When a connection is established between two relatively connected anchors of UI elements (e.g., the left anchor of the first UI element is connected to the right anchor of the second UI element), an error can be found.
[0055] Processor 236 uses UI construction engine 222 to diffuse errors to the first connection between the third UI element and the first UI element and the second connection between the third UI element and the second UI element. In this regard, the error diffused to the two connections can be expressed as the ratio of the error amount diffused to the first connection and the error amount diffused to the second connection. Processor 236 uses UI construction engine 222 to modify the ratio of the error amount on the first connection to the error amount on the second connection, and based on the modified ratio, adjust the error amount and the error amount on the first connection. Processor 236 uses UI construction engine 222, and the ratio can be modified based on user input and / or user preferences stored in memory 220. In some aspects, the ratio is modified from a predetermined ratio value to a custom ratio value, wherein the error amount on each connection can be minimized. In one or more embodiments, processor 236 uses UI construction engine 222, based on the modified ratio, reduces the error to a predetermined value. The layout constraint indicates the error amount diffused to each of the connections. The processor 236 then uses the UI construction engine 222 to determine the coordinates of the third UI element based on the adjusted error amount on the first connection and the adjusted error amount on the second connection.
[0056] In one or more embodiments, the layout constraints include a first spacing associated with a first connection from a third UI element to the first UI element and a second spacing associated with a second connection from the third UI element to the second UI element, wherein the first spacing indicates a distance between the first UI element and the third UI element, and the second spacing indicates a distance between the second UI element and the third UI element.
[0057] Processor 236 uses UI construction engine 222 to provide virtual objects on the page for display. In one or more embodiments, the virtual object is a visual guide on the page. The visual guide can be used as a vertical and / or horizontal grid line, which helps the user locate the widget on the page. The visual guide is triggered for display by the user selecting a display control, or the visual guide is automatically rendered when the layout tool is started. Processor 236 uses UI construction engine 222 to receive user input, which indicates an instruction to locate a first UI element or a second UI element relative to the virtual object. Processor 236 uses UI construction engine 222 to render the final layout of the page based on the layout data file. When the final layout is running, the virtual object is removed from the display. The virtual object is intended to help developers during the layout of the page and is not part of the final layout rendering (i.e., not included in the document object model (DOM) hierarchy).
[0058] Figure 4 Shows the use of Figure 2The exemplary process 400 of user interface development and deployment of an exemplary client and server of FIG. When a user loads the application 221 on the client 110, for example, for user interface development and deployment, the process 400 starts at step 401.
[0059] Next, in step 402, the application 221 on the client 110 provides a representation of the page via the editor interface of the application 221 for display. The representation of the page may include a layout of which UI elements such as display objects can be positioned, moved, arranged and constrained within it relative to other display objects, wizards or container features. In step 403, the client 110 uses the application 221 and the UI construction engine 222 to generate a constraint set based on the connection between the UI elements on the page. For example, the constraint set may include rules defining certain connections between UI elements, such as indicating certain connections between UI elements and / or certain spatial relationships between UI elements (e.g., making the first element on the page, the position to the right of the second element). In this regard, the UI construction engine 222 facilitates mapping these connections to linear equations representing various layout constraints without requiring user input indicating or providing these linear equations. Then, in step 404, the client 110 provides the constraint set and the library associated with the application 221 to the linear equation solver (e.g., 224) in the client 110. The linear equation solver may process the constraints to diffuse any amount of error found in the constraints and generate modified constraints with minimized errors.
[0060] In step 405, the client 110 solves the linear equation from the constraint set via the linear equation solver based on the library. Next, in step 406, the client 110 generates a final layout of the UI based on the solved linear equation. In turn, in step 407, the client 110 identifies a request for a resource associated with the UI. In step 408, the client 110 sends the request to the server 130.
[0061] Turning to the server 130, in step 409, the server 130 receives a request for a resource from the client 110. Next, in step 410, the server 130 obtains the requested resource from a data source accessible to the server 130. In step 411, the server 130 provides the requested resource to the client 110.
[0062] Turning to the client 110, at step 412, the client 110 receives the requested resource from the server 130. Next, at step 413, the client 110 provides the requested resource via the UI having the final layout for display. In this regard, the UI may be rendered with content (e.g., text, images, video) from the received resource, providing the widgets and associated content arranged in the final layout for display.
[0063] Constraint System Semantics
[0064] Figure 5 A schematic diagram 500 of exemplary layout constraint semantics is shown. Specifically, Figure 5 The concept of "anchors" presented on user interface elements (e.g., widgets) is provided. A "widget" has one anchor on each side (left, top, right, bottom) and a center (horizontal and vertical) anchor (e.g., 502), as well as a baseline anchor (e.g., 506).
[0065] The general idea of anchor connections is that the connections define the first constraint type, where a connection is established between anchors from each widget. The anchor of the widget that initiates the connection can be called the "source anchor", and the anchor of the widget that receives the connection as the destination can be called the "target anchor". In some embodiments, the source anchor can be located at the same position as the target anchor, or in other embodiments, the source anchor and the target anchor can be located in different positions defined by the spacing. The layout system of the present disclosure then collects all defined connections and tries to minimize errors as much as possible to generate the final layout. Widgets can then be defined entirely relative to each other, rather than relative to their parent container (as is often the case in traditional layout systems).
[0066] Anchor Spacing
[0067] Fig. 6A and 6B Schematic diagrams 600 and 650 are shown that identify exemplary layout constraints for anchor spacing. Fig. 6A In , anchors (e.g., 502) can be connected together. For example, a layout in which a widget "TextView2" (e.g., 604) is located to the right of a widget "TextView1" (e.g., 602) can be created by connecting the left anchor (e.g., 502) of TextView2 to the right anchor (e.g., 502) of TextView1 (e.g., 602). Figure 6B In , when an anchor is connected to another anchor (e.g., a target anchor), it means that the two anchors can occupy the same position.
[0068] In some aspects, the spacing between two anchor points on a connection may also be defined. Fig. 6A shows a spacing (e.g., 606) of 56 pixels between the left anchor (e.g., 604) of TextView2 and the right anchor (e.g., 602) of TextView1, while Figure 6B The same relationship is shown without a defined spacing (eg, 652). Depending on the implementation, the spacing may be any value. Fig. 6Aand 6B The constraints shown in may sometimes be referred to herein as unilateral constraints.
[0069] Relative connection anchor
[0070] Fig. 7A and 7B Schematic diagrams 700 and 750 of exemplary layout constraints identifying relatively connected anchors are shown. When two relatively connected anchors (e.g., 708, 710) are connected, neither anchor can be positioned on their target. In essence, the layout system must accept that there are unavoidable "errors" (errors that the system attempts to minimize).
[0071] exist Fig. 7A In the specific case of relative connection anchors (e.g., 708, 710) in , the layout system of the present disclosure decides to "spread" the error equally across both connections, thereby producing a centered widget (e.g., 706). Figure 7B In , the spacing can be defined and the spacing is Fig. 7A A portion of the layout constraints shown in (e.g., 752, 754).
[0072] Figures 8A-8C A schematic diagram of an exemplary layout constraint identifying a diffusion ratio of errors is shown. The layout system of the present disclosure can also be set up to decide how errors will be diffused on a connection based on the ratio. The diffusion ratio of errors can be redefined on a dual connection. In this regard, the spacing can still be active, as shown in Figs. 8A-8C. For example, instead of 50% on each link, the layout system can decide to locate 100% (e.g., 802, 804) or a different ratio such as 80% / 20% (e.g., 806, 808). The ratio can vary depending on the implementation. Figures 7A-8C The constraint shown in is sometimes referred to herein as a center constraint.
[0073] Dimension constraints
[0074] Fig. 9 Schematic diagram 900 of an exemplary layout constraint identifying constraint dimensions is shown. In addition to positioning, widget dimensions can also be constrained. The simplest approach is to allow the widget to stretch to reduce errors by the defined constraints. Fig. 9 , the schematic diagram shows how the width of the widget stretch (e.g., 906) adapts to the left and right constraints (e.g., 902, 904).
[0075] In a similar manner, the layout system of the present disclosure can define one dimension of a widget (width or height) in terms of another dimension, so ratios can be applied (e.g., enforcing the fact that a given widget must be square, or that an image is displayed in a 16:9 ratio).
[0076] Fig.10 Schematic diagram 1000 shows an example of dimension-dependent layout constraints. Fig.10 , widget dimensions are constrained by different types of constraints. In a first type of dimension-related constraints, a widget may be constrained by a spread constraint 1002, wherein the length of the widget is spread between fixed virtual guides. In a second type of dimension-related constraints, a widget may be constrained by a wrap constraint 1004, wherein the length of the widget is wrapped between fixed virtual guides with a defined distance between them. In a third type of dimension-related constraints, a widget may be constrained by a wrap constraint 1006, wherein the wrap is limited by the dimensional constraints of the widget. In a fourth type of dimension-related constraints, a widget may be constrained by a wrap constraint 1008, wherein the wrap consists of a minimum width for the widget being set. In a fifth type of dimension-related constraints, a widget may be constrained by a wrap constraint 1010, wherein the wrap consists of a maximum width for the widget being set.
[0077] Fig.11A and 11B shows an example of a chain-related placement constraint. Fig.11A , layout constraints 1100 include a chain-based relationship between widgets 1102 and 1104. Widgets 1102 and 1104 include defined distances 1108 and 1110, respectively, relative to side edges. The spatial relationship between widgets 1102 and 1104 is bidirectional due to connection 1106. In this regard, layout constraints can be applied to other widgets bound to the chain.
[0078] exist Fig. 11B, the positioning of widgets may be subject to different classes of chain-based constraints. In a first class of chain-based constraints, widgets may be subject to a diffusion chain 1152, wherein each widget along the chain is equally spaced within fixed limits including a fixed distance to the side edges. In a second class of chain-based constraints, widgets may be subject to a diffusion chain 1154, wherein each widget at the end of the chain abuts the side edge and is equally spaced from the middle widget. In a third class of chain-based constraints, widgets may be subject to a weighted chain 1156, wherein certain widgets are more heavily weighted than other widgets along the chain to occupy the entire length between the side edges. In a fourth class of chain-based constraints, widgets may be subject to a packing chain 1158, wherein widgets are packed together in series at equal distances on either end relative to the side edges. In a fifth category of chain-based constraints, widgets may be constrained by biased packing chains 1160, where widgets are packed together in series at varying distances on either end relative to a side edge, depending on the amount of bias applied to either end.
[0079] Finding the center constraint
[0080] In some aspects, centering a widget in a layout based on a constraint system works by setting two relative constraints of equal strength. For example, if a constraint system has three widgets A, B, and C aligned horizontally, and the intent is to have B be centered between A and C, the constraint system may set two constraints on B: (1) a left constraint to the right of A, and (2) a right constraint to the left of C. Since the constraints have equal strength, widget B will end up being centered. When rapidly prototyping a UI, setting these constraints one by one may be undesirably time consuming. More notably, requiring the user to provide linear equations to express such constraints unnecessarily burdens the user experience of developing the UI, and presents greater challenges in minimizing the amount of error that may appear in the linear equations received from user input.
[0081] The layout system of the present disclosure automates the centering of widgets. The layout system can achieve this by finding the appropriate constraints as described in the problem (using traditional layout systems) and setting them for the user. The algorithm applied in the layout system works as follows (vertical centering will work in a similar manner as described for horizontal centering):
[0082] a. The layout system can keep the best candidate for centering on each side (left and right).
[0083] b. The layout system can initialize the best candidates to the parent widget side: the left side of the parent is the left candidate, and the right side of the parent is the right candidate.
[0084] c. The layout system can then collect a list of widgets r that are siblings in the view hierarchy. These are the widgets that can be centered.
[0085] d. The layout system can iterate over the list of siblings, looking only at a particular side depending on the orientation of the center and the direction of the search. For example, if the layout system intends to horizontally center widget B, the layout system will perform two searches: one on the left and one on the right. When searching on the left, the layout system will only look to the right of the widget that the layout system examines, and similarly, when searching on the right, the layout system will only look to the left of the widget.
[0086] e. For each side of the widget that the layout system evaluates, the layout system can perform two steps: 1. If the side actually intersects with the other side. For example, for horizontal centering, the layout system can check that the two sides intersect on the vertical axis. 2. The layout system can keep the best candidate for connection. A better candidate is one that has a smaller distance to the other side than the current candidate.
[0087] f. Repeat the process on the other side of the widget. This approach allows the layout system to easily generate constraints that allow the layout system to center the widgets, thereby avoiding overlapping of widgets (an undesirable result).
[0088] Virtual Containers
[0089] Fig.12 A schematic diagram 1200 of an exemplary layout constraint identifying a virtual container is shown. The present disclosure provides a constrained layout system designed to easily represent arbitrary one-to-one relationships between widgets. A common problem is that sometimes a given layout is simpler to express as a group, rather than considering a single widget. For example, a user (e.g., a developer) may want to vertically center a group of widgets.
[0090] Most traditional layout systems (and / or engines) meet this requirement by introducing specific containers that lay out the widgets they contain in a given way. Fig.12In the layout system of the present disclosure, a virtual container (e.g., 1202) is introduced to represent a group of widgets. Constraints can then be set on the group itself rather than its widgets. Importantly, such virtual containers do not layout their child nodes, but instead keep layout calculations in the parent constraint system. Virtual containers provide developers and UI designers with a more flexible way to express their UI designs, allowing them to provide a hierarchy of widgets and subdivide containers into separate components. Since the virtual containers themselves do not require any calculations, the layout system maintains the performance advantage of a central location where the layout system can perform complete layout calculations. This also results in a shallower layout hierarchy, further enhancing the performance of the layout system. In traditional layout systems, layout containers are part of the view hierarchy and need to be traversed when drawing, which adversely affects performance.
[0091] The layout system of the present disclosure allows virtual containers (e.g., 1202) to express specific layout strategies. This allows the layout system to reach a higher level because the hierarchy allows for a more powerful way to express the layout. In contrast to traditional layout systems, virtual containers must only exist when constraints are established; they do not need to be in the final view hierarchy, which has performance advantages. For example, fewer hierarchies need to be traversed when drawing the screen, and in some ways, all constraints are collected in one place. This allows the layout system of the present disclosure to completely separate the concept of a view hierarchy (determining which widgets to display) and the concept of how to position widgets. This in turn provides the layout system with greater flexibility in how the layout system can change the way the layout system can position widgets, animate them, etc. at runtime. In this regard, the virtual container does not become part of the final layout and / or the DOM of the page.
[0092] exist Fig.12 In the present invention, the virtual container (e.g., 1202) can also provide additional assistance in visually creating a layout, such as (1) providing a virtual visual guide (e.g., 1204) for display that can be used to help create a layout, (2) tables, templates, or other higher-level structures are available to the user, and (3) the virtual assistant tool can be limited to the design tool.
[0093] Virtual Template
[0094] It may be necessary to position a set of widgets in a well-defined way, such as aligning all widgets one after another, or organizing a set of widgets into a table, or providing a specific full-screen layout that follows a given specification. In traditional layout systems, a separate layout component is responsible for this.
[0095] In some aspects, virtual templates are defined to determine how to position their child widgets. One difference between the layout system of the present disclosure and traditional layout systems is that virtual templates do not perform layout calculations themselves; instead, they simply use a common system of conceptual constraints defined in the layout system to express how their children are constrained to each other. The constraints set on the children are passed to the root constraint system container, which performs the actual calculations. This provides several advantages to the layout system of the present disclosure: (1) all calculations are kept in a single place, which has performance and flexibility benefits; (2) because such virtual templates directly use the constraint vocabulary defined in the layout system, they are generally easier to create than manually programmed templates; (3) external developers have the same access rights as platform developers and can develop these custom templates, thereby providing them with the same benefits of quickly creating their own custom templates. The ease of creating virtual templates via the layout system of the present disclosure can help developers create custom layouts more easily. Finally, this also significantly reduces the need for deep hierarchies in the UI because it conceptually separates the layout itself from the view system, which brings a separate set of performance and flexibility advantages. In this regard, the virtual template does not become part of the final layout and / or the DOM of the page.
[0096] Virtual Assistant Object
[0097] User interfaces can sometimes be described by developers (e.g., users of layout systems) using elements that do not exist in the actual view system. For example, wizards are often used to represent dimensions / positions.
[0098] The layout system of the present disclosure defines virtual helper objects that can be used to set constraints on widgets at creation time, but can be safely discarded during draw / layout time. These objects only need to exist at concept time, and can be deleted once the constraint system is created (using them). A canonical example is a virtual guide (e.g. Fig.12 1204) which allows the user to position widgets relative to horizontal and vertical guides. These virtual guides have their own positions set relative to their parent (either relative to a given side of their parent or relative to a percentage-based position). When the user loads a given constraint system that uses the guides, the guides allow the user to correctly generate the correct linear equations in the layout system. However, such virtual helpers do not need to be part of the view hierarchy at layout / draw time. In other words, the virtual helper objects do not become part of the final layout and / or the DOM of the page. Other such helper objects that do not need to be part of the final view hierarchy can be created to allow the user to express layouts more simply.
[0099] Renders the background of a virtual component
[0100] In a typical view hierarchy representing a user interface, there are various elements: (1) widgets required for interaction (displaying information or allowing actions), (2) intermediate layout components responsible for positioning the widgets, and (3) various decorative additional widgets used only for drawing decoration, such as background content, etc. This results in a view hierarchy that is more complex and deeper in the number of hierarchy layers than it would be from a purely functional perspective, and thus has poorer performance. Specifically, since these "decorative" widgets are only used as decoration and have no function, using widgets (complex objects) just for this task is theoretically inefficient.
[0101] As described above, the layout system of the present disclosure is based on constraints expressed as a system of linear equations. In the layout system of the present disclosure, a virtual container (e.g., Fig.12 1202), which largely eliminates the need for intermediate layout components other than the top layout component (where a full constraint system is parsed to position all widgets). The key to solving the problem of the need for decorative widgets in the hierarchy is to exploit the concept of the existence of virtual containers, thereby allowing users of the layout system to specify, for example, backgrounds (colors or images) that can be drawn on virtual containers. Although these virtual containers are not part of the view hierarchy, they are still accessible to the user in terms of the drawing of the view hierarchy. On the other hand, the top-level layout component that contains all widgets and virtual containers has the opportunity to draw first. By tracking the drawables specified in the virtual containers, the top-level layout component can perform a single drawing process before traversing the widgets it contains and drawing them. This single drawing process will be responsible for drawing all drawables defined in the virtual container. This approach allows users to create visually identical UIs, but with a more efficient way of drawing lists of these contents, as well as helping to reduce the complexity of the view hierarchy.
[0102] Custom widgets and templates
[0103] For users of traditional layout systems, creating custom widgets is often a complex task. Often, they are only interested in creating "composite" widgets, such as custom widgets that are purely defined using other existing widgets. A common problem in these traditional layout systems when customizing widgets is the performance and flexibility of the system. In the layout system of the present disclosure, such composite objects can be defined with little to no performance penalty.
[0104] The collapse of embedded restraint systems
[0105] Some traditional layout systems work by having a hierarchy of layout managers that contain the widgets to be laid out as child nodes. A layout manager can also be a child of another layout manager. This approach often leads to a deep view hierarchy, especially when complex UIs must be laid out and / or when nested simpler layout managers are used. The problem with deep hierarchies is that the hierarchy is not optimal in terms of performance. For example, each time a layout is required, the hierarchy needs to be traversed.
[0106] In some aspects, the layout system of the present disclosure centralizes all constraints in a single location to be resolved by a linear equation engine solver (e.g., 224). Traditional layout systems support virtual layout managers that add their constraints to the top-level parent constraint system. In a similar manner, the layout system of the present disclosure automatically embeds other complete constraint layout manager instances that are descendants of the parent layout manager, thereby moving the resolution to the root constraint manager. The benefit of doing so is that the resources used are reduced and the solving process is centralized, allowing the layout system to achieve better performance. In this regard, it may be cheap to broadcast new layout managers, especially if they are direct descendants. This also leads to the practice of flattening the hierarchy in the view system, which also helps to improve performance without sacrificing the expressibility brought by the layout hierarchy.
[0107] Insert and / or remove constrained UI elements
[0108] The layout of widgets in a running application often requires the ability to dynamically change the hierarchy, such as adding, removing, or hiding widgets. In a general-purpose constraint-based layout, these changes need to be handled correctly and efficiently. Removing and hiding widgets is critical in a general-purpose constraint-based layout engine.
[0109] In some aspects, the removal process associated with the layout system of the present disclosure is two-pronged: (1) the layout system will need to consider removing constraints on the widget differently, and (2) removing the widget itself. The layout system of the present disclosure utilizes two intrinsic constraints on the widget to specify its fixed position. When decomposing the constraints on the widget, the layout system can calculate the current (pre-decomposition) position of the widget and store it in the intrinsic constraints, and then safely delete the constraints, with the widget remaining in the same position as before, thereby providing a simple workflow for users of the layout system. The second part of the problem involves deleting the widget itself. The problem at this point is not the constraints set on the widget (through widget removal), but rather the constraints set on other widgets in the layout that point to the widget to be removed. The layout system of the present disclosure will need to iterate over those widgets and remove the associated constraints following rules similar to step #1 of constraint removal.
[0110] Fig.13A and 13B An example of a layout constraint indicating an object that disappears at runtime is shown. In one or more embodiments, the last behavior that needs to be specified by the layout system is what happens when a widget is marked as having disappeared - that is, it still exists but may not affect the layout (e.g., is not visible). In this regard, the layout system of the present disclosure passes the widget through the solver, but sets its size to zero. Another approach is to perform an optimization pass in which the layout system will follow the constraints set on the widget marked as disappearing and use the constraints set on it to replace the constraints that another widget has on the disappearing widget. For example, in Figure 13, there is provided a widget A (e.g., 1304) pointing to the left of widget B (e.g., 1302), where the position of widget B (e.g., 1302) is constrained by some other current constraints. If widget B (e.g., 1302) is marked as disappearing, the layout system can set the dimensions of widget B to zero (e.g., Fig. 13B 1306 in ), and everything will work fine; alternatively, the layout system can evaluate the constraints that set the position of the left side of widget B and replace the constraints of widget A with the evaluated constraints.
[0111] Fig.14 An example of an editor interface 1400 for an application 221 for developing and deploying user interfaces is shown. The editor interface 1400 includes a widget palette 1402, a layout editor window 1404, a layout inspector window 1406, and a user interface 1408. The widget palette 1402 may include a list of widget options for positioning within the layout editor window 1404. The layout editor window 1404 provides a canvas where a user can manipulate one or more locations of widgets, including adding and / or removing connections between positioned widgets. Some of these connections may include establishing connections between placed widgets via anchors of individual widgets. The layout inspector window 1406 provides an interface for obtaining additional information about a widget selected within the layout editor window 1404. The additional information may include a description of the widget, a text title of the widget, an identifier of the widget, or a specific size of the widget. The user interface 1408 provides a runtime representation of the user interface layout for display based on the layout constraints defined within the layout editor window 1404.
[0112] exist Fig.14In the example of , a user interface 1408 having various display objects such as an image 1410 and a text view 1412 can be analyzed (e.g., via UI reasoning engine 223) to determine constraints between the various display objects therein. As indicated by arrow 1416, a layout editor window 1404 is then provided to a developer (e.g., using UI construction engine 222) that includes the various constraints 1418 determined for an existing user interface (e.g., 1408). User interface code can also be generated for user interface 1408 that now includes the determined constraints. In this manner, new or updated code for user interface 1408 can be generated with improved functionality and additional improvements (e.g., due to a shallower hierarchy), and complexity can be added to user interface 1408 by adding, reducing, or otherwise modifying the determined constraints.
[0113] Hardware Overview
[0114] Fig.15 It shows that it can be achieved Figure 1 1 is a block diagram of an exemplary computer system 1500 of client 110 and server 130. In some aspects, computer system 1500 can be implemented in hardware or a combination of software and hardware, or integrated in another entity, or distributed among multiple entities, in a dedicated server.
[0115] The computer system 1500 (e.g., client 110 and server 130) includes a bus 1508 or other communication mechanism for communicating information, and a processor 1502 (e.g., processors 212 and 236) coupled to the bus 1508 for processing information. For example, the computer system 1500 may be implemented using one or more processors 1502. The processor 1502 may be a general purpose microprocessor, a microcontroller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a state machine, gate logic, discrete hardware components, or any other suitable entity that can perform computations or other manipulations of information.
[0116] In addition to hardware, the computer system 1500 may include code that creates an execution environment for the computer programs described, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them, stored in an included memory 1504 (e.g., memories 220 and 232), such as a random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable PROM (EPROM), registers, a hard disk, a removable disk, a CD-ROM, a DVD, or any other suitable storage device, which is coupled to the bus 1508 to store information and instructions to be executed by the processor 1502. The processor 1502 and the memory 1504 may be supplemented by, or incorporated in, special purpose logic circuitry.
[0117] Instructions may be stored in memory 1504 and implemented in one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by computer system 1500 or for controlling the operation of computer system 1500, and according to any method known to those skilled in the art, including but not limited to computer languages, such as data-oriented languages (e.g., SQL, dBase), system languages (e.g., C, Objective-C, C++, Assembly), framework languages (e.g., Java, .NET), and application languages (e.g., PHP, Ruby, Perl, Python). The instructions may also be implemented in computer languages such as array languages, aspect-oriented languages, assembly languages, editing languages, command line interface languages, compiled languages, concurrent languages, curly bracket languages, data flow languages, data structure languages, declarative languages, esoteric languages, diffuse languages, fourth generation languages, functional languages, interactive mode languages, interpreted languages, iterative languages, list-based languages, niche languages, logic-based languages, machine languages, macro languages, metaprogramming languages, multi-paradigm programming languages, numerical analysis, non-English-based languages, object-oriented class-based languages, object-oriented prototype-based languages, offside rule languages, procedural languages, reflective languages, rule-based languages, scripting languages, stack-based languages, synchronous languages, syntax processing languages, visual languages, wirth languages, and xml-based languages. The memory 1504 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 1502.
[0118] Computer programs as discussed herein do not necessarily correspond to files in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordination files (e.g., files storing portions of one or more modules, subroutines, or codes). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed on multiple sites and interconnected by a communication network. The processes and logic flows described in this specification may be executed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output.
[0119] The computer system 1500 further includes a data storage device 1506, such as a disk or optical disk, coupled to the bus 1508 for storing information and instructions. The computer system 1500 can be coupled to various devices via an input / output module 1510. The input / output module 1510 can be any input / output module. An exemplary input / output module 1510 includes a data port such as a USB port. The input / output module 1510 is configured to connect to a communication module 1512. An exemplary communication module 1512 (e.g., communication modules 218 and 238) includes a network interface card such as an Ethernet card and a modem. In some aspects, the input / output module 1510 is configured to connect to multiple devices, such as an input device 1514 (e.g., input device 216) and / or an output device 1516 (e.g., output device 214). An exemplary input device 1514 includes a keyboard and a pointing device, such as a mouse or a trackball, through which a user can provide input to the computer system 1500. Other types of input devices 1514 may be used to provide interaction with the user, such as a tactile input device, a visual input device, an audio input device, or a brain-computer interface device. For example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, tactile, or brain wave input. Exemplary output devices 1516 include display devices, such as LCD (liquid crystal display) monitors, for displaying information to the user.
[0120] According to one aspect of the present disclosure, the client 110 and the server 130 can be implemented using the computer system 1500 in response to the processor 1502 executing one or more sequences of one or more instructions contained in the memory 1504. Such instructions can be read into the memory 1504 from another machine-readable medium such as a data storage device 1506. The execution of the sequence of instructions contained in the main memory 1504 causes the processor 1502 to perform the process steps described herein. One or more processors in a multi-processing arrangement can also be employed to execute the sequence of instructions contained in the memory 1504. In alternative aspects, hard-wired circuits can be used instead of or in combination with software instructions to implement various aspects of the present disclosure. Therefore, aspects of the present disclosure are not limited to any specific combination of hardware circuits and software.
[0121] Various aspects of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, such as a data server; or includes a middleware component, such as an application server; or includes a front-end component, such as a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described in this specification; or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by digital data communication (e.g., a communication network) in any form or medium. The communication network (e.g., network 150) can include, for example, any one or more of a LAN, a WAN, the Internet, etc. In addition, the communication network can include, but is not limited to, for example, any one or more of the following network topologies, including a bus network, a star network, a ring network, a mesh network, a star bus network, a tree or hierarchical network, etc. The communication module can be, for example, a modem or an Ethernet card.
[0122] Computer system 1500 can include client and server.Client and server are usually far away from each other and typically interact through communication network.The relationship of client and server is produced by means of computer program running on respective computers and having client-server relationship each other.Computer system 1500 can be, for example, but not limited to desktop computer, laptop computer or tablet computer.Computer system 1500 can also be embedded in another device, for example, but not limited to mobile phone, PDA, mobile audio player, global positioning system (GPS) receiver, video game console and / or TV set-top box.
[0123] As used herein, the term "machine-readable storage medium" or "computer-readable medium" refers to any medium or media that participates in providing instructions to processor 1502 for execution. Such media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical or magnetic disks, such as data storage device 1506. Volatile media include dynamic memory, such as memory 1504. Transmission media include coaxial cables, copper wires, and optical fibers, including wires that comprise bus 1508. Common forms of machine-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punch cards, paper tapes, any other physical media with hole patterns, RAM, PROMs, EPROMs, flash EPROMs, any other memory chips or cartridges, or any other media from which a computer can read. A machine-readable storage medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a material component that implements a machine-readable propagation signal, or a combination of one or more of them.
[0124] In the following, various examples of aspects of the present disclosure are described. These are provided as examples, and do not limit the subject technology.
[0125] According to aspects of the present disclosure, a computer-implemented method is provided. The method includes providing a layout application for developing a user interface (UI) on a computing device, wherein the layout application includes a representation of a page. The method includes receiving a first user input via the layout application, wherein the first user input indicates an instruction to position a first UI element on the page. The method includes receiving a second user input via the layout application, wherein the second user input indicates an instruction to position a second UI element on the page. The method includes receiving a third user input via the layout application, wherein the third user input indicates an instruction to create a connection from the second UI element to the first UI element. The method includes generating a layout constraint indicating a spatial relationship between the first UI element and the second UI element based on the connection. The method also includes providing the layout constraint in a layout data file associated with the UI.
[0126] In generating the layout constraints, the method also includes mapping the connection to spatial positioning data and relationship data. In one or more aspects, the spatial positioning data indicates the position on the page relative to the first UI element and the second UI element. In one or more aspects, the relationship data indicates the directional relationship between the first UI element and the second UI element relative to one of the first UI element or the second UI element. The method also includes determining the coordinates on the page for the first UI element and the second UI element, respectively, based on the spatial positioning data and the relationship data. The method also includes providing the first UI element and the second UI element for display at the determined coordinates.
[0127] In determining the coordinates, the method also includes processing the spatial positioning data and the relationship data. The method also includes determining whether the relationship data and the spatial positioning data are ambiguous based on the processing. In one or more aspects, the relationship data and the spatial positioning data are ambiguous when more than one potential coordinate set is calculated for at least one of the first UI element or the second UI element. The method also includes not relying on a linear equation solver to parse the relationship data and the spatial positioning data to determine the coordinates when it is determined that the relationship data and the spatial positioning data are not ambiguous. The method also includes transforming the relationship data and the spatial positioning data into one or more linear equations when it is determined that the relationship data and the spatial positioning data are ambiguous. The method also includes solving the one or more linear equations by the linear equation solver to determine the coordinates when the relationship data and the spatial positioning data are transformed into one or more linear equations.
[0128] In determining the coordinates, the method also includes processing the spatial positioning data and the relationship data. The method also includes classifying the relationship data and the spatial positioning data into a specific class of constraints. The method also includes determining whether the specific class of constraints is associated with a first class of constraints or a second class of constraints that is different from the first class of constraints. The method also includes not relying on a linear equation solver to parse the relationship data and the spatial positioning data to determine the coordinates when it is determined that the specific class of constraints is associated with the first class of constraints. The method also includes transforming the relationship data and the spatial positioning data into one or more linear equations when it is determined that the specific class of constraints is associated with the second class of constraints. The method also includes solving the one or more linear equations by the linear equation solver to determine the coordinates when the relationship data and the spatial positioning data are transformed into one or more linear equations.
[0129] In one or more aspects, the first UI element includes a first anchor and the second UI element includes a second anchor. Based on the instruction to create a connection from the second UI element to the first UI element, the second anchor of the second UI element connects to the first anchor of the first UI element.
[0130] In one or more aspects, the instructions to create a connection from the second UI element to the first UI element indicate that the first anchor and the second anchor occupy the same target location on the page.
[0131] In one or more aspects, the instructions to create a connection from the second UI element to the first UI element indicate a defined distance between the first anchor and the second anchor.
[0132] In generating the layout constraints, the method further includes determining that the connection includes an error. In one or more aspects, the error indication limits the first anchor and the second anchor to be positioned at a target location on the page. The method also includes modifying one or more dimensions of at least one of the first UI element or the second UI element. The method also includes reducing an amount of error to a predetermined value based on the modified one or more dimensions.
[0133] The method also includes receiving a fourth user input indicating an instruction to position a third UI element on the page. The method also includes receiving a fifth user input indicating an instruction to create a first connection from the third UI element to the first UI element and a second connection from the third UI element to the second UI element. The method also includes determining that the position of the third UI element relative to the first UI element and the second UI element includes an error. The method also includes modifying the ratio of the amount of error of the first connection to the amount of error of the second connection. The method also includes adjusting the amount of error of the first connection and the amount of error of the second connection based on the modified ratio. The method also includes determining the coordinates of the third UI element based on the adjusted amount of error of the first connection and the adjusted amount of error of the second connection.
[0134] In one or more aspects, the layout constraints include a first spacing associated with the first connection from the third UI element to the first UI element and a second spacing associated with the second connection from the third UI element to the second UI element. In one or more aspects, the first spacing indicates a distance between the first UI element and the third UI element, and the second spacing indicates a distance between the second UI element and the third UI element.
[0135] The method also includes providing a virtual object for display on the page, the virtual object being a visual guide on the page. The method also includes receiving user input indicating an instruction to position one or more of the first UI element or the second UI element relative to the virtual object, the spatial relationship between the first UI element and the second UI element being based on the virtual object. The method also includes rendering a final layout of the page based on the layout data file. In one or more aspects, during the runtime of the final layout, the virtual object is removed from the display on the page.
[0136] According to aspects of the present invention, a system is provided, comprising one or more processors; and a computer-readable storage medium coupled to the one or more processors, the computer-readable storage medium comprising instructions. The instructions cause the one or more processors to provide a layout application for developing a UI on a computing device, wherein the layout application includes a representation of a page. The instructions also cause the one or more processors to receive a first user input via the layout application, wherein the first user input indicates an instruction to position a first UI element on the page. The instructions also cause the one or more processors to receive a second user input via the layout application, wherein the second user input indicates an instruction to position a second UI element on the page. The instructions also cause the one or more processors to receive a third user input via the layout application, wherein the third user input indicates an instruction to create a connection from the second UI element to the first UI element. The instructions also cause the one or more processors to generate a layout constraint indicating a spatial relationship between the first UI element and the second UI element based on the connection. The instructions also cause the one or more processors to provide the layout constraint in a layout data file associated with the UI.
[0137] In one or more aspects, the user input indicates a defined distance between at least two of the two or more UI elements.
[0138] The instructions further cause one or more processors to map one or more connections to spatial positioning data and relationship data. In one or more aspects, the spatial positioning data indicates a position on the page relative to the first UI element and the second UI element. In one or more aspects, the relationship data indicates a directional relationship between the first UI element and the second UI element relative to one of the first UI element or the second UI element. The instructions further cause the one or more processors to determine the coordinates on the page for the first UI element and the second UI element, respectively, based on the spatial positioning data and the relationship data. The instructions further cause the one or more processors to provide the first UI element and the second UI element for display at the determined coordinates.
[0139] The instructions further cause one or more processors to process the spatial positioning data and the relationship data. The instructions further cause one or more processors to determine whether the relationship data and the spatial positioning data are ambiguous based on the processing. In one or more aspects, the relationship data and the spatial positioning data are ambiguous when more than one potential coordinate set is calculated for at least one of the first UI element or the second UI element. The instructions further cause one or more processors to not rely on a linear equation solver to parse the relationship data and the spatial positioning data to determine the coordinates when it is determined that the relationship data and the spatial positioning data are not ambiguous. The instructions further cause one or more processors to transform the relationship data and the spatial positioning data into one or more linear equations when it is determined that the relationship data and the spatial positioning data are ambiguous. The instructions further cause one or more processors to solve the one or more linear equations by the linear equation solver to determine the coordinates when the relationship data and the spatial positioning data are transformed into one or more linear equations.
[0140] The instructions further cause one or more processors to process the spatial positioning data and the relationship data. The instructions further cause one or more processors to classify the relationship data and the spatial positioning data into a specific class of constraints. The instructions further cause one or more processors to determine whether the specific class of constraints is associated with a first class of constraints or a second class of constraints that is different from the first class of constraints. The instructions further cause one or more processors to resolve the relationship data and the spatial positioning data without relying on a linear equation solver to determine the coordinates when determining that the specific class of constraints is associated with the first class of constraints. The instructions further cause one or more processors to convert the relationship data and the spatial positioning data into one or more linear equations when determining that the specific class of constraints is associated with the second class of constraints. The instructions further cause one or more processors to solve the one or more linear equations by the linear equation solver to determine the coordinates when the relationship data and the spatial positioning data are converted into one or more linear equations.
[0141] The instructions further cause the one or more processors to provide a virtual object for display on the page. In one or more aspects, the virtual object is a visual guide on the page. The instructions further cause the one or more processors to receive user input indicating an instruction to position at least one of the two or more UI elements relative to the virtual object. In one or more aspects, the spatial relationship between the two or more UI elements is based on the virtual object. The instructions further cause the one or more processors to render a final layout of the page based on the layout data file. In one or more aspects, when the final layout is run, the virtual object is removed from the display on the page.
[0142] According to aspects of the present disclosure, a non-transitory computer-readable storage medium is provided, comprising instructions, which, when executed by a processor, cause the processor to perform a method. The method includes providing a layout application for developing a UI on a computing device, wherein the layout application includes a representation of a page. The method includes receiving a first user input via the layout application, wherein the first user input indicates an instruction to position a first UI element on the page. The method includes receiving a second user input via the layout application, wherein the second user input indicates an instruction to position a second UI element on the page. The method includes receiving a third user input via the layout application, wherein the third user input indicates an instruction to create a connection from the second UI element to the first UI element. The method includes generating a layout constraint indicating a spatial relationship between the first UI element and the second UI element based on the connection. The method also includes providing the layout constraint in a layout data file associated with the UI.
[0143] The instructions further cause the one or more processors to map each of the one or more connections to a linear equation representing a spatial relationship between the first UI element and the second UI element. The instructions further cause the one or more processors to solve the linear equations to produce a solution with a minimum amount of error in the one or more connections. The instructions further cause the one or more processors to determine coordinates on the page for the first UI element and the second UI element, respectively, based on the solved linear equations. The instructions further cause the one or more processors to provide the determined coordinates to the layout data file.
[0144] The instructions further cause the one or more processors to provide a virtual object for display on the page, the virtual object being a visual guide on the page. The instructions further cause the one or more processors to receive user input indicating an instruction to position one or more of the first UI element or the second UI element relative to the virtual object. In one or more aspects, the spatial relationship between the first UI element and the second UI element is based on the virtual object. The instructions further cause the one or more processors to render a final layout of the page based on the layout data file. In one or more aspects, the instructions further cause the one or more processors to remove the virtual object from display on the page when the final layout is run.
[0145] According to aspects of the present disclosure, a system is provided, comprising a device for storing instructions, and a device for executing the stored instructions, wherein the instructions, when executed by the device, cause the device to perform a method. The method includes providing a layout application for developing a UI on a computing device, wherein the layout application includes a representation of a page. The method includes receiving a first user input via the layout application, wherein the first user input indicates an instruction to position a first UI element on the page. The method includes receiving a second user input via the layout application, wherein the second user input indicates an instruction to position a second UI element on the page. The method includes receiving a third user input via the layout application, wherein the third user input indicates an instruction to create a connection from the second UI element to the first UI element. The method includes generating a layout constraint indicating a spatial relationship between the first UI element and the second UI element based on the connection. The method also includes providing the layout constraint in a layout data file associated with the UI.
[0146] In generating the layout constraints, the method also includes mapping the connection to spatial positioning data and relationship data. In one or more aspects, the spatial positioning data indicates the position on the page relative to the first UI element and the second UI element. In one or more aspects, the relationship data indicates the directional relationship between the first UI element and the second UI element relative to one of the first UI element or the second UI element. The method also includes determining the coordinates on the page for the first UI element and the second UI element, respectively, based on the spatial positioning data and the relationship data. The method also includes providing the first UI element and the second UI element for display at the determined coordinates.
[0147] In determining the coordinates, the method also includes processing the spatial positioning data and the relationship data. The method also includes determining whether the relationship data and the spatial positioning data are ambiguous based on the processing. In one or more aspects, the relationship data and the spatial positioning data are ambiguous when more than one potential coordinate set is calculated for at least one of the first UI element or the second UI element. The method also includes not relying on a linear equation solver to parse the relationship data and the spatial positioning data to determine the coordinates when it is determined that the relationship data and the spatial positioning data are not ambiguous. The method also includes transforming the relationship data and the spatial positioning data into one or more linear equations when it is determined that the relationship data and the spatial positioning data are ambiguous. The method also includes solving the one or more linear equations by the linear equation solver to determine the coordinates when the relationship data and the spatial positioning data are transformed into one or more linear equations.
[0148] In determining the coordinates, the method also includes processing the spatial positioning data and the relationship data. The method also includes classifying the relationship data and the spatial positioning data into a specific class of constraints. The method also includes determining whether the specific class of constraints is associated with a first class of constraints or a second class of constraints that is different from the first class of constraints. The method also includes not relying on a linear equation solver to parse the relationship data and the spatial positioning data to determine the coordinates when it is determined that the specific class of constraints is associated with the first class of constraints. The method also includes converting the relationship data and the spatial positioning data into one or more linear equations when it is determined that the specific class of constraints is associated with the second class of constraints. The method also includes solving the one or more linear equations by the linear equation solver to determine the coordinates when the relationship data and the spatial positioning data are converted into one or more linear equations.
[0149] In one or more aspects, the first UI element includes a first anchor and the second UI element includes a second anchor. Based on the instruction to create a connection from the second UI element to the first UI element, the second anchor of the second UI element connects to the first anchor of the first UI element.
[0150] In one or more aspects, the instructions to create a connection from the second UI element to the first UI element indicate that the first anchor and the second anchor occupy the same target location on the page.
[0151] In one or more aspects, the instructions to create a connection from the second UI element to the first UI element indicate a defined distance between the first anchor and the second anchor.
[0152] In generating the layout constraints, the method further includes determining that the connection includes an error. In one or more aspects, the error indication limits the first anchor and the second anchor to be positioned at a target location on the page. The method also includes modifying one or more dimensions of at least one of the first UI element or the second UI element. The method also includes reducing an amount of error to a predetermined value based on the modified one or more dimensions.
[0153] The method also includes receiving a fourth user input indicating an instruction to position a third UI element on the page. The method also includes receiving a fifth user input indicating an instruction to create a first connection from the third UI element to the first UI element and a second connection from the third UI element to the second UI element. The method also includes determining that the position of the third UI element relative to the first UI element and the second UI element includes an error. The method also includes modifying the ratio of the amount of error of the first connection to the amount of error of the second connection. The method also includes adjusting the amount of error of the first connection and the amount of error of the second connection based on the modified ratio. The method also includes determining the coordinates of the third UI element based on the adjusted amount of error of the first connection and the adjusted amount of error of the second connection.
[0154] In one or more aspects, the layout constraints include a first spacing associated with the first connection from the third UI element to the first UI element and a second spacing associated with the second connection from the third UI element to the second UI element. In one or more aspects, the first spacing indicates a distance between the first UI element and the third UI element, and the second spacing indicates a distance between the second UI element and the third UI element.
[0155] The method also includes providing a virtual object for display on the page, the virtual object being a visual guide on the page. The method also includes receiving user input indicating an instruction to position one or more of the first UI element or the second UI element relative to the virtual object, the spatial relationship between the first UI element and the second UI element being based on the virtual object. The method also includes rendering a final layout of the page based on the layout data file. In one or more aspects, during the runtime of the final layout, the virtual object is removed from the display on the page.
[0156] According to aspects of the present disclosure, a computer-implemented method is provided. The method includes providing a layout application for developing a UI on a computing device, wherein the layout application includes a representation of a page. The method includes receiving user input via the layout application, wherein the user input indicates the placement of two or more user interface elements on the page. The method includes detecting one or more connections between the two or more user interface elements. The method includes determining a layout constraint based on the one or more connections, wherein the layout constraint indicates a spatial relationship between the two or more UI elements. The method also includes updating a layout data file associated with the UI to include the layout constraint.
[0157] According to aspects of the present disclosure, a computer-implemented method is provided. The method includes providing a layout application for developing a UI on a computing device, wherein the layout application includes a representation of a page. The method includes detecting one or more connections between a first UI element and a second UI element located on the page, wherein each of the one or more connections indicates a layout constraint of the first UI element and the second UI element, and wherein the layout constraint indicates a spatial relationship between two or more UI elements. The method includes determining whether the one or more connections detected include an error. The method includes adjusting the layout constraint so that the error is distributed in the one or more connections when the one or more connections detected include an error. The method includes providing the layout constraint to a layout data file having a constraint list. The method also includes using the layout data file to generate a final layout of the UI.
[0158] In one or more aspects, examples of clauses are described below.
[0159] A method comprising one or more methods, operations or a portion thereof described herein.
[0160] An apparatus includes one or more memories and one or more processors (eg, 110), the one or more processors being configured to cause one or more methods, operations, or portions thereof described herein to be performed.
[0161] A device, comprising one or more memories (e.g., 220, one or more internal, external or remote memories, or one or more registers) and one or more processors (e.g., 212) coupled to the one or more memories, the one or more processors being configured to cause the device to perform one or more methods, operations, or a portion thereof described herein.
[0162] An apparatus includes means (eg, 110) adapted to perform one or more methods, operations, or portions thereof described herein.
[0163] A processor (eg, 212) includes modules for performing one or more methods, operations, or portions thereof described herein.
[0164] A hardware device includes a circuit (eg, 110) configured to perform one or more methods, operations, or a portion thereof described herein.
[0165] An apparatus includes means (eg, 110) adapted to perform one or more methods, operations, or portions thereof described herein.
[0166] An apparatus includes components (eg, 110) operable to perform one or more methods, operations, or portions thereof described herein.
[0167] A computer-readable storage medium (e.g., 220, one or more internal, external, or remote memories, or one or more registers) including instructions stored therein, the instructions including code for performing one or more methods or operations described herein.
[0168] A computer-readable storage medium (e.g., 220, one or more internal, external or remote memories, or one or more registers) storing instructions that, when executed by one or more processors, cause the one or more processors to perform one or more methods, operations, or portions thereof described herein.
[0169] In one aspect, a method may be an operation, an instruction, or a function, and vice versa. In one aspect, a clause or claim may be modified to include some or all of the words (e.g., instructions, operations, functions, or components) cited in other clause(s), words(s), sentences(s), phrase(s), paragraph(s), and / or claims(s).
[0170] To illustrate the interchangeability of hardware and software, items such as various exemplary blocks, modules, components, methods, operations, instructions, and algorithms have generally been described in terms of their functionality. Whether such functionality is implemented as hardware or software or a combination of hardware and software depends on the specific application and design constraints imposed on the overall system. A skilled person may implement the described functionality in different ways for each specific application.
[0171] Unless explicitly stated, reference to an element in the singular does not mean one and only one, but rather one or more. For example, "a" module may refer to one or more modules. An element prefixed with "a", "an", "the" or "said" does not exclude the presence of additional identical elements without further limitations.
[0172] Headings and subheadings, if any, are used for convenience only and do not limit the invention. The word "exemplary" is used to mean used as an example or illustration. To the extent that the terms "including," "having," and the like are used, such terms are intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when used as a transitional word in the claims. Relational terms such as "first" and "second," etc. may be used to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
[0173] Phrases such as "on the one hand", "this aspect", "on the other hand", "some aspects", "one or more aspects", "an embodiment", "this embodiment", "another embodiment", "some embodiments", "one or more embodiments", "an example", "this example", "another example", "some examples", "one or more examples", "a configuration", "this configuration", "another configuration", "some configurations", "one or more configurations", "the subject technology", "the disclosure", "the present disclosure", and other variations thereof are for convenience and do not mean that the disclosure related to these phrases is essential to the subject technology or that such disclosure applies to all configurations of the subject technology. The disclosure related to these phrases may apply to all configurations or one or more configurations. The disclosure related to these phrases may provide one or more examples. Phrases such as "aspect" or "some aspects" may refer to one or more aspects and vice versa, and this applies similarly to the other aforementioned phrases.
[0174] The phrase "at least one of" preceding a series of items (where the terms "and" or "or" are used to separate any of the items) modifies the list as a whole rather than each member of the list. The phrase "at least one of" does not require selection of at least one of the items; rather, the phrase allows for a meaning that includes at least one of any one item and / or at least one of any combination of items and / or at least one of each item. As an example, each of the phrases "at least one of A, B, and C" or "at least one of A, B, or C" means only A, only B, or only C; any combination of A, B, and C; and / or at least one of each of A, B, and C.
[0175] It should be understood that the specific order or level of steps, operations or processes disclosed is a diagram of an exemplary method. Unless otherwise explicitly stated, it should be understood that the specific order or level of steps, operations or processes can be performed in different orders. Some of the steps, operations or processes can be performed simultaneously. The attached method claims (if any) present the elements of each step, operation or process in an exemplary order, and are not meant to be limited to the specific order or level presented. These can be performed in serial, linear, parallel or different orders. It should be understood that the instructions, operations and systems described can usually be integrated together in a single software / hardware product or packaged into multiple software / hardware products.
[0176] The present disclosure is provided to enable any person skilled in the art to implement various aspects described herein. In some cases, well-known structures and components are shown in block diagram form to avoid blurring the concept of the subject technology. The present disclosure provides various examples of the subject technology, and the subject technology is not limited to these examples. Various modifications to these aspects will be apparent to those skilled in the art, and the principles described herein may be applied to other aspects.
[0177] All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly stated in the claims. No claim element will be interpreted under 35 U.S.C. §112, sixth paragraph, unless the element is explicitly set forth using the phrase "means for..." or, in the case of a method claim, the element is set forth using the phrase "step for..."
[0178] The title, background, brief description of the drawings, abstract, and drawings are hereby incorporated into the present disclosure and are provided as illustrative examples of the present disclosure, not as limiting descriptions. It is submitted with the understanding that they will not be used to limit the scope or meaning of the claims. In addition, in the detailed description, it can be seen that for the purpose of simplifying the present disclosure, the description provides illustrative examples and various features are combined together in various embodiments. The disclosed method should not be interpreted as reflecting the intention that the claimed subject matter requires more features than those expressly stated in each claim. On the contrary, as reflected in the claims, the inventive subject matter lies in less than all the features of a single disclosed configuration or operation. The claims are thereby incorporated into the detailed description, wherein each claim itself serves as a separately claimed subject matter.
[0179] The claims are not intended to be limited to the aspects described herein, but should be given the full scope consistent with the language of the claims and encompass all legal equivalents. Nevertheless, these claims are not intended to encompass subject matter that does not satisfy the requirements of applicable patent law and should not be interpreted in such a manner.
Claims
1. A method comprising: Determining, by the computing device, that a plurality of connections relate to a plurality of user interface (U) elements, wherein the plurality of connections correspond to a plurality of layout constraints, and wherein each layout constraint of the plurality of layout constraints defines a spatial relationship of at least two UI elements of the plurality of UI elements; Determining, by the computing device, that the plurality of connections result in positioning errors of the plurality of UI elements; determining, by the computing device, a ratio of an amount of error to be propagated into a first layout constraint among the plurality of layout constraints to an amount of error to be propagated into a second layout constraint among the plurality of layout constraints; and At least two of the plurality of placement constraints are adjusted, by the computing device, to diffuse the error between at least two of the plurality of connections, wherein the error diffuses according to the determined ratio.
2. The method of claim 1, wherein: Determining that the plurality of connections involve the plurality of UI elements comprises: providing, on the computing device, a layout application for developing a UI, the layout application comprising a representation of a page that allows UI elements to be visually positioned relative to each other; and User input is received via the layout application, the user input indicating that the plurality of connections relate to the plurality of UI elements.
3. The method of claim 2, further comprising: After adjusting the at least two layout constraints, the plurality of layout constraints are provided in a layout data file associated with the UI. 4 . The method of claim 1 , further comprising modifying one or more dimensions of one or more UI elements of the plurality of UI elements to facilitate spreading the error between at least two connections of the plurality of connections.
5. The method of claim 1, wherein: Determining the ratio is based on user input.
6. The method of claim 1, wherein: Determining the ratio is based on user preferences stored in a memory of the computing device.
7. The method of claim 1, wherein: Determining the ratio includes the computing device modifying a predetermined ratio value to a custom ratio value.
8. The method of claim 1, further comprising determining coordinates of one or more UI elements of the plurality of UI elements based on a ratio of an amount of error that will propagate to the first of the plurality of connections to an amount of error that will propagate to the second of the plurality of connections. 9 . The method of claim 1 , further comprising generating a final layout of the UI based on the at least two adjusted layout constraints.
10. The method of claim 1, wherein: Adjusting at least two of the plurality of placement constraints to cause the error diffusion to occur between at least two of the plurality of connections is performed using a linear equation solver.
11. The method of claim 10, wherein: The linear equation solver is applied in response to determining that more than one potential set of coordinates can be calculated for at least one UI element of the plurality of UI elements.
12. The method of claim 10, wherein: The linear equation solver is applied to a linear equation representing two relatively connected anchors on one of the plurality of UI elements.
13. The method of claim 12, wherein: The linear equation solver spreads the error equally between the two relatively connected anchors.
14. The method of claim 12, wherein: The linear equation solver diffuses errors such that more error is applied to a first one of the two relatively connected anchors than to a second one of the two relatively connected anchors.
15. A non-transitory computer-readable storage medium comprising instructions that, when executed by at least one processor, cause the at least one processor to: Determining multiple connections involves multiple user interface UI elements, where: The plurality of connections corresponds to a plurality of layout constraints, and wherein each layout constraint of the plurality of layout constraints defines a spatial relationship of at least two UI elements of the plurality of UI elements; Determining that the plurality of connections cause positioning errors of the plurality of UI elements; determining a ratio of an amount of error to be propagated into a first one of the plurality of layout constraints to an amount of error to be propagated into a second one of the plurality of layout constraints; and At least two of the plurality of placement constraints are adjusted to diffuse the error between at least two of the plurality of connections, wherein the error is diffused according to the determined ratio.
16. A system comprising: one or more processors; A computer-readable storage medium coupled to the one or more processors, the computer-readable storage medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to: determining that a plurality of connections involve a plurality of user interface (U) elements, wherein the plurality of connections correspond to a plurality of layout constraints, and wherein each layout constraint of the plurality of layout constraints defines a spatial relationship of at least two UI elements of the plurality of UI elements; Determining that the plurality of connections cause positioning errors of the plurality of UI elements; determining a ratio of an amount of error to be propagated into a first one of the plurality of layout constraints to an amount of error to be propagated into a second one of the plurality of layout constraints; and At least two of the plurality of placement constraints are adjusted to diffuse the error between at least two of the plurality of connections, wherein the error is diffused according to the determined ratio.
17. The system of claim 16, wherein: The instructions cause the one or more processors to determine that the plurality of connections involve the plurality of user interface (UI) elements by: providing a layout application for developing a UI, the layout application comprising a representation of a page that allows UI elements to be visually positioned relative to each other; and User input is received via the layout application, the user input indicating that the plurality of connections relate to the plurality of UI elements.
18. The system of claim 17, wherein: The instructions cause the one or more processors to provide a visual indication of at least two adjusted layout constraints in the layout application.
19. The system of claim 16, wherein: The instructions cause the one or more processors to generate a final layout of the UI based on the at least two adjusted layout constraints.
Citation Information
Patent Citations
Layout decision method, apparatus, and program
US20060174194A1
Method for generating presentation oriented XML schemas through a graphical user interface
US20080120316A1
Display and Resolution of Incompatible Layout Constraints
US20120124492A1
Method of solving page layout constraints
US20130194297A1
