Systems and methods for automated transformation of interactive sites and applications to support mobile and other display environments - Patents.com

Abstract

To provide a converter capable of being run via a client / server system including at least one processor which processes instructions defining the converter.SOLUTION: In a system 100, a converter 200 includes a reception unit which receives a website page from a website construction system on a server 150. The website page has a source layout for a source display region and an object model representation of components. The converter also includes a processor which determines an order and a set of relationships between the components, and forms an objective layout of the website page for an objective display region. The objective layout is based on the order in which the relationship has been determined and semantic analysis and arbitration of the set, and attributes of the source display region differ from attributes of the objective display region.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the transformation of interactive applications, and in particular to mobile platforms. [Background technology]

[0002] With the advent of the Internet, in recent years, users have increasingly been using smartphones, tablets, and other mobile devices to access websites and interactive applications. These devices are gradually replacing or complementing both full-featured desktop personal computers and older, less capable "feature phones." This applies not only to websites that exist on the World Wide Web, but also to other interactive applications that are available today from the many application stores offered by major companies such as Apple, Google, Microsoft, and Amazon.

[0003] The display form factors and features of websites and interactive applications vary depending on where they are viewed (eg, on a desktop PC, a small mobile device, and a mid-sized mobile tablet). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent Application Serial No. 14 / 207,761 [Patent Document 2] U.S. Patent No. 7,203,901 [Patent Document 3] US Patent Application Publication No. 2013 / 0219263 Summary of the Invention [Problem to be solved by the invention]

[0005] According to a preferred embodiment of the present invention, there is provided a converter executable via a client / server system having at least one processor for processing instructions defining a converter. The converter has a receiver for receiving a website page from a website building system, the page having a source layout for a source display area, the page having an object model representation of components. The converter also has a processor for determining an order and set of relationships between the components and for forming a destination layout of the website page for a destination display area, the destination layout being based on a semantic analysis and reconciliation of the determined order and set of relationships, and attributes of the source display area being different from attributes of the destination display area.

[0006] Furthermore, in accordance with a preferred embodiment of the present invention, the converter includes a re-transformation converter that combines modifications to the source layout and the destination layout, taking into account modifications made independently to the destination layout, to form an updated destination layout configuration.

[0007] Further in accordance with a preferred embodiment of the present invention, the processor comprises a pre-processor for analyzing object properties of the components and modifying their suitability for presentation on the destination display area, a supernode former for locating groups of the components that should remain together and forming a hierarchy of supernodes based on the locations of the components and content relationships of the components, an orderer for determining an order of elements within each of the supernodes, a positioner for positioning the elements within the destination layout based on at least one of the determined order and the groups, and a post-processor for adjusting the positioned elements prior to display.

[0008] Furthermore, in accordance with a preferred embodiment of the present invention, the component is at least one of an atomic component, a single-page container component, and a multi-page container component.

[0009] Further in accordance with a preferred embodiment of the present invention, the converter is executable on at least one of a client, a server, and a third party server.

[0010] Furthermore, in accordance with a preferred embodiment of the present invention, the pre-processor includes at least one of a template manipulation unit that modifies instances of the components within a template, a component filter unit that filters out components that are not suitable for the target layout, a component adaptation unit that adapts the components to the target layout, a component adjustment unit that adjusts the components to fit the target layout, and a component analysis unit that analyzes the attributes of the components to determine the appropriateness of usage of the components.

[0011] Further, according to a preferred embodiment of the present invention, the supernode former comprises an overlap group locator for locating the group of components when the components overlap significantly and replacing the group with a virtual element according to a grouping criterion, and a predefined group locator for locating the group of components according to at least one hint at a template, application, page or component level and replacing the group with a virtual element according to a grouping criterion. The supernode former also comprises a text over image group locator for locating the component when the component is a text component overlaid on a specific background image and replacing the group with a virtual element according to a grouping criterion, a scorer for determining a certainty score for the accuracy of the overlap group locator, the predefined group locator and the text over image group locator, and a node former for forming a hierarchy of supernodes based on the components and the virtual elements.

[0012] Further, according to a preferred embodiment of the present invention, the ordering unit includes a basic ordering unit that determines an order of the elements of the supernode according to predefined ordering criteria, a partial ordered set locator that analyzes at least one of semantics, content, attributes, edit history, and geometry of the elements of the supernode, a scorer that determines a confidence score for the accuracy of the basic ordering unit and the partial ordered set locator, and an order integration unit that integrates the order determined by the basic ordering unit with the detected partial ordered sets located by the partial ordered set locator to form a combined modified order.Further, according to a preferred embodiment of the present invention, the post-processor includes an auto-add component insertion unit that inserts at least one of an auto-add component and a mobile-related component, and a dynamic layout adjustment unit that at least one of forming, modifying, and removing dynamic layout anchors, the dynamic layout anchors being adjusted according to the target layout. Further, in accordance with a preferred embodiment of the present invention, the basic ordering unit includes at least one of: a primary direction ordering unit that orders the elements of the supernode according to at least one of rows and columns; a primary direction with split and merge orderer that orders the elements of the supernode according to at least one of rows and columns and tracks any row / column splits and merges; and a horizontal / vertical slicer that slices the elements of the supernode alternately horizontally and vertically to form an internal tree of divisions and define the order of display of the elements.

[0013] Further, in accordance with a preferred embodiment of the present invention, the positioning unit includes a hint interpretation unit that interprets hints attached to the elements of the supernode, a line break formation unit that forms line breaks in accordance with the hints formed by the node formation unit and the ordering unit, a size adjustment unit that applies at least one of width and height adjustments to the elements of the supernode, and a resizing unit that resizes decorative images.

[0014] Furthermore, in accordance with a preferred embodiment of the present invention, the partially ordered set locator unit includes: a cluster partial ordered set locator unit for detecting cluster partial ordered sets when the elements of the supernode are closer together compared to a normal spacing between the elements within the supernode; a semantic partial ordered set locator unit for detecting semantic relationship partial ordered sets when a certain combination of the elements of the supernode of a predetermined type that are close to each other exists; a pattern partial ordered set locator unit for detecting partial ordered sets when a set pattern exists between the elements of the supernode; a predefined partial ordered set locator unit for detecting partial ordered sets based on at least one of existing edit related grouping definitions, existing dynamic layout anchor information, and reuse of component templates for different component sets; and an edit session information based partial ordered set locator unit for detecting partial ordered sets of the elements of the supernode based on information collected from previous edit sessions.

[0015] Additionally in accordance with a preferred embodiment of the present invention, at least one of the auto-add component and the mobile-related component includes at least one of a widget, a navigation menu, advertisements, and promotional materials specific to the target device.

[0016] Further, according to a preferred embodiment of the present invention, the converter includes a pattern partially ordered set locator, and the horizontal / vertical slicer includes an element splitter for determining a slicing direction of the elements of the supernode based on at least one of a number of dividers, a size of a gap found in a given projection direction, the located pattern partially ordered set, and an alignment quality to an axial projection in both horizontal and vertical directions.Further, according to a preferred embodiment of the present invention, the component filter includes a removal unit for removing components that are not suitable for display in the target layout, and a concealment unit for hiding components that are not suitable for display in the target layout.

[0017] Further, in accordance with a preferred embodiment of the present invention, the component adaptation unit includes a modification unit that modifies the size and width of the components, an integration unit that integrates menu components, a content update unit that updates content-related adaptations, a creation unit that forms composite menus, and a character converter that converts character-based graphics.

[0018] Further in accordance with a preferred embodiment of the present invention, the component adjustment unit comprises a mapping unit for forming a font size mapping between the source layout and the destination layout.

[0019] Further in accordance with a preferred embodiment of the present invention, the component analyzer includes an image stitcher for stitching the components into a single image if the components are image components, and an ornamental image manipulation unit for distinguishing ornamental images.

[0020] Furthermore, according to a preferred embodiment of the present invention, the reconversion converter comprises a page deletion unit which deletes a page from the destination layout when the page is deleted from the source layout, a page addition unit which adds the page to the destination layout when the page is added to the source layout, a component deletion unit which deletes a component from the destination layout when the component is deleted from the source layout, a component addition unit which adds the component to the destination layout when the component is added to the source layout, a component modification unit which modifies the component in the destination layout when the component is modified in the source layout, and a mobile operation unit which manipulates the modifications to the destination layout when the modifications to the destination layout are independent of the modifications to the source layout.

[0021] Further, according to a preferred embodiment of the present invention, the component adding unit includes a parent / ancestor search unit that searches for at least one of a closest ancestor component and a parent component that is closest to the component added to the source layout, and a mobile layout adding unit that inserts the added component into the destination layout according to the position of at least one of the closest ancestor component and the parent component.

[0022] According to a preferred embodiment of the present invention, the method includes the steps of receiving a website page from a website building system, the website page having a source layout for a source display area, the website page having an object model representation of components, and determining an order and set of relationships between the components and forming a destination layout for a destination display area of ​​the website page, the destination layout being based on a semantic analysis and reconciliation of the determined order and set of relationships, and attributes of the source display area being different from attributes of the destination display area.

[0023] Furthermore, in accordance with a preferred embodiment of the present invention, the method also includes the step of combining modifications to the source layout and the destination layout, taking into account modifications made independently to the destination layout, to form an updated destination layout configuration.

[0024] Further in accordance with a preferred embodiment of the present invention, the determining and forming steps include the steps of: analyzing object properties of the components and modifying their suitability for presentation on the target display area; locating groups of the components that should remain together and forming a hierarchy of supernodes based on the locations of the components and their content relationships. The determining and forming steps also include the steps of determining an order of elements within each of the supernodes, positioning the elements in the target layout based on at least one of the determined order and the groups, and adjusting the positioned elements prior to display.

[0025] Furthermore, in accordance with a preferred embodiment of the present invention, the component is at least one of an atomic component, a single-page container component, and a multi-page container component.

[0026] Further in accordance with a preferred embodiment of the present invention, the analyzing and modifying step includes at least one of the steps of modifying instances of the components in a template, filtering the components that are not suitable for the target layout, adapting the components to the target layout, adjusting the components to the target layout, and analyzing the attributes of the components to determine appropriateness of usage of the components.

[0027] Further, according to a preferred embodiment of the present invention, the locating and forming step includes the steps of locating a group of components when the components overlap significantly and replacing the group with a virtual element according to a grouping criterion; locating the group of components according to at least one hint at a template, application, page or component level and replacing the group with a virtual element according to a grouping criterion; locating a component when the component is a text component overlaid on a specific background image and replacing the group with a virtual element according to a grouping criterion; determining a certainty score for the accuracy of the steps of locating the group of components when the components overlap significantly, locating the group of components according to at least one hint at a template, application, page or component level and locating the component when the component is a text component overlaid on a specific background image; and forming a hierarchy of supernodes based on the components and the virtual elements.

[0028] Further, according to a preferred embodiment of the present invention, the step of determining the order includes the steps of determining the order of the elements of the supernode according to a predefined order criterion, analyzing at least one of the meaning, content, attributes, editing history, and geometric shape of the elements of the supernode, determining the order of the elements of the supernode according to a predefined order criterion, as well as determining a confidence score for the correctness of the step of analyzing at least one of the meaning, content, attributes, editing history, and geometric shape of the elements of the supernode, and integrating the orders determined by the steps of determining the order of the elements of the supernode according to a predefined order criterion, as well as determining a confidence score for the correctness of the step of analyzing at least one of the meaning, content, attributes, editing history, and geometric shape of the elements of the supernode, and forming a combined modified order.

[0029] Further in accordance with a preferred embodiment of the present invention, the adjusting step includes inserting auto-add components and at least one of creating, modifying, and removing dynamic layout anchors, the dynamic layout anchors being adjusted in accordance with the target layout.

[0030] Furthermore, in accordance with a preferred embodiment of the present invention, the step of determining the order of the elements of the supernode according to a predefined ordering criterion includes at least one of the steps of ordering the elements of the supernode according to at least one of rows and columns, ordering the elements of the supernode according to at least one of rows and columns and tracking any row / column splits and joins, and slicing the elements of the supernode alternately horizontally and vertically to form an internal tree of partitions and define an order of display of the elements.

[0031] Further, in accordance with a preferred embodiment of the present invention, the positioning step includes the steps of interpreting hints attached to the elements of the supernode, forming line breaks according to the hints formed by the steps of identifying positions of groups of the components that should remain together and forming a hierarchy of supernodes based on the positions of the components and the content relationships of the components, applying at least one of width and height adjustments to the elements of the supernode, and resizing decorative images.

[0032] Further, according to a preferred embodiment of the present invention, the step of determining the order of the elements of the supernode according to a predefined ordering criterion includes the steps of: detecting cluster partially ordered sets where the elements of the supernode are closer to each other compared to a normal spacing between the elements of the supernode; detecting semantic relationship partially ordered sets when there is a particular combination of the elements of the supernode of a predetermined type that are close to each other; detecting partially ordered sets when a set pattern exists between the elements of the supernode; detecting partially ordered sets based on at least one of grouping definitions related to existing edits, existing dynamic layout anchor information, and reuse of component templates for different component sets; and detecting partially ordered sets of the elements of the supernode based on information collected from previous editing sessions.

[0033] Additionally in accordance with a preferred embodiment of the present invention, at least one of the auto-add component and the mobile-related component includes at least one of a widget, a navigation menu, advertisements, and promotional materials specific to the target device.

[0034] Further, according to a preferred embodiment of the present invention, the steps of detecting a partially ordered set when a set pattern exists between the elements of the supernode, slicing the elements of the supernode alternately horizontally and vertically to form an internal tree of a partitioned partition, and defining the order of display of the elements include a step of determining a slicing direction of the elements of the supernode based on at least one of the number of partitions and the size of gaps found in a predetermined projection direction, the step of detecting a partially ordered set when a set pattern exists between the elements of the supernode, and alignment quality to the axial projection in both horizontal / vertical directions.

[0035] Furthermore, in accordance with a preferred embodiment of the present invention, the filtering step includes the steps of removing components that are not suitable for display in the target layout, and hiding components that are not suitable for display in the target layout.

[0036] Further in accordance with a preferred embodiment of the present invention, the adapting step includes the steps of modifying the size and width of the components, integrating menu components, updating content related adaptations, forming composite menus, and converting character-based graphics.

[0037] Further in accordance with a preferred embodiment of the present invention the adjusting step includes font size mapping between the source layout and the destination layout.

[0038] Further in accordance with a preferred embodiment of the present invention, the analyzing step includes stitching the components into a single image when the components are image components, and distinguishing decorative images.

[0039] Further in accordance with a preferred embodiment of the present invention, the reconverting step includes the steps of: deleting a page from the destination layout when a page is deleted from the source layout; adding a page to the destination layout when a page is added to the source layout; deleting a component from the destination layout when a component is deleted from the source layout; inserting a component into the destination layout when a component is added to the source layout; modifying a component in the destination layout when a component is modified in the source layout; and manipulating the modifications to the destination layout when the modifications to the destination layout are independent of modifications to the source layout.

[0040] Further, in accordance with a preferred embodiment of the present invention, the step of adding the component includes the steps of searching for at least one of a closest ancestor component and a parent component that is closest to the component added to the source layout, and inserting the added component into the destination layout according to a position of at least one of the closest ancestor component and the parent component.

[0041] The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of this specification, however, the invention, both as to its objects, features and advantages, together with the organization and method of operation, may best be understood by reference to the following detailed description when read in conjunction with the accompanying drawings. [Brief description of the drawings]

[0042] [Figure 1] 1 is a schematic diagram of a system for cross-platform translation of visual applications, constructed and operative in accordance with the present invention; [Diagram 2]2 is a schematic diagram of elements of an initial layout converter, constructed and operative in accordance with the present invention; [Diagram 3] FIG. 1 is a schematic diagram of dynamic layout anchors collapsing when a component is moved. [Figure 4] FIG. 13 is a schematic diagram showing why vertical lines are inappropriate for mobile displays. [Diagram 5] 1 is a schematic diagram of the elements of a supernode formation portion constructed and operative in accordance with the present invention; [Figure 6] FIG. 13 is a schematic diagram of text reflow necessitated by font size and field size changes. [Figure 7] This is a photo that does not maintain its aspect ratio when resized, resulting in an incorrect appearance. [Figure 8A] FIG. 2 is a schematic diagram illustrating a mapping between a visual application and a corresponding supernode structure implemented in accordance with the present invention; [Figure 8B] FIG. 2 is a schematic diagram illustrating a mapping between a visual application and a corresponding supernode structure implemented in accordance with the present invention; [Figure 8C] FIG. 2 is a schematic diagram illustrating a mapping between a visual application and a corresponding supernode structure implemented in accordance with the present invention; [Figure 9] FIG. 1 is a schematic diagram of components within a container before rearrangement. [Figure 10] FIG. 10 is a schematic diagram of the components of FIG. 9 after rearrangement. [Figure 11] 2 is a schematic diagram of elements of an ordering unit constructed and operative in accordance with the present invention; [Figure 12] FIG. 1 is a schematic diagram of four text paragraphs with multiple possible reading orders. [Figure 13] FIG. 1 is a schematic diagram of an arrangement including two text paragraphs and two photographs with multiple reading orders. [Figure 14A] FIG. 2 is a schematic diagram illustrating the transformation of a set of elements in a supernode into an element graph performed in accordance with the present invention; [Figure 14B]FIG. 2 is a schematic diagram illustrating the transformation of a set of elements in a supernode into an element graph performed in accordance with the present invention; [Figure 14C] FIG. 2 is a schematic diagram illustrating the ordering of elements of an element set within a supernode, as implemented in accordance with the present invention; [Figure 15A] 4 is an algorithm illustrating the functioning of a primary directional orderer with disjunction and join, constructed and operative in accordance with the present invention; [Figure 15B] 4 is an algorithm illustrating the functioning of a primary directional orderer with disjunction and join, constructed and operative in accordance with the present invention; [Figure 15C] 4 is an algorithm illustrating the functioning of a primary directional orderer with disjunction and join, constructed and operative in accordance with the present invention; [Figure 16A] FIG. 16 is a schematic diagram of the process of the primary direction ordering with separation and combination of FIGS. 15A-15C, constructed and operative in accordance with the present invention; [Figure 16B] FIG. 16 is a schematic diagram of the process of the primary direction ordering with separation and combination of FIGS. 15A-15C, constructed and operative in accordance with the present invention; [Figure 16C] FIG. 16 is a schematic diagram of the process of the primary direction ordering with separation and combination of FIGS. 15A-15C, constructed and operative in accordance with the present invention; [Figure 16D] FIG. 16 is a schematic diagram of the process of the primary direction ordering with separation and combination of FIGS. 15A-15C, constructed and operative in accordance with the present invention; [Figure 16E] FIG. 16 is a schematic diagram of the process of the primary direction ordering with separation and combination of FIGS. 15A-15C, constructed and operative in accordance with the present invention; [Figure 16F] FIG. 16 is a schematic diagram of the process of the primary direction ordering with separation and combination of FIGS. 15A-15C, constructed and operative in accordance with the present invention; [Figure 16G] FIG. 16 is a schematic diagram of the process of the primary direction ordering with separation and combination of FIGS. 15A-15C, constructed and operative in accordance with the present invention; [Figure 16H]FIG. 16 is a schematic diagram of the process of the primary direction ordering with separation and combination of FIGS. 15A-15C, constructed and operative in accordance with the present invention; [Figure 17] 1 is an example of a website layout. [Figure 18] 2 is a schematic diagram of horizontal and vertical partitions of a supernode, constructed and operative in accordance with the present invention; [Figure 19] 1 is an example of an evaluation of vertical and horizontal partitions based on previous pattern similarity analysis, constructed and operative in accordance with the present invention; [Figure 20] FIG. 11 is a schematic diagram illustrating how vertical split partitions of a supernode are preferable when components are better aligned along horizontal lines, constructed and operative in accordance with the present invention; [Figure 21] 1 is a schematic diagram of an interlocking element configuration. FIG. [Figure 22] FIG. 13 is a schematic diagram of component pairs with their positions swapped. [Figure 23] FIG. 2 is a schematic diagram showing how four photo components can be split into pairs in two ways. [Figure 24] FIG. 13 is a schematic diagram of a non-rectangular display shape remaining after engraving an auto-add component. [Diagram 25] 1 is a schematic diagram of elements of a reconversion converter constructed and operative in accordance with the present invention; [Figure 26] 2 is a schematic diagram of elements of a component addition section constructed and operative in accordance with the present invention; [Figure 27] FIG. 26 is a functional schematic diagram of the reconversion converter of FIG. 25, constructed and operative in accordance with the present invention. [Figure 28A] 1 is a schematic diagram of component positioning during conversion or reconversion of a web page from a desktop layout configuration to a mobile layout configuration, constructed and operative in accordance with the present invention; [Figure 28B] 1 is a schematic diagram of component positioning during conversion or reconversion of a web page from a desktop layout configuration to a mobile layout configuration, constructed and operative in accordance with the present invention; [Figure 28C] 1 is a schematic diagram of component positioning during conversion or reconversion of a web page from a desktop layout configuration to a mobile layout configuration, constructed and operative in accordance with the present invention; [Figure 28D] 1 is a schematic diagram of component positioning during conversion or reconversion of a web page from a desktop layout configuration to a mobile layout configuration, constructed and operative in accordance with the present invention; [Figure 28E] 1 is a schematic diagram of component positioning during conversion or reconversion of a web page from a desktop layout configuration to a mobile layout configuration, constructed and operative in accordance with the present invention; [Figure 28F] 1 is a schematic diagram of component positioning during conversion or reconversion of a web page from a desktop layout configuration to a mobile layout configuration, constructed and operative in accordance with the present invention; [Figure 28G] 1 is a schematic diagram of component positioning during conversion or reconversion of a web page from a desktop layout configuration to a mobile layout configuration, constructed and operative in accordance with the present invention; [Figure 29] FIG. 2 is a schematic diagram illustrating a method for inserting a component into a mobile version of a web page. [Diagram 30] FIG. 1 is a schematic diagram illustrating the removal of a component from a mobile version of a web page. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] It will be appreciated that for simplicity and clarity of illustration, elements in the figures have not necessarily been drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.

[0044] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.

[0045] Applicant has observed that existing sites and applications are typically designed for desktop PCs, which have the large, high-resolution displays that are very common. When such sites and applications are accessed using smaller displays, many problems become apparent, including the need for extra scrolling (especially horizontal scrolling), confusing navigation, font sizes that are not adapted for mobile, design and display components that are not adapted to the touch screen of a mobile device, etc.

[0046] Because these sites may be viewed on different sized platforms, such as tablets and mobile phones, developers may be required to create multiple versions of the site or application to support these multiple usage scenarios and maintain the look and feel of the original site. The multiple versions may differ in multiple ways, including visual layout, display components (widgets) when used, touchscreen orientation, etc.

[0047] Applicant has also observed that existing systems for transforming website views to conform to an associated viewing platform have generally been aimed at solving the above problem to transform websites designed using markup languages ​​such as HTML (HyperText Markup Language) and XML (Extensible Markup Language). These websites may be sites directly designed in a given markup language or sites created using code generators that create markup language files for subsequent hosting.

[0048] Existing systems also focus on analyzing original websites, extracting content information from the original websites, and creating modified mobile sites. Such technologies generally originate from Optical Character Recognition (OCR) and page analysis systems, which are focused on content extraction rather than content and design adaptation. Thus, existing systems generally operate to extract content and make the content readable on small devices, while destroying much of the design, look, and feel of the original site.

[0049] It will be appreciated that analyzing an existing website can be complex. In particular, some parts of the site may be dynamically generated, and the relationships between the elements of the site may also be implemented procedurally. Thus, it is difficult to understand the site accurately unless the transformation is performed while the site is running (e.g., via a headless browser running on a transformation server). However, such latter methods can be quite burdensome, in particular requiring that a given page be adapted to a mobile size each time it is retrieved by each user, rather than reusing the transformation results once for all users.

[0050] Another problem with existing systems is supporting changes and edits to the original website during transformation. It will be appreciated that several systems have been developed that attempt to compare a recent version of a page with an earlier version of the same page and identify differences. This comparison is done using comparison rules based on text similarity, location in the source file, font usage, etc. This often requires designers to manually create bindings between elements in the mobile version and the original desktop version.

[0051] However, because page edits are made separately from the conversion process, the site may change rapidly between the two versions. For example, a new developer (with a completely different style) may have started developing the site. Or new technologies or libraries may have been incorporated into the site. Existing systems allow the two versions of the site to be compared, but the two versions may be very different from each other (internally) even if they look the same. Comparing two versions of the same website can be done heuristically, but will not be able to match heavily modified site elements.

[0052] Applicant has realized that the above limitations may be overcome by using an object-oriented visual design system. Such a system may be used to create interactive applications and web sites, and may also be used to maintain and create corresponding layout definitions across various display platforms for a given application. The visual design system may also provide an internal, consistent object data model of the site with precise specification attributes of all objects of the site. Thus, the visual design system may provide multiple visual layouts for a single site or application having a single set of components.

[0053] Applicant has also realized that using such a visual design system may include collected editing session history information that may be used to detect component changes and component relationships. Thus, for example, if a designer duplicates a pair of objects (e.g., a photo and a description) multiple times, it may be inferred that such a pair is related and that the members of the pair should remain close together when transformed for mobile. Thus, for example, container objects may be transformed while maintaining their original object hierarchy and object relationships. Other information may include objects that have been edited or moved together, objects that were previously grouped, objects or object sets formed by duplicating existing objects or object sets, and the timing of object editing (i.e., were there certain changes that were applied to a given subset of objects in sequence).

[0054] A visual design system may typically have a unique identifier (ID) for each object. Such IDs may be used to match different site versions and keep site changes consistent. Additionally, a visual design system may utilize dynamic layouts that include anchors between displayed components that control layout changes made to the displayed components. Such anchors may be used to obtain additional grouping and layout information and to fine-tune the layout for various mobile display sizes. Indeed, the translation subsystem may automatically generate additional dynamic layout anchors for later use during display.

[0055] Furthermore, with the proliferation of smart phones, full-scale advanced HTML browsing (browsing similar to that available on the desktop) is becoming common in the mobile environment. Therefore, a system is needed that can use similar operations for all platforms (desktop, tablet, and mobile) to take advantage of the advanced browsers available on all platforms.

[0056] A visual application may be a stand-alone system, such as the PowerPoint presentation program available from Microsoft Corporation, or may be embedded within a larger editing system, such as the Microsoft Word AutoShape Editor, also available from Microsoft Corporation. Such applications are typically composed of multiple pages and components, which may be further arranged within a hierarchy of containers (single-page and multi-page) within a page that contains atomic components. A multi-page container may also display multiple mini-pages.

[0057] Pages may also include list applications (such as those discussed in U.S. Patent Application Publication No. 2014 / 0133634, entitled "WEBSITE BUILDING SYSTEM INTEGRATING DATA LISTS WITH DYNAMIC CUSTOMIZATION AND ADAPTATION," filed March 13, 2014, and assigned to the assignee of the present invention and a common assignee) and third party applications. Pages may also use templates, such as general page templates or component templates. In certain cases, an application master page is used that contains components that are replicated in all other regular pages. The arrangement of components within a page or set of pages is known as a layout. The following description should be understood to describe layouts consisting of rectangular, axis-aligned components and containers. Other layouts may also include non-rectangular components and containers, particularly including rotated or skewed objects, and object classes such as objects consisting of multiple regions, such as a video playback component that may have an animation display region and a video control region. Such regions may be joined, non-intersecting, or intersecting. Other layouts may also include objects of any geometric shape.

[0058] It will be appreciated that such non-rectangular multi-region objects may be manipulated by using enclosing rectangles for each of the non-rectangular objects or by fitting geometric primitives for manipulating the extended non-rectangular object. Such fitting may include projection onto axes, distance between shapes; minimum / maximum directional distance between shapes (i.e. vertical or horizontal distance), finding shape intersections; finding shape / line intersections; calculating intersection areas and calculating area of ​​shapes.

[0059] It will be further appreciated that each application may have multiple layout configurations, such as desktop (portrait and landscape) and mobile. Some properties of the website components may also have values ​​per layout configuration, such as the containment of the actual component (e.g., a particular component may be included only in mobile or only in desktop / tablet). Such components may be container components, and all contained sibling components are affected together. Other properties may include size (h,w), position (x,y), z-order information, style (font, size, color), color, polymorphic component type (e.g., a given gallery may use different gallery types for desktop list components and mobile list components), menu / gallery settings (e.g., grid gallery # of rows / columns), third party application variants (for third party applications that have multiple variants of different display sizes), and list application view association (for a given item type or items associated with a view, different views corresponding to different list components can be specified).

[0060] It will also be understood that each layout configuration may have a default screen size value (pixels). Actual screen sizes vary to some extent, and such differences are handled by the dynamic layout support of the associated visual design system. The desktop layout configuration may be the main layout configuration, since it has the largest screen width. Other layout configurations may be considered shadow layout configurations of the main layout configuration, and may have different screen widths and heights than the main layout configuration. In particular, the width of the shadow layout configuration may be smaller (e.g., display on a mobile phone), equal (e.g., tablet), or larger (e.g., larger display screen) than the width of the main layout configuration.

[0061] Furthermore, it will be appreciated that when converting an application page or a website page from a desktop version to a mobile version, there are two conflicting objectives: maintaining the readability of the content on the scaled-down page and maintaining the visual layout of the page. Most of the existing systems aim to extract content at the expense of design (see, for example, U.S. Patent No. 5,399,363, entitled "SMALL FORM FACTOR WEB BROWSING"). These systems are derived from published OCR / page analysis algorithms that extract text from the target area and do not attempt to maintain (or put minimal effort into maintaining) the existing visual website design.

[0062] Some algorithms attempt to preserve some design and / or layout by shrinking the entire page or elements of the page, but such shrinking is very limited, especially since text content becomes rapidly illegible as font size is reduced.

[0063] Reference is now made to Figure 1. Figure 1 illustrates a system 100 for cross-platform visual application translation according to an embodiment of the present invention.

[0064] The system 100 may include an initial layout converter 200, a layout reconversion converter 300, and a database 50. The initial layout converter 200 may further include a page receiver 210 and a processor 220. The converter 200 may first convert a main layout configuration from a desktop to a shadow layout configuration for mobile. The reconversion converter 300 may reconvert the layout configuration after editing the applications of the main layout configuration. All the applications and configurations may be stored on the database 50.

[0065] The system 100 may be installed as part of a website building system on a server 150 that can communicate with various clients 5 representing various platforms such as desktops 5A, smartphones 5B, and third party system interfaces (inter-program communication) 5C via a suitable communication medium such as the Internet. The server 150 may store various layout configurations corresponding to each platform (including multiple layout configurations corresponding to a single platform). In alternative embodiments, the system 100 may also run locally on the client or be adapted to provide services to other systems via a suitable API (Application Programming Interface).

[0066] It will be appreciated that the initial layout converter 200 may convert each page in the application into a sufficiently narrow version so that the page may be viewed on a narrow screen with vertical scrolling and minimal (if any) horizontal scrolling. It will be appreciated that if the system 100 is properly integrated (e.g., running on an associated client of a website building system), the system 100 may perform the conversion in a responsive mode in which a new layout is calculated each time the screen dimensions change.

[0067] Reference is now made to FIG. 2, which illustrates the elements of the processor 220. The processor 220 comprises a pre-processor 201, a supernode formation unit 230, an ordering unit 240, a positioning unit 270, and a post-processor 202. The pre-processor 201 may comprise a template operation unit 206, a component filter unit 221, a component matcher 225, a component adjustment unit 227, and a component analyzer 229. The post-processor 202 may comprise an automatic additional component insertion unit 281 and a dynamic layout adjustment unit 284. It will be understood that the formed target layout configuration may be displayed by a display unit 290 via an associated website building system. It will be understood that if the system 100 is fully integrated with an associated website building system and directly modifies the data structure of the website building system, the post-processor 202 may also perform a post-processing step to restore the data structure, thereby adapting the modified website building system data to be displayed by the display unit 290.

[0068] It will be appreciated that the initial layout converter 200 may operate on each page of an application or website individually. In normal mode, the initial layout converter 200 may convert each page into a narrow enough, usually elongated, version so that it can be viewed with vertical scrolling and minimal horizontal scrolling on a narrower mobile screen. It will further be appreciated that some destination devices, such as large tablet and outdoor display screens, may have wider screens than the original desktop screen, and thus may not require narrowing, but rather may need to fit the larger display width.

[0069] The initial layout converter 200 may process a set of components in three dimensions (x and y coordinates of the screen and z order of the display) and convert the set of components into a mathematically ordered set. The total order generated may represent the order in which the information is read by a user on the associated page. In this case, the initial layout converter 200 may display the ordered components in a narrow mobile view.

[0070] The initial layout converter 200 may process the pages to be converted and divide the relevant pages into supernodes based on an analysis of their components and their content relationships, as described in more detail herein below.

[0071] The preprocessor 201 may analyze the components of the relevant page for suitability for the layout of the target configuration and may modify existing components if necessary. Various combinations of layouts may be constructed, but the following example illustrates the conversion from a desktop layout configuration to a mobile layout configuration. The template manipulation unit 206 may create modified instances of elements in the template if an instance of the element exists. The component filter unit 221 may filter components according to their suitability for the mobile layout configuration, the component adaptor 225 may adapt the components specifically for the mobile layout configuration, the component adjuster 227 may adapt the components specifically for the mobile layout configuration, and the component analyzer 229 may analyze the components (including their content, geometry, and other attributes) to determine their actual usage. For example, a photo component may be a background image or a content image, which may affect the manipulation of the photo component by other parts of the system 100.

[0072] The supernode former 230 may locate groups of components that should stay together (e.g., components that overlap significantly). The supernode former 230 may further form a hierarchy of supernodes based on the current page and elements in the container hierarchy (possibly modifying the container hierarchy as described further below). The ordering unit 240 may determine an order for the elements of each supernode. The positioning unit 270 may position the elements of the node in the mobile layout region based on the determined order, and the post-processor 202 may make any final adjustments to the layout as necessary. The auto-add component inserter 281 may insert any auto-add components (described in further detail herein below) and other mobile-related components (e.g., mobile-specific menus) that are inserted after ordering, and the dynamic layout adjuster 284 may modify any existing anchors as necessary after positioning. The display unit 290 may display the new layout on the mobile platform, and may make final adjustments to the specific mobile platform as described in further detail herein below.

[0073] The page receiver 210 may receive a web page having a desktop layout configuration to be converted and may send the page to the page preprocessor 201 .

[0074] As described above in this specification, there may be a container hierarchy that may provide further information about the logical arrangement of pages. Such a hierarchy may include normal containment relationships and parallel containment relationships, such as multiple parallel minipages contained within a multi-page container. Preprocessor 201 may analyze such a hierarchy and any particular container relationships at the beginning of the process to gather information that may be useful to the process. Preprocessor 201 may also determine the screen size of the target application and extract navigation menus ("site-global and page-specific") and combine them to form one or more integrated page menus, such as top-down menus.

[0075] The associated visual design system may support templates, including multi-level and multiple template inheritance. Such templates may include single-page, multiple-page, or page elements. The template handler 206 may create modified instances of the template's elements, so that the template instance may consist of template components with local modifications applied. Additionally, the template handler 206 may make specific modifications necessary to make the template instance conform to a mobile layout configuration. For example, if inherited components are close enough to the top or bottom of a page, they may be appropriately included in a page-specific header or footer. Inherited components may be logically replicated (with modifications) in each page that inherits them.

[0076] The associated visual design system may keep a single copy of each inherited component, but allow per-page layouts to be applied to those copies of components (since mobile adaptations may differ for each page). The template manipulation unit 206 may use information about the original template of a given set of components and form template-based component groups to ensure a better match to the original intent of the template. For each page of the associated website individually, the preprocessor 201 may review the elements, taking into account parameters such as the target screen dimensions. The component filter unit 221 may remove or hide components that are not suitable for mobile display. As an example, such components may include vertical separator lines (as opposed to horizontal lines), Adobe Flash content (which cannot be played on some mobile systems), or decorative frames. Components that are completely unsuitable for mobile display are removed before further processing. For components that are not recommended for mobile display, the component filter unit 221 may choose to hide them rather than remove them completely. In this case, the website designer can reinsert those components into the mobile layout configuration when they are visible and reinsertable by the mobile modification display GUI, as described in more detail herein below. Some components may be inappropriate or not recommended for the mobile platform, but may still provide substantial information to the initial layout converter 200.

[0077] Reference is now made to FIG. 4, which illustrates vertical lines E between components on a desktop version. For example, vertical lines between components may be fairly useless on a mobile layout configuration as shown, since they may typically translate into long (empty) spaces between sets of components. As shown, when components A, B, C, and D are rearranged and stacked vertically for a narrow mobile display, vertical lines E may waste display space and should be removed. However, the vertical lines are still very useful in indicating that the four components A, B, C, and D should be arranged as A+B and C+D, and not as A+C and B+D. To take advantage of such information, the component filter unit 221 may mark the vertical lines as “remove completely” or “hide behind”, but still add invisible, widthless lines that provide the necessary allocation information to the initial layout converter 200.

[0078] The component adaptor 225 may employ width reduction measures similar to those employed by the positioner 270, as described in more detail herein below (whenever such measures can be made operational in this pre-processing stage). For example, the component adaptor 225 may switch components to "lighter" mobile versions; one example being from a gallery that may display multiple minipages (e.g., accordion-style multiple minipages) to a gallery that displays the contained minipages one at a time. The component adaptor 225 may also provide specific "lightweight" versions of given components for use in the mobile version of the authored application (via alternate versions or different views customized for mobile display).

[0079] The component adaptor 225 can also handle content-related adaptations, such as converting character-based graphics to vector-based scalable graphics. Designers may use text characters as decorations or separators, such as " / / / / / / / / / / / ", "----------------", "-=-=-=-=-=-=". Once converted to similar vector-based shapes, those graphics can be accurately resized without the need for text manipulation.

[0080] The component adaptor 225 may also consolidate menu components into a unified mobile-friendly (small format) menu. Pages often contain multiple navigation menus used for both inter-page and intra-page navigation. Additionally, some pages may contain multiple menus defined in various ways, such as menus defined within the page itself, menus formed from page templates used within the page (e.g., menus included in the header and footer of the entire application), and menus automatically defined based on the application structure (e.g., navigation menus of top-level pages). The component adaptor 225 may also form composite menu(s) and combine menus into a composite menu depending on criteria such as the proximity of the evaluated menus to a given formed composite menu (e.g., consolidating all menus in the top 200 pixels of a page into one top-level menu). The component adaptor 225 may directly insert the modified (unified and mobile-friendly) menu or menus or may "post" them for inclusion at a later stage (e.g., by the positioner 270 or the automatic additional component inserter 281). This is especially relevant when the menu(s) are added not as part of the normal component's layout, but as a "floating", context-activated layout, etc.

[0081] The component adjuster 227 may modify component parameters to reduce the width of a component. An example of this would be modifying a matrix gallery component to only one or two columns. The component adjuster 227 may also modify a component defined with a large decorative border into a simpler, less decorative version of the same component.

[0082] The component adjuster 227 may resize the component to reflect the actual size in which it will be used. For example, the component adjuster 227 may shrink a text component using a bounding rectangle depending on the actual text content.

[0083] The component adjuster 227 may also "dissolve" containers that tightly wrap an inner component. For example, if a container tightly wraps a sub-container that has no further components within a larger container, the component adjuster 227 may remove the inner container and combine the two containers into one. It will be appreciated that this saves a level of hierarchy because the containers are transparent to the process.

[0084] A designer may add multiple portions of the same image that together visually form a single image. The component adjuster 227 may utilize "image stitching" to detect whether multiple such image components should be stitched together into a single image using their attributes and content. The decision may be based on the edit history (i.e., whether the image was created and / or modified based on the similarity of the edge dimensions of adjacent images, the length of overlap of the edges of adjacent images compared to the separation between the images, or the use of similar colors / features on the borders (detected using an image content analyzer)).

[0085] The component adjuster 227 may utilize font size mapping. Many different font sizes may be used in the text of any application or page. These sizes should be mapped to a predefined (smaller) range for use on the mobile device, preserving font size differentiation if possible. Some font sizes will be too small and some too large. The mapping is not linear, i.e., not multiplied by a fixed coefficient. However, the mapping is a monotonic function of font size. The component adjuster 227 may collect the range of font sizes used and map this range to a range of allowed mobile font sizes. The component adjuster 227 may provide such system-wide or user, application, page, or component level specific mapping.

[0086] In application / page specific font size mapping, the component adjuster 227 may measure the amount of text (characters) at each given font size, and then using a normal (Gaussian) cumulative distribution function, the most common character size is mapped to a font size at the center of the range of allowed mobile font sizes. It will be appreciated that such font sizes may be further modified by mobile layout configuration specific font size rescaling / rescale readjustment.

[0087] Furthermore, it will be appreciated that when processing component structures, it is important to distinguish between decorative images and content images (which are part of the actual page-specific data). Decorative images can be scaled and cropped more freely since the content of the decorative images is not critical to the actual use of the application. Furthermore, content images should be analyzed like other components in the analysis algorithms described below, whereas decorative images should not be analyzed. The component analyzer 229 may recognize decorative images based either on the specific "decorative image" component type, or on specific hints at the template, object, component type, or designer level. The component analyzer 229 may also recognize whether an image covers an area (e.g., the image covers all (or most) of the container in which it resides), whether the image is the background for multiple components, or whether the image is fitted to the display area of ​​a component using operators such as "repeat photo in row" to indicate that the photo is used as decoration. Decorative images are not always "tight" with respect to the components contained within them, often by selecting the specific image associated with it.

[0088] As described herein above, after the elements are modified, the supernode former 230 may locate groups of components that may be significantly overlapping or otherwise related and that should be manipulated together. Such locating is necessary, for example, because the significantly overlapping components may form compositions and therefore must be positioned together to maintain the same internal proportions when resized. After all groups are located, the supernode former 230 may convert the formed hierarchical organization of components into a hierarchy of supernodes based on the current page and / or container hierarchy, so that it is understood that there are multiple total orders of different structural levels. For each individual page or minipage (page or minipage in a single-page or multi-page container), the supernode is known as a page supernode. It is also understood that each supernode may be considered a separate entity and, once defined, the objects may be rearranged as described in more detail herein below.

[0089] It will be further understood that some located groups may also be converted into virtual supernodes (more fully described herein below) that may be manipulated as a single component similar to the page supernodes described above herein. It will be understood that each virtual supernode type may have associated mobile adaptation methods defined at the time of supernode formation, such as re-scaling, limited resizing, reorganization, etc. in a particular manner.

[0090] Reference is now made to Figure 5, which illustrates elements of the supernode former 230. The supernode former 230 may include an overlapping group locator 232, a text-on-image group locator 234, a predefined group locator 236, a node former 238, and a scorer 500 (the functionality of which is described herein below in connection with Figure 11).

[0091] The overlap group locator 232 may determine a set of components that are largely overlapping, which together may typically form a particular design, and the proportions between the group components should remain the same when positioning the group in the new layout. For example, the images and text that form a logo should maintain the same proportions and relative positions, regardless of the size and position of the images and text in the new layout, in order to maintain the structure of the logo.

[0092] The overlap group locator 232 may loop over all possible component pairs in the supernode. For each pair of intersecting components, the overlap group locator 232 may calculate a relative overlap as the amount of overlap compared to the smaller of the two enclosing rectangles of the two components. The calculation may be based on the intersection area or may be determined by combining the relative intersections on each axis (e.g., using an average). If such relative intersection exceeds a predefined threshold, the overlap group locator 232 considers the component pair to be overlapping. After looping, the overlap group locator 232 may combine the overlapping pairs into overlap groups according to their common members. For example, the component pairs [a,b] and [b,c] may be combined into [a,b,c]. The area for each virtual component formed may be the area of ​​the smallest enclosing rectangle that encloses all the grouped components. Furthermore, since overlap groups generally represent detailed design elements, they should only be resized and not repositioned. In particular, text components within such groups should be matched to their original size rather than using the font size mapping described herein above performed by the component matcher 225. Thus, text should be scaled either to avoid font size mapping entirely, or using a scale factor that reverses the effect of font size mapping.

[0093] It will be further understood that some components may be omitted from the definition of an overlap group. These may include certain components such as horizontal lines, screen width containers, and text sizes above a certain limit (e.g. 25 characters) that are interpreted as text that is a regular paragraph and not as logo text. The regular paragraph should not be marked as an overlap group and should not be resized as a composition.

[0094] The text-on-image group locator 234 may group text components that are overlaid on a particular background image. The text-on-image group locator 234 may detect when a text component is set on a background image and the two should be operated on together. The text-on-image group locator 234 may search for pairs of text and image components in the current supernode where the text component is either completely contained within the image component or where the text component is sufficiently close (up to a predetermined threshold) to an enclosing image component on each of its four sides.

[0095] If the component is a logo rather than text-on-image, the text-on-image group locator 234 may instead instruct the overlap group locator 232 to form overlap groups rather than text-on-image groups. This may be done based on scoring according to criteria such as the amount of text characters being below a predefined threshold, the use of decorative or highly unusual fonts, or the use of a text baseline other than a regular horizontal line (e.g., the text follows a curve).

[0096] The predefined group locator 236 may group components according to template, application, page, or component level hints, which can often be seen as a persisted version of the grouping done using a particular design system's GUI so that multiple components can be manipulated together.

[0097] After the groups of overlapping elements are located and grouped, the node former 238 may combine and convert the groups into virtual supernodes, as described in more detail herein below, and then form the final supernodes (including virtual supernodes and page supernodes). The remaining elements may be considered disjointed, and therefore may be easier to order, as described herein below.

[0098] Reference is now made to Figures 8A-8C. Figures 8A-8C illustrate how the supernode forming unit 230 may form a hierarchy of supernodes. As described above in this specification, a supernode formed from a page and a minipage may be considered a page supernode, and a supernode formed from a detected group may be considered a virtual supernode. For page P1 in Figure 8A, a page (minipage) containing three text elements may be considered a page supernode (PSN1) having three elements. In Figure 8B, page P2 may be considered a page supernode (PSN2) having a text element and a virtual supernode (VSN1) containing a text element and a drawing element. If pages P1 and P2 are minipages contained within page P3, the node structure formed may be a page supernode (PSN3) having two page supernodes (PSN1 and PSN2) each having the elements shown in Figure 8C.

[0099] Additionally, the node former 238 may define a tightly wrapped container as a container A inside a container B where the sizes of containers A and B are very similar (i.e., there is a small amount of white space on all sides between containers A and B, and container A is the only component inside container B). The node former 238 may also rearrange the containers based on a semantic analysis of the components within the containers. For example, as shown in FIG. 9, referenced herein, text components T1, T2, and T3 are contained within a container C1, and photo components P1, P2, and P3 are contained within a container C2. The node former 238 may perform a semantic analysis and recognize related text and photo component pairs based on the component types, proximity, and relationships to other components, as described further below. Based on such semantic analysis, the node formation unit 238 can recognize not only that T1 and P1 form a related pair, but also the T2-P2 pair and the T3-P3 pair, and can remove these six components (T1, T2, T3, P1, P2, P3) from the containers C1 and C2 and position them within the container page while maintaining the information of the relationships between the above six components as shown in FIG. 10 referred to herein.

[0100] Additionally, the node former 238 may classify components that substantially overlap a given container (e.g., overlap by more than 75% of area) as members of the particular container for supernode generation purposes so that the supernode hierarchy better reflects the visual arrangement. This is relevant for visual design systems where components that are not logically contained within a given container may still overlap the container.

[0101] After the supernode structure is defined, an ordering unit 240 may order the component sets, the page supernodes contained within each (individual) supernode, and the virtual supernode. Furthermore, such basic ordering may be further modified as a result of discovery by a partial ordering unit, which will be described in further detail herein below. Reference is now made to FIG. 11, which illustrates the elements of ordering unit 240. Ordering unit 240 may include a basic ordering unit 247, a partial order set (POS) locator unit 250, and an order merger unit 245. Basic ordering unit 247 may further include a primary direction ordering unit 241, a primary direction ordering unit with separation and union 242, and an H / V slicer 243. H / V slicer 243 may further include an element splitter 244, which will be described in further detail herein below.

[0102] The POS locator 250 may include a cluster POS locator 251, a semantic relationship POS locator 252, a pattern POS locator 253, a predefined POS locator 254, and an ESI (Editing Session Information) based POS locator 255. It will be appreciated that the POS locator 250 may analyze the semantics, content, and geometry of the components.

[0103] It will be appreciated that the ordering unit 240 may mimic the order in which a human reader might view elements on a page (or within a particular supernode), and furthermore, it will be appreciated that because a page is a two-dimensional entity (or even three-dimensional, including z-order), such an order may not be well-defined even for a human reader.

[0104] Reference is now made to Figure 12, which shows an arrangement of four text paragraphs arranged in a rectangle. Assuming that the paragraphs are in English (read top to bottom and left to right), it is not clear whether the reading order should be A1, A2, B1, B2 or A1, B1, A2, B2. Both orders could be considered correct. Furthermore, when the elements are text and photos, as shown in Figure 13, which is now referred to, it is not clear which are the related text and photo elements.

[0105] The primary direction ordering unit 241 may predefine a primary direction (i.e., sort row first or sort column first) and then sort the components in the two-dimensional page based on the primary direction and then based on the second direction. For example, the primary direction ordering unit 241 operating in a row-first sorting mode may sort elements in a supernode such that element A precedes element B if the overlap Y of elements A and B is adequate (e.g., at least 25% of the lower of the heights of the two elements) and element A is on the left (i.e., in-row order), or if the overlap Y of elements A and B is not sufficient and element A is higher (i.e., between multiple rows order).

[0106] Also, the primary direction ordering unit with disjunctions and connections 242 (hereinafter, PDSM ordering unit 242) may predefine a primary direction and keep track of row / column disjunctions and connections. This may be considered more general for columns than rows. Now, please refer to Figs. 14A-14C. Figs. 14A-14C show components (1-10) in a supernode. It will be understood that these components may be ordered according to their order in the generated column graph in the node. The PDSM ordering unit 242 may essentially convert a set of rectangles (as shown in Fig. 14A) into a graph of columns (which may be connected, disjoint, or contiguous as shown in Fig. 14B), and this graph is then used for ordering the components (see Fig. 14C). The rectangles may intersect in some cases.

[0107] Reference is now made to Figures 15A-15C and 16A-16G. Figures 15A-15C and 16A-16G show the steps taken by the PDSM ordering unit 242 to define the string graph and the order to be formed. Figure 16A shows a set of elements C that need to be processed. The PDSM ordering unit 242 may define a two-element relationship between the elements in the set (step 400), and then "sandwich" the elements to be ordered (set C) between a dummy header element A and a dummy footer element B as shown. The PDSM ordering unit 242 may then form two new sets, namely a set X of the generated (current state) strings (steps 420 and 430) and the elements of these strings, and a set Y of elements to be processed as strings (steps 440 and 450). The PDSM ordering unit 242 may then sort the set Y (step 460) and traverse the elements of the set Y from top to bottom according to the y-order of the elements (step 470), and for each element Q of the set Y, may select a subset R of elements (in all columns in the set X) that control the element Q. The elements may then be arranged according to their order in the graph (top to bottom and left to right), or, in the case of isolated columns, according to their X-order (step 480). Now refer to FIG. 16B, which shows how elements A, E, and F all control Q. The PDSM ordering unit 242 may then remove any duplicates from the set R. As can be seen in FIG. 16B, not only do A, E, and F all control Q, but A controls E and E controls F. In this example, the only connection the PDSM ordering unit 242 needs may be the connection between Q and F (the last controlling element). If R is empty, this may be considered the start of an unconnected orphan string, and thus a new string is started in X with element Q, as shown in Figure 16C, referenced here. Furthermore, if R has more than one element, the PDSM ordering unit 242 may connect element Q to the string containing each of elements C1-Cn, forming a string sequence (if connected at the end of the string) or a separation of each string (if connected in the middle of the string).If there is more than one such string, the PDSM ordering unit 242 may combine the strings as shown in Figure 16D, to which reference is now made. As shown in Figure 16D, the PDSM ordering unit 242 may combine strings G and H into one string that is continuous by element Q. Reference is now made to Figure 16E, which shows string G continuing from an element that is not the last element in string G. In this example, element Q forms a separation in string G, with F and Q forming two consecutive strings.

[0108] It will be appreciated that when a column splits at a given point, the split set is ordered from left to right (i.e., columns successive to the split column have an order defined for them according to the x-coordinate of their leftmost). The same is true for column combinations. It will also be appreciated that the PDSM ordering unit 242 may have a situation where splits and combinations are combined, as shown in FIG. 16F, which is referred to herein. As shown, element Q forms a split of two columns including elements G and H. The PDSM ordering unit 242 may immediately combine the splits into one column that is successive by element Q.

[0109] Thus, the PDSM ordering unit 242 may move elements from Y to X and connect them to one or more existing columns in X (or possibly form new columns) based on geometric criteria. It will be appreciated that such a traversal process may form a series of columns that may start, separate, join, or terminate in X. The process ends when a dummy footer element B is reached. The PDSM ordering unit 242 may then traverse the graph (defined according to the columns in set X) and calculate a resulting order. The PDSM ordering unit 242 may then arrange the elements according to their order in the graph from top to bottom and left to right, as shown in reference FIG. 16G.

[0110] Further, in the case of isolated strings, it will be appreciated that the PDSM ordering unit 242 may arrange the isolated strings according to a numerical order (1-8) as shown in FIG. 16H, referenced herein. If the pair of elements ac and bd were close enough to be connected, they would form a complete string and the ordering between them would be acbd instead of abcd. Since the pair is not close enough, they are not connected, making elements c and d an isolated string, and the ordering is corrected to the specified numerical order as described above in this specification.

[0111] It will be appreciated that the functionality of the PDSM ordering unit 242 may be appropriately modified for supernodes having content arranged from right-to-left (rather than left-to-right).

[0112] The H / V slicer 243 may define a display order between elements by slicing the elements in a supernode alternately horizontally and vertically to form an internal tree of the partitions so divided. It will be understood that such an internal tree is specific to a given supernode and is not related to the level tree of the supernode in general. The H / V slicer 243 is best suited for "chocolate table" style sites, classically newspaper layouts such as the exemplary page shown in FIG. 17, referenced here, that are constructed from horizontal and vertical partitions. The H / V slicer 243 may take the set of elements of one supernode and form an internal tree of the node (a node is different from a higher level supernode, since each supernode may contain such a separate internal tree). The nodes may be of the following types:

[0113] V Node (Vertically Arranged Node) - A set of subnodes / elements arranged from top to bottom.

[0114] H-node (Horizontally-ordered node) - a set of subnodes / elements arranged from left to right (right-to-left arrangement is explained herein below).

[0115] Leaf Node - A node that represents the last element of an internal tree, i.e. a component or a wrapped supernode. A leaf node may contain multiple components that still fit within their allocated width.

[0116] Unresolved Node (UR Node) - A node that contains multiple elements that have not yet been placed. A UR node is the initial state of any node before it is transformed into a V / H / leaf node.

[0117] Each node may contain not only a reference to the set of elements, but also the coordinates and size of each underlying element, and the complete bounding rectangle for the node's elements. The entire collection of nodes (of the four types above) within a supernode may be known as the internal tree. H / V slicer 243 may adjust the procedure for supernodes that are identified or detected as containing right-to-left (R2L) material or having an R2L layout.

[0118] On initial invocation, H / V slicer 243 may form an internal tree with one UR node and assign all elements in the supernode a reference in the internal tree. H / V slicer 243 may operate on a UR node by getting all elements in the node. If the bounding rectangle of an element fits within the allowed width, H / V slicer 243 may convert the node to a leaf node. H / V slicer 243 may also require element splitter 244, which may suggest a specific (horizontal or vertical) split of the UR node into subgroups of elements.

[0119] The H / V slicer 243 may then convert the UR node to an H node or a V node as appropriate. It will be appreciated that for each subgroup returned by the element splitter 244, the H / V slicer 243 may calculate a bounding rectangle R for the elements in the subgroup. If the width of R fits within the allowed width, the H / V slicer 243 may form a leaf node of the descendant of node N from the subgroup. In other cases, if the subgroup has more than one component, the H / V slicer 243 may form a leaf node of the descendant of node N from the subgroup and manipulate this UR node as appropriate as described above in this specification (i.e., by recursively applying the H / V slicer 243). If the width of R does not fit within the allowed width, the H / V slicer 243 may form a leaf node of the descendant of node N from the subgroup containing one element. The H / V slicer 243 may then apply width reduction, such as resizing, reflowing text, etc., as described above in this specification.

[0120] The H / V slicer 243 may then recursively (using a depth-first traversal) traverse the generated internal tree and emit components (such as leaf nodes) according to the traversal order. The H / V slicer 243 may traverse each of the H / V nodes according to their natural order (e.g., from top to bottom for V nodes). It will be appreciated that at this stage, such a formed internal tree of supernodes is no longer necessary, since a basic order of elements has been generated.

[0121] As described herein above, the element divider 244 may calculate partitions for a group of elements. The element divider 244 may ignore borders and calculate a bounding rectangle for each element. The element divider 244 may then project these rectangles onto both the X and Y axes.

[0122] The element divider 244 may then divide each set of projections onto the axes into segments with different numbers of components projected onto them. Segments with no projected components represent gaps in a given projection direction. It will be appreciated that the element divider 244 may provide certain (possibly non-displayable) separator line components to aid in determining the correct separation direction (e.g., to be used as a hint to the designer in the process). Such components may also have a "weight" assigned to them, such that they are worth one or more components (with respect to the calculation of the projected rectangle).

[0123] If gaps are found in both directions, the element splitter 244 may generate horizontal and vertical partitions of the element based on partitions perpendicular to the gap segments, e.g., as shown in Figure 18, referenced herein, case A is a horizontal partition (into columns) using one partition, and case B is a vertical partition (into rows) using two partitions.

[0124] The element divider 244 may calculate a Division Quality Rating (DQR) for each of the two directions of the division partition using a weighted average of the maximum number of partitions, the maximum total size of gaps found, and the minimum number of similar elements divided by the division partition into separate subgroups. Such information may be available from the Pattern POS Locator 253 (described in further detail herein below) during the formation stage of the supernodes, which defines the similarity relationships between descendant supernodes. As shown in FIG. 19, referenced here, a vertical division (into rows) is preferred to keep the service / project / client boxes (A, B, C) together.

[0125] The element splitter 244 may also take into account in which direction the components are better aligned. For example, as shown in FIG. 20, referenced herein, in example A, the split partition should be vertical (i.e., components are placed / aligned in rows) even though there may be large gaps when making the split partition horizontally (into columns). This can be seen more clearly when alignment lines are added in example B.

[0126] The element splitter 244 may also add a constant preference factor to horizontal split sections (that reduce width) relative to vertical split sections (that do not reduce width), in which case the element splitter 244 may return split sections with higher DQR.

[0127] If a gap is found in only one direction, the element splitter 244 may return a split partition in that direction. If no gap is found, a case of interlocking elements may exist. In such a case, the element splitter 244 may form a partition that intersects one of the elements. For example, as shown in FIG. 21, referenced herein, the elements are split using a vertical split line A that may intersect only with element B. All elements marked with R (including element B) may be associated with the right side of the split partition, and element L may be associated with the left side of the split partition.

[0128] In such an example, the element divider 244 may find a projected segment with a minimum number of intersecting elements (from the projected segments in both the X-axis and the Y-axis). If only one such segment exists, the element divider 244 may define a partition region based on that segment. If two or more such segments (in both directions) exist with the same minimum number of intersecting elements, the element divider 244 may form a set of partition regions with one partition based on one of the segments with the minimum number of intersecting elements.

[0129] For each of these partitions, the element divider 244 may then calculate a DQR based on a weighted average of the usual DQR calculation criteria (described above in this specification), the smallest area of ​​the intersecting elements, and the restrictiveness of the cut (i.e., the proportion of the area of ​​the intersecting elements is smallest on one side of the partition that it intersects), so that the intersecting elements more clearly belong to one of the two sides of the partition. Alternatively, the element divider 244 may use the partition with the highest DQR.

[0130] In this case, the element splitter 244 may return the selected split section in which each intersecting element belongs to the side of the partition that contains the larger area of ​​the intersecting element.

[0131] In an alternative embodiment, element divider 244 may form multiple partitions based on multiple projected segments having the same number of intersecting elements and may evaluate a partition section that includes multiple such intersecting elements.

[0132] It will be appreciated that element splitter 244 may recursively invoke alternative DQR calculations for each possible partition and check the DQR values ​​of the partitions formed after each potential partition has been tried, which may result in further partitions being better since element splitter 244 may find one best element to intersect.

[0133] It will be appreciated that in parallel with the basic ordering unit 247, the POS locator unit 250 may detect sets of components that are somehow related and that should remain together, if possible, when the elements are reordered for display on the mobile device. For example, such components may be a text heading and a corresponding text paragraph.

[0134] The cluster POS locator 251 may detect a cluster partial ordered set when components (of any type) are very close to each other compared to the normal spacing between components in a particular supernode. The cluster POS locator 251 may calculate the average distance between components in a supernode, and then loop through all component pairs in the supernode to find those whose distance is a predetermined percentage of the calculated average distance. The cluster POS locator 251 may then combine these component pairs into sets and enforce a predetermined maximum distance (specified as a percentage of the average distance) between the two most distant components of each of the sets. The cluster POS locator 251 may also use a middle or median calculation rather than an average calculation to account for a predetermined number of outliers that may skew the average distance value. In an alternative embodiment, the cluster POS locator 251 may use any clustering algorithm known to those skilled in the art. In this case, the POS locator 251 checks the extracted clusters and forms a cluster partial ordered set if it is dense enough (e.g., if the maximum distance between cluster members is less than a predetermined threshold).

[0135] The semantic relation POS locator 252 may detect a semantic relation partially ordered set when there are certain combinations of components of a given type, such as a photo and a description that are close together. The semantic relation POS locator 252 may scan all possible component pairs in a supernode and check for each potential pair that each of the components has the correct type (e.g., one is text and the other is a photo), that the components are close together (the distance is less than a predetermined threshold), and that there are no intervening components between them. For an evaluated component pair [A,B], the semantic relation POS locator 252 may check that there is no third component C that can be substituted for B, that is of the right type as B, and that is closer to A than B is. Similarly for the relationship between A and C.

[0136] Other combinations of component pairs may include a partially ordered set of text and buttons. In such an example, the semantic relationship POS locator 252 may link a button that depicts a text field to the actual text field. This relationship may be determined according to position alone.

[0137] Other combinations may be text-linked partially ordered sets. In this example, the semantic relationship POS locator 252 may join multiple text elements that are consecutive to one another. It will be understood that the semantic relationship POS locator 252 may only recognize one text element above another text element. It will be further understood that any number of such elements (not just two) may be present in a given set, all of which are linked.

[0138] The pattern POS locator 253 may detect a pattern partially ordered set when there are a predetermined number of components (e.g., pairs or triplets of components) of a repeating pattern with a particular type, property, and layout at a predetermined distance. The pattern POS locator 253 may also detect a pattern partially ordered set in the case of a switching pattern as shown in FIG. 22, which is referred to herein. Component pair A and C have text on the left and a photo on the right, and component pair B and D have text on the right and a photo on the left. The pattern POS locator 253 may detect such a pattern based on component pairs with a particular type and property, but with a horizontal distance of one of two options (negative or positive numbers with the same absolute value) within each component pair. The pattern POS locator 253 may scan all components in a supernode (in the example of a two-member pattern that may be merged later) and, for each component, locate the nearest neighbors on all four sides of the component. The pattern POS locator 253 may include neighboring components that overlap the component up to a certain threshold limit. In this case, the pattern POS locator 253 may store a relationship list between the component and the neighboring components, and then scan the generated relationship list to select similar relationship pairs according to attributes such as components having the same type (e.g., component [pic,txt] is similar to another component [pic,txt]), components having the same direction (including reverse direction if switching patterns are supported), and similar distance (subject to a certain threshold of difference), etc.

[0139] It will be appreciated that the two components of the selected relationship should have different types, or else a set of four components of the same type arranged in a rectangular shape may generate two opposing sets of similarity relationship pairs. For example, as shown in Figure 23, referred to herein, the four photo components may be divided into horizontal pairs (related pair H1, H2) or vertical pairs (related pair V1, V2).

[0140] The pattern POS locator 253 may scan the list of selected relations and combine the selected relations into sets (e.g., if r1=r2 and r2=r3, then a set [r1, r2, r3] may be generated). It will be appreciated that a set of patterns (e.g., a set of multiple component sets [txt, txt, pic]) should be operated on together.

[0141] The predefined POS locator 254 may detect the partially ordered set formed based on certain hints that may be provided by the application designer or may be present in the original application template. Such hints may take many forms. Certain composite components that include predefined partial order system hints, such as a "photo + text description" component, may include concatenating the two components to generate a predefined partially ordered set. Other hints may be explicitly specified associations of any components in the page or mobile-related associations derived from other types of associations available in the visual design system (e.g., grouping for editing). This is because many visual design systems can associate and group components so that they can be removed, resized, or modified together. Other associations may be dynamic layout anchors, because the visual design system may support dynamic layout anchors (either explicitly specified or automatically formed from the layout definition). Such anchors may also be used to form the partially ordered set. Still other associations may be templates (i.e., a component set formed from multiple instances of the same multi-component template).

[0142] It will be appreciated that the predefined POS locator 254 may provide both predefined groups and predefined partial ordered sets, which are different elements of the system, and allows the designer to specify predefined group hints that are used to group components to form virtual supernodes, and predefined partial ordered sets / ordering hints that are used to guide the ordering process.

[0143] The ESI-based POS locator 255 may detect automatic formation of partially ordered sets based on ESI (if available). In particular, the ESI-based POS locator 255 may associate components (as one partially ordered set) based on information collected from previous editing sessions, such as component sets formed using duplication or copy-and-paste, component sets that were grouped and edited together (e.g., when the visual design system only supports ad-hoc groupings that are not saved in a database), and component sets that were edited sequentially.

[0144] After the ordering unit 240 has formed a basic order and the POS locator unit 250 has determined any partial order sets, the order integrator 245 may integrate the basic order and the detected partial order sets to form a combined revised order that is a total order of all relevant elements, thereby resolving conflicting partial order sets along the way.

[0145] It will be appreciated that some elements of the basic ordering unit 247 may not be intended to form a complete order between all components, but rather to split the page / supernode into page portions that fit the allowed width. This may be relevant when a basic structure needs to be maintained (e.g., when an application is translated to a device (e.g., tablet, large display) that is the same width as or larger than a desktop display). In this case, a minimal "slicing" may be visually superior, since breaking down into linearly arranged components would destroy the basic component arrangement. To this end, elements of the basic ordering unit 247 may be modified to operate on page portions (that still fit the available width) rather than on individual components.

[0146] In this case, the basic ordering unit 247 may use such information as a guideline so that page portions that are connected via substantial POS connections are not sliced.

[0147] It will be appreciated that the supernode former 230, the basic ordering unit 247, and the POS locator 250 may further form results that conflict with each other. For example, one of the sub-elements of the POS locator 250 may generate a particular partial ordered set definition for some elements, and another element may generate a different, conflicting partial ordered set definition. The order unifier 245 may combine the results from the various elements to form a single output and may resolve the conflict.

[0148] Additionally, to resolve such conflicts, the supernode 230, basic ordering unit 247, and POS locator 250 may all use quality scores. For example, a design based on a designer's explicit requirements will usually have the highest quality score, since it can be assumed that the designer is fully aware of the configuration of his or her design. The supernode 230, basic ordering unit 247, and POS locator 250 may all check their scores against the quality scores maintained by the scorer 500. This scorer 500 may provide each generated result with a certainty score of how certain the elements are about the correctness of the particular result based on the analyzed parameters.

[0149] The order combiner 245 may collect results from the basic ordering unit 247 and the POS locator 250 and combine the results. It will be appreciated that results that are not in conflict with each other may simply be combined. Results that are in conflict with each other may require the order combiner 245 to select which result to use. This selection may be made depending on the calculation of the combined quality score and certainty score of a particular result. If the results do not meet a predefined (optional) certainty threshold, they may be discarded.

[0150] The order integrater 245 may also consider additional information, such as results defined with respect to components present in a template that may have a higher certainty than a result that includes only components defined in the current page (e.g., because the template is better designed than a normal page).The order integrater 245 may also consider results that may be similar to results found in previous pages or supernodes that have been previously visited and therefore assigned a higher certainty.

[0151] The order aggregator 245 can be extended to learn the page structure created by a given designer, possibly using feedback from that designer rating the results of elements.

[0152] Specific elements may return result sets that should be implemented together or not at all. For example, pattern POS locator 253 may discover repeating patterns of components. Such patterns should be used for all associated components or not used at all.

[0153] After the components are properly ordered, the locator 270 may place the components, and any auto-added components (described in more detail later in this specification), in their appropriate locations within the page. The locator 270 may position the ordered elements on the component lines as necessary to fit the available width according to the determined total order and the allowed space. It will be appreciated that the available width may be defined by the width of the page supernode minus other constraints such as predefined margins and reserved space. For example, if the main page width is 320 pixels, the left and right margins are 10 pixels each, and there is no reserved space, the available width of the main page supernode may be 320-(2x10)=300 pixels. The available width of the page supernode contained within the main page supernode and taking the maximum available width may be 300-(2x10)=280 pixels, etc.

[0154] The positioner 270 may apply layout hints that come as part of the desktop layout composition if they are marked "keep in mobile layout composition." Such components may function more similarly to auto-add components in that they become part of the container architecture and may affect the construction of the component line rather than being part of the component line itself.

[0155] The locator 270 may then form line breaks according to explicit hints (e.g., "put this group / POS on a separator line") formed by the supernode former 230 and the orderer 240. If the components to be added exceed the available width (considering the reserved space), the locator 270 may keep the component group together, i.e., form line breaks before the group as necessary, so that the entire group may fit on the same component line. The locator 270 may avoid adding line breaks that would break between components that have a semantic relationship (detected earlier, such as by the POS locator 250).

[0156] The positioner 270 may also apply width reduction measures to each element depending on the element type. The width reduction measures may include modifying components (e.g., text components) that may be adapted depending on the component type. Such modifications may include rescaling, rescaling, changing font size, reflowing text, etc. The width reduction may also use per-virtual supernode manipulation methods defined depending on the type of each virtual supernode. These width reduction measures may also include transformations similar to those performed by the component adjuster 227 (as a pre-processing step). Such transformations may now be performed using additional information about the final position and size of the component in the mobile layout configuration. The positioner 270 may also expand the width, for example, to expand components that are positioned in isolation within a component line.

[0157] The locator 270 provides specific resizing means for specific element types. For example, the locator 270 may resize a supernode representing a text-on-image group (located by the text-on-image group locator 234) by using the background image as a virtual container. It then resizes the text to the desired height and width. This resizing may include font size mapping and text reflowing, which may change the aspect ratio of the text component (e.g., forming a more elongated text component with a larger character inside), as shown in FIG. 6, referenced herein. As shown in FIG. 6, when moving from example 1 to example 2, the font size is increased while the field width is reduced, requiring text reflowing. The text-on-image group locator 234 may also resize the group when the background image is resized to the next text size, which would usually change the aspect ratio.

[0158] It will be appreciated that a "visual accident" may occur if the photo has real content (e.g., a photo of a person) that may not look good when the aspect ratio is changed, as shown in Figure 7, referenced herein. The text-on-image group locator 234 may eliminate this problem by checking if the image content contains detailed information, or by preserving the internal aspect ratio of the image by zooming and cropping instead of resizing.

[0159] Some components have a degree of autonomy in resizing themselves. Examples include third party application components (which may support multiple variants of different display sizes) and list applications (which may have multiple view associations). Such components resize themselves based on a target size provided by the positioner 270.

[0160] The positioner 270 may also resize / repeat decorative images as necessary to fit the new page size. If the page exceeds a predetermined length parameter, the positioner 270 may potentially form an in-page navigation menu and appropriately add this menu to the page as a separate menu or to an existing unified menu.

[0161] The positioner 270 may rescale both the height and width while preserving the aspect ratio of the component. Some components may retain their height and only change their width (e.g., a gallery that is modified to use fewer columns). Components that must use a specific height when rescaled (e.g., a map component with internal scrolling that must leave space above and below the component for page scrolling but not map scrolling) may be handled appropriately. It will be appreciated that width reductions may be handled similarly.

[0162] The positioner 270 may also apply width reduction / expansion processes to components, as well as recursively to supernodes and descendants of supernodes. Such processes may include actual component resizing; text font size mapping; text reflow; text reduction by separating visible portions from "show more" extensions (which are only displayed when requested), etc. Other processes may include switching to "lighter" mobile versions of components, modifying component parameters to change widths. Processes may also include resizing components to reflect the actual sizes used.

[0163] After the positioner 270 has positioned the relevant components in their correct locations within the mobile configuration layout, the post-processor 202 may handle any final layout adjustments.

[0164] The auto-add component inserter 281 may insert auto-add components into new mobile layout configurations, such as system menus or other mobile-related widgets that may be added to all or some of the converted applications (to all or some of the pages). Such inserted auto-add components may also include, for example, advertisements and / or other promotional materials. Such inserted auto-add components may be mandatory (i.e., always inserted) or optional. If inserted optionally, the auto-add components may be conditionally inserted based on a parameter or combination of parameters, such as the type or profile of the user (e.g., a specific auto-add component is inserted for all users in Europe) or the type or profile of the mobile device (e.g., a specific auto-add component is inserted for all Android users with a screen size of 480x320 or more). The auto-add component may also be based on specific page parameters or conditions (e.g., a particular auto-add component is inserted for pages that do not contain a photo component of size 320x200 or larger), user behavior or application usage history, availability of space for the auto-add component to be inserted, any designer-defined parameters, and any website building system-defined parameters.

[0165] It will be appreciated that the auto-add component inserter 281 may position such an auto-add component in a number of ways. The auto-add component inserter 281 may position the auto-add component as an absolutely positioned auto-add component that is inserted at a predefined location within the page or within a particular container (i.e., a supernode), typically using a reserved space as described herein below. The auto-add component inserter 281 may also insert the auto-add component as a relatively positioned auto-add component that is inserted at a predefined location relative to some page element (e.g., a predefined component, a component located by a query, etc.). This is similar to the insertion of a component at the mobile layout configuration level, as described in more detail herein below.

[0166] The auto-add component inserter 281 may also add auto-add components as remaining space auto-add components that are inserted based on the availability of space in the layout. Because components may be moved relative to separation component lines, in some cases there is additional empty space that can be used for such auto-add components in a mobile layout configuration compared to a desktop layout configuration.

[0167] For absolutely positioned auto-add components, the auto-add component inserter 281 may be required to reserve a certain space for these elements on the mobile display area (by carving out said area from the rectangular mobile display area allocated to the page / supercontainer). Therefore, the area used for the component lines may be non-rectangular. Refer now to FIG. 24, which shows that after carving out the area reserved for auto-add components A and B, the area remaining in the main container C is non-rectangular.

[0168] It will be appreciated that the associated visual design system may also support dynamic layouts, including using implicit (automatically formed) anchors as well as explicit (designer specified) anchors as described in US Patent Application Publication No. 2013 / 0133634, entitled "A SERVER BASED WEB SITE DESIGN SYSTEM INTEGRATING DYNAMIC LAYOUT AND DYNAMIC CONTENT," published Aug. 22, 2013, and assigned to the assignee of the present invention and a common assignee. Thus, the visual design system may have anchors (explicit and implicit) that may be broken by the repositioning process inherent in adapting to the width of a mobile device.

[0169] After the components are properly positioned, the dynamic layout manipulator 284 may modify any existing dynamic layout anchors according to the new layout. For example, the dynamic layout manipulator 284 may remove anchors of components that have been moved such that the anchors become meaningless, as shown in Figure 3, referenced herein. As shown in Figure 3, a horizontal anchor that exists between components A and B in Example 1 should be broken when component B is moved below A in Example 2.

[0170] The dynamic layout manipulator 284 may also retain anchors that may be retained (e.g., unmodified sets of components in unmodified containers). The dynamic layout manipulator 284 may also modify anchors that may be retained but may require parameter modification (e.g., changing the length of an anchor for components that have been moved closer together). The dynamic layout manipulator 284 may also create new dynamic layout anchors (including both component-component anchors and component-container anchors) between closely positioned components. This may be based on automatic anchor creation criteria built into the system (e.g., based on the amount of overlap and distance between components).

[0171] It will be appreciated that the newly formed anchors are important because a mobile adapted application may still be an absolute visual design application. Thus, a mobile adapted application will have to be dynamically modified with changes such as modifications to the content of a component such as the amount of text contained. These modifications may include modifications coming from external sources (external data feeds, concurrent user activity, switching between data records in a list application, etc.). The dynamic layout manipulator 284 will have to perform further adaptation modifications in response to small modifications in the screen size of the target mobile device. Thus, the dynamic layout manipulator 284 may modify the anchor structure and implement new anchor structures for final adaptation.

[0172] It will be appreciated that the alternative layouts that are formed (including grouping, POS, and ordering information) may be stored in database 50 along with the application or separately from the application for further use or as a basis for further modification as described herein below.

[0173] The display unit 290 may then display the newly modified layout on the destination platform.

[0174] As described above in this specification, the layout reconversion converter 300 may combine modifications made on a given desktop layout configuration and modifications to a shadow mobile layout configuration that corresponds to that desktop layout configuration to form an updated mobile layout configuration.

[0175] After operation of the initial layout converter 200, there are two (or more) versions of the application: a desktop (main) layout configuration and one (or more) (shadow) mobile layout configurations. When referring to a mobile (or shadow) layout configuration, we are referring to other additional configurations, such as configurations related to tablets, wide screen displays, etc. In this case, the designer may apply (separate) modifications to both the desktop and mobile layout configurations through the editing tools of the associated visual design system to form modified versions of each configuration. The visual design system may provide a single editor for both versions (possibly with constraints when editing the mobile layout), or separate desktop and mobile site editors.

[0176] Since the mobile layout configuration is derived from the desktop layout configuration, the initial layout converter 200 may apply modifications made to the desktop layout configuration (e.g., adding and removing pages and components, and changing component content) to the mobile layout configuration. The reconversion converter 300 may combine any desktop layout configuration modifications and any individual mobile layout configuration modifications, while better aligning the mobile layout configuration content to the desktop layout configuration, to form the final mobile layout configuration displayed to the mobile user. It will be appreciated that this procedure is complicated, for example, when adding a component to the desktop layout configuration that is also to be added to the modified mobile layout configuration (the modified mobile layout configuration component may have been moved (including moved out of, into, or between containers), resized, or deleted), the reconversion converter 300 must determine the appropriate location for positioning the added component in the modified layout configuration as well as the layout to be used for the added component itself.

[0177] In such an example, it will be understood that modifications of any mobile layout configuration are not merged into the desktop layout configuration. While editing the desktop site may change the mobile site, editing the mobile site does not affect the desktop site. It will be understood that pure layout changes (i.e., changes in position and size) made to either the desktop or mobile layout configuration do not affect the other layout configuration, unlike, for example, changing or deleting component content (which would affect the mobile layout configuration if made in the desktop layout configuration). It will further be understood that since the system 100 may restrict mobile editing, components may be deleted (or hidden) from the mobile layout configuration, but cannot be added, and the content of components cannot be edited (with the possible exception of mobile-oriented components, which are described in more detail herein below). Thus, for example, components in a mobile layout configuration cannot be moved from one page to another by a mobile editor, since such a move would involve adding a component to a page.

[0178] Reference is now made to Fig. 25, which illustrates the elements of the reconversion converter 300. The reconversion converter 300 may include a page deleter 310, a page adder 320, a component deleter 330, a component adder 340, a component modifyer 350, and a mobile operation unit 360. The component adder 340 may further include an add component identifier 342, a parent / ancestor searcher 344, and a mobile layout adder 346, as illustrated in Fig. 26, which is referred to herein. The functions of these elements are described in further detail herein below.

[0179] Reference is now made to Figure 27, which illustrates the functionality of the reconversion converter 300. The initial layout converter 200 may convert DP1 (desktop page 1) to form a mobile layout configuration MP2. In this case, a designer may edit MP1 using a local mobile editor 560 to form MP2. The designer may then edit DP1 using a local desktop editor 550 to form DP2. The reconversion converter 300 may then form an updated mobile layout configuration that integrates DP2 and the changes made in MP2.

[0180] It will be understood that such automatic integration may only be relevant if the desktop editing stage (DP1 → DP2) is performed after the mobile editing stage (MP1 → MP2) (see FIG. 26). In the opposite case (desktop editing is performed first), the initial layout converter 200 may automatically reconvert the modified desktop layout configuration DP2 to the mobile layout configuration MP2 upon completion of the editing of DP2. Thus, the designer may make MP1 → MP2 changes on the already modified MP1 and need to manually integrate said changes during the (later) mobile modification.

[0181] Additionally, it will be appreciated that in order for the same site to work on multiple platforms, one will have as much of the same content in both sites, however, as noted above in this specification, not all desktop components are mobile compatible, for example, through the use of drop-down menus, vertical lines, etc.

[0182] It will also be appreciated that a major requirement of web-based interactive applications (e.g., websites) is search-engine compatibility. Thus, both desktop and mobile layout configurations should present the same general page structure to search engines, and the same pages should be available in both layout configurations. Otherwise, a page may be indexed by a search engine spider that reads the page in the desktop layout configuration, but may be unavailable (generating a "web page not found" or similar error condition) when a user reaches the page directly via a search engine on a mobile device. In the opposite case, a page that is available in the mobile layout configuration but unavailable in the (indexed) desktop layout configuration will not be discovered by the search engine. Some search engines may support various page structures corresponding to different versions of a site, but the use and availability of such options is an internal property of the particular search engine and such options cannot be relied upon.

[0183] It will be understood that although the desktop and mobile sites may look different, they may share the same data whenever possible. Otherwise, a user may reach a mobile page, for example, by searching for a given text phrase present in the content of the desktop page, only to find that the mobile page is different and does not contain the searched phrase. In particular, a mobile page should not contain components that are not present on the desktop page. Some exceptions may apply, such as adding a new mobile-adapted menu to replace an existing menu, or removing text components in the mobile layout configuration only.

[0184] It will also be appreciated that designers may need to make certain modifications to the mobile layout configuration, and may purposely make the mobile layout configuration different from the desktop layout configuration in violation of search engine policies. As an example, a system that displays a feed of summaries of new articles may display longer summaries on the desktop layout configuration and shorter summaries on the mobile layout configuration. Designers may certainly modify the size, position, and ordering of components, and such changes should generally not affect the operation of the mobile site by search engines.

[0185] To resolve, or at least ameliorate, such conflicts, the reconverter converter 300 may limit the scope of modifications that the designer may make to the mobile layout configuration. For example, the designer's modifications may be limited to modifications that may only affect the mobile layout configuration, but not the actual content. Such limitations may be enforced by the system's mobile editor 560 before the various modifications are manipulated by the reconverter 300.

[0186] Modifications to the desktop layout may include adding pages, removing pages, adding components, removing components, changing the attributes of components, moving and resizing components, and changing the content of components (e.g., the text within a text paragraph). As noted above, pure layout changes (e.g., moving and resizing) do not affect the mobile layout composition.

[0187] As described above in this specification, the visual design system may have an internal unique ID for each component that does not change when the component is modified or moved. The reconversion converter 300 may use these IDs to track components that have been modified from one version of a page to another. Because the reconversion converter 300 assumes that all modifications are made through the visual design system editor and are fully tracked, when evaluating the modified version, the reconversion converter 300 may have a detailed list (for each modified component) of the modified attributes and value changes (e.g., for component X, the style was changed from S1 to S2 and the width was changed from W1 to W2).

[0188] When removing an existing desktop page, the page remover 310 may also remove the page from the mobile layout configuration. It should be understood that this means that any specific mobile layout configuration modifications made to that page will be lost. The only way to recover those modifications is via an "undo" function (e.g., undoing at the session level during the current session, or reverting to a version at the database level).

[0189] When a new page is added to the desktop, the page adder 320 may instruct the initial layout converter 200 to convert the entire page to a mobile layout configuration, as described above in this specification. It will be appreciated that the converted page may appear in the appropriate location in the mobile layout configuration. The page adder 320 may update any mobile navigation menus to include the newly inserted page.

[0190] When a component is removed from an existing desktop layout configuration, the component remover 330 may remove the corresponding component from the corresponding mobile layout configuration. The component remover 330 may also instruct the mobile controller 360 to update the mobile layout display, and may close gaps in a given component line or remove the entire component line, as described in the description of the mobile controller 360 below.

[0191] It will be appreciated that when one or more new components are added to a desktop page, the new components may form a new hierarchical component order corresponding to the modified desktop page including the newly added components. Only when no changes have been made to the mobile layout configuration with respect to the changes to the desktop layout configuration, the reconversion converter 300 may instruct the initial layout converter 200 to reconvert the desktop page, since there is no need to combine the desktop layout configuration and the modifications to the mobile layout configuration.

[0192] It will be appreciated that the component adder 320 must identify a suitable location in the mobile layout composition to add the added component to the layout composition. This task can be particularly complex because components may have been re-moved (including moved between containers), resized, repositioned, or removed in the mobile layout composition. Additionally, components in the desktop layout composition may have been moved or removed. Thus, the component adder 320 must find a suitable location to add the component according to a preprocessor and / or parent (one of which is always present).

[0193] For each component X added to the mobile layout composition, the added component identifier 342 may identify the added component (e.g., by component ID comparison, editing session history, or access to an underlying database) and may instruct the component filter 221 to perform component filtering to determine whether the component is suitable for mobile display (e.g., not an empty component, not a mobile-unsupported type, etc.).

[0194] The parent / ancestor searcher 344 may then perform a parent / ancestor search to locate the closest ancestor component PD(X) and / or parent component PT(X) that are closest to the added component X and that are also present in the mobile layout composition (i.e., ancestor components PD(X) and / or parent components PT(X) that have not been removed from the mobile layout composition explicitly or due to their type). The ancestors are determined according to the order identified by the ordering unit 240, i.e., the closest ancestor that appears in the mobile layout composition is selected. The parents are determined according to the page container hierarchy, i.e., the closest level parent container that appears in the mobile layout composition is selected. It will be appreciated that some components may be manually deleted (or hidden) in the mobile layout composition and therefore excluded from the parent / ancestor search. It will also be appreciated that, for example, if X is the first component in the page / container (excluding components not visible in mobile) and has no ancestors, there is no closest ancestor component. Further, it will be understood that the parent component cannot be absent because the search necessarily reaches the page used as the top-level container. After the parent / ancestor finder 344 has determined the ancestors and parents (if available), the mobile layout adder 346 may position the added component X after the closest ancestor component in a position that has the same relationship to the ancestor / parent component in the mobile layout composition as the added component X had to the ancestor / parent component in the desktop layout composition, and may push down all components after the ancestors.

[0195] It will be appreciated that when working with multiple added desktop layout configuration components (X[1], X[2], ...), there are two main ways to perform the above procedure: a one-by-one method or a clustered method.

[0196] In the one-by-one method, the added desktop layout configuration components X[i] are operated according to their order (e.g., defined by the ordering unit 240). For each component X[i], the parent / ancestor search unit 344 performs an ancestor / parent search considering the preceding added components X[j] (1 ≤ j < i). Thus, each component X[i] is operated considering other added components.

[0197] In the clustering method, the parent / ancestor search unit 344 can perform a parent / ancestor search individually for each added desktop layout configuration component X[i], ignoring other added desktop layout configuration components X[j] (j ≠ i). Thus, the parent / ancestor search unit 344 performs a parent / ancestor search for each X[i] as if X[i] were the only component added to the desktop layout configuration. After such a search is completed, the component X[i] is clustered according to the common combination of the ancestor component PD(X[i]) and the parent component PT(X[i]). The components within each cluster can be grouped together as one virtual page (e.g., a virtual supernode that can contain other supernodes inside). In this case, the component addition unit 340 can instruct the initial layout converter 200 to operate individually on each such virtual page and recursively perform the complete conversion process (the process including the preprocess, analysis, anchor, etc. described above in this specification). By this process, each virtual page can be converted into a converted virtual page. This converted virtual page is then positioned as a unit within the mobile layout configuration by the mobile layout addition unit 346 based on the ancestors and parents common to the cluster.

[0198] Now refer to FIGS. 28A to 28G. FIGS. 28A to 28G show examples of addition and editing of various components and how those components are operated by the component addition unit 340.

[0199] Reference is now made to Figure 28A, which shows a desktop page DP1 containing two components A and B. The initial layout converter 200 may convert such a page into a mobile page MP1. If a new component X is added between A and B in DP1 to form a page DP2, the parent / ancestor lookup unit 344 performs a parent / ancestor lookup for X and finds that it follows A in the page containing X (desktop layout configuration). Thus, the mobile layout appender inserts X after A in the page containing X in MP1 to form MP2.

[0200] In another example, as shown in FIG. 28B referred to herein, mobile page MP1 is edited via mobile editor 560 to swap the vertical positions of A and B to form MP2. After such editing, the designer then inserts a new component X between A and B in DP1 to form page DP2. The component adder 340 may reconcile the changes made to DP2 and MP2. The parent / ancestor lookup unit 344 may perform a parent / ancestor lookup for component X and find that it follows A (the closest ancestor) in the page containing X (the closest parent in the desktop layout configuration). Thus, the mobile layout adder 346 may insert X into MP2 in the page containing X, following component A (here below B instead of above B), to form an updated mobile page MP3.

[0201] In yet another example, as shown in FIG. 28C, referenced herein, mobile page MP1 is edited via mobile editor 560 to remove (hide) component B below component A and above component C (forming MP2). DP1 is then edited via desktop editor 550 to insert component X after B and before C (forming DP2). Component adder 340 may then reconcile the changes to DP2 and MP2. Parent / ancestor lookup unit 344 may perform a parent / ancestor lookup of added component X and determine that X's ancestor is A, not B, because B is marked as "removed in mobile". Thus, mobile layout adder inserts X after A (and before C) in MP2 to form MP3.

[0202] In yet another example, as shown in FIG. 28D, which is referred to herein, a page DP1 includes a container A that includes an inner container B and another component D, all of which are converted to MP1. The mobile page MP1 is then edited via the mobile editor 560 to remove (hide) the inner container B (forming MP2). The DP1 is then edited via the desktop to insert a component C inside the inner container B (forming DP2). The component adder 340 may then reconcile the changes to DP2 and MP2. In such an example, when performing a parent / ancestor lookup of the added component C, the parent / ancestor lookup unit 344 may determine that the parent is A and not B, since B is marked as "removed in mobile". The parent / ancestor lookup unit 344 may further determine that C has no ancestors, since C is the first component in A (without considering B). Thus, the mobile layout adder 346 may insert C within A (at the top and before D) in MP2, forming MP3.

[0203] In yet another example, as shown in FIG. 28E, to which reference is now made, mobile page MP1 is edited via mobile editor 560 to swap the vertical positions of A and B to form MP2. The designer then inserts two new largely overlapping components X1 and X2 between A and B in DP1 to form page DP2. Component adder 340 may reconcile the changes to DP2 and MP2 in one of two possible ways to generate MP3 (corresponding to the one-by-one method) or MP4 (corresponding to the clustering method).

[0204] It will be appreciated that if a one-by-one method is used and X2 is geometrically positioned "later" than X1, the parent / ancestor lookup unit 344 may first perform a parent / ancestor lookup of X1 and determine that A is an ancestor of X1. The parent / ancestor lookup unit 344 may then perform a parent / ancestor lookup of X2 (taking X1 into account) and determine that X1 is an ancestor of X2. Thus, the mobile layout adder 346 may in this case position component X1 after A and component X2 after X1 to form MP3. In such a method, components X1 and X2 are analyzed and moved as separate entities, so that their compositions are separate.

[0205] When using the clustering method, the parent / ancestor lookup unit 344 may perform a parent / ancestor lookup of both X1 and X2 without considering X1 or X2 (for the purpose of the lookup) and may determine that both X1 and X2 have the same parent (the main page) and the same ancestor (component A). This may cause the component adder 340 to cluster X1 and X2 together, form a virtual supernode that includes X1 and X2, and instruct the initial layout converter 200 to form a mobile ready version of the composition of X1 and X2. It will be appreciated that such a converted composition may be positioned under A and an MP4 may be formed. Because X1 and X2 were significantly overlapping, X1 and X2 may be resized (as described above) to maintain their composition.

[0206] In yet another example, as shown in FIG. 28F referred to herein, an original desktop page DP1 including two components (A and B) is edited via the desktop editor 550 to add four new components (two pairs of semantically related text and image components (T1 / I1 and T2 / I2)) to form a new page DP2. If DP2 is converted to mobile using the one-by-one method (described above in this specification), the parent / ancestor lookup unit 344 may determine that T1 is an ancestor of T2. Thus, the (vertical) order of components generated may be A-T1-T2-I1-I2-B (as shown in MP2), and the connection between the text description and the image is broken. If DP2 is converted to mobile using the clustering method (described above in this specification), the parent / ancestor lookup unit 344 may determine that T1 / T2 / I1 / I2 all have the same ancestor (A) and parent (main page). Therefore, the mobile layout adder 346 may cluster these four components in a new virtual page and run the initial layout converter 200 on these four components. Because T1 is semantically related to I1 and T2 is semantically related to I2, these pairs may be kept together. Thus, the (vertical) order formed may be A-T1-I1-T2-IB (as in an MP3), preserving the semantic information.

[0207] In yet another example, as shown in FIG. 28G, which is referred to herein, a desktop page DP1 includes three components (A-C) and is converted to mobile. The resulting mobile page MP1 is edited via the mobile editor 560 to remove component C and swap the order of A and B to form MP2. DP1 is then edited via the desktop editor 550 to add two overlapping components X1 and X2 between B and C, and add an additional component X3 below C to form DP2. Since the component adder 340 is configured to use a clustering method, the three added components (X1, X2, and X3) are clustered together because the parent / ancestor lookup unit 344 has determined that B is an ancestor of all three new components (because C has been removed in the mobile version). Thus, the mobile layout adder 346 may position all three new components below B and above A while preserving the composition of X1 and X2 (as shown in MP3).

[0208] It will be appreciated that positioning of the additional component by the mobile layout adder 346 may be performed in a limited version where the component is added to the desktop version or added vertically between two vertically separated components to the mobile version only.

[0209] In other embodiments, the component adder 340 may add a "cluster" of multiple additional components anywhere inside the "line" of components that spans the mobile version, such that the insertion of new components is not limited to below a given component in the mobile version.

[0210] In yet another embodiment, the component adder 340 may paste the additional component B into a container in the desktop layout configuration that contains the additional component B rather than having the additional component B as an ancestor. In such a configuration, it will be understood that component B remains pasted into the container C that contains component B, and is positioned after the last element in container C that exists in both the desktop layout configuration and the mobile layout configuration and that precedes component B. This approach may be used as long as container C (and the corresponding supernode) is not empty or is not hidden in the mobile layout configuration.

[0211] In yet other embodiments, the component adder 340 may add components to the mobile layout composition by dividing the mobile layout composition horizontally into segments and adding the component or component cluster to be added after the segment that contains the ancestor, rather than directly after the ancestor.

[0212] It will be appreciated that the component modifier 350 may modify components in the mobile layout configuration when there are changes to attributes such as style, font, or text size of the desktop components, including changes to general properties such as changing the number of rows or columns in a gallery component. Because the mobile layout configuration defines a visual application subject to dynamic layouts, the mobile layout configuration can accommodate such changes and readjust component sizes and positions appropriately.

[0213] It will also be appreciated that while changes to the size and position of desktop components affect the desktop layout configuration, they do not affect the mobile layout configuration unless the mobile layout configuration is explicitly regenerated as described herein below. This is permitted because the system 100 allows for aesthetic differences between multiple layout configurations as long as the content is not changed.

[0214] When there are changes to the data / content of the desktop components, the component modifier 350 can update the mobile layout configuration accordingly. Similar to modifications to attributes, such data changes can cause reformatting of components on the mobile layout configuration that are manipulated using dynamic layouts.

[0215] It will be appreciated that any modifications to the mobile layout configuration may be made by the user, typically via a dedicated GUI view (mobile editor 560) or a mobile modification view having a mobile preview layout configuration. Such a mobile editor 560 may only allow the designer to perform certain operations such as adding mobile-only components, removing components (hiding components), moving components, or only modifying components. Thus, the mobile editor 560 may be correlated with the mobile controls 360 that handle the mobile-side only changes.

[0216] The mobile operator 360 may use a number of methods for positioning the (mobile only) add component. The mobile operator 360 may only allow insertion of a new component between two components on a component line, or on a new component line that follows or precedes an existing component line, as shown in FIG. 29, referenced herein. A new component X may be inserted between A and B, or a new component Y may be inserted below P and above A, B, and C. The mobile operator 360 may also insert line breaks before / after the add component.

[0217] In this case, the inserted component (X or Y) may be anchored to the preceding component (or container) and may be reinserted at said location if the mobile layout composition is regenerated and it is necessary to "keep the added mobile component." If an anchor is not available (e.g., if the anchored-to component has been removed from within the desktop layout composition), the mobile manipulation unit 360 may also search for the closest previous component or container as described above with respect to adding a component to the desktop layout composition.

[0218] The mobile operator 360 may also allow a new (mobile only) component X to be located anywhere in the page. In this case, the mobile operator 360 may search for an anchor component by searching for the component that most intersects with X. If no such component is available, the mobile operator 360 may search for the closest neighboring component (on all four sides) based on a weighted average of distance and overlap, and by using distance and overlap thresholds. If no such component is found in the container containing X, the mobile operator 360 may use the start of the page as the anchor, which may be defined by the distance and offset from the nearest edge of the anchor.

[0219] To avoid non-compliance with search engine policies, the reconversion converter 300 may completely avoid operations that would not comply with search engine policies by the reconversion converter 300. Alternatively, the reconversion converter 300 may be limited to adding components that do not affect the search engine view of the page (e.g., decoration type components (e.g., lines, decorative shapes)), components that have no content (e.g., in-page navigation menus that do not add extra menu entry information), components that have content that is not provided to the search engine (e.g., background images), and components that are only relevant to a mobile environment (e.g., buttons that provide functionality to make a phone call, send a text / MMS message, navigate to a location, or switch to a desktop site version).

[0220] It will also be appreciated that the mobile editor 560 may initially position the deprecated monents in a "removed components" display, requiring a second step to actually incorporate the deprecated monents into the mobile layout composition.

[0221] The additional components may be listed in a mobile revision display as described herein below, to facilitate removal of components that are added only to a particular mobile.

[0222] The mobile manipulator 360 may also remove components from the mobile layout configuration (this removal may be performed by actually removing the components or simply hiding them). It will be appreciated that when components are removed from the mobile layout configuration, the mobile manipulator 360 does not reposition the components in the same line. When an entire component line is removed, the mobile manipulator 360 may move the lower components up as close as possible, as shown in FIG. 30, referred to herein. If any of components A, B, or C (but not all three) are removed, the remaining components in the line (A, B, or C) are not affected or moved. If all three components (A, B, and C) are removed, component Q and the components following component Q are moved up towards the adjacent component P. The removed components are listed in the mobile revision display, as described herein below, to facilitate reinserting the hidden components into the application, if necessary at a later stage.

[0223] It will also be appreciated that components within a mobile layout composition may be moved by the user via the normal editing visual data system GUI, and the mobile manipulation unit 360 may retain the modifications until the page is recomposed, and may list the modifications in the mobile modification display, as described herein below, to facilitate undoing a particular change.

[0224] The mobile editor 560 may also allow components to be moved "out" (right or left) of the mobile's narrow display strip. In this example, the component may maintain a certain minimum horizontal overlap with the "strip" and may be displayed clipped to the strip size. Otherwise, this option may be equivalent to a "hide component" option. This option may be used to show only a portion of a component / container, or to temporarily position a component aside to aid in repositioning the component in a mobile configuration.

[0225] The mobile editor may also allow a user to selectively change attributes, properties, and styles of components in the mobile layout composition. The mobile editor may mark such changed attributes as "changed in mobile" so that changes to a particular attribute in the desktop layout composition may not affect (and may not override) changes to such attributes in the mobile layout composition.

[0226] One specific change that may be desirable as a change specific to a mobile layout configuration is font size. Font size changes may be required if the component adjuster 227 generates a different size than the designer may have requested. Thus, a specific font size change may be added and applied to the display text for each field (as a factor applied in addition to the font size generated by the initial layout converter 200). The mobile editor 560 may change the font size using a local "scale factor" attribute and (for example) 5% font size increase / decrease buttons that may affect such attribute. The mobile editor 560 may further limit the new font size to a predefined range of font sizes suitable for mobile. Modifications of any attribute (including font size changes) may be listed in a mobile modification display described herein below.

[0227] As described above in this specification, the reconversion converter 300 may provide a separate GUI that displays a list of modifications (e.g., component additions, component removals, and component modifications) specific to a given mobile layout configuration. Such a GUI may be useful for viewing the modifications of a particular mobile layout configuration and distinguishing them from modifications of other layouts made by the initial layout converter 200. The GUI may also be useful for viewing various changes to various mobile layout configurations and being able to undo modifications of a particular mobile layout configuration (e.g., reinserting a particular "removed" component).

[0228] It will be appreciated that such a display may be divided according to the page on which the modification was made (i.e., the current page, another page, any template, or the master page) or according to the modification type (i.e., insertion, removal, or modification of a component).

[0229] The display may further be used to categorize these modifications according to timestamp, location within the page, or affected component type, etc. The mobile editor may support an undo operation for a selected set of modifications, in which case the mobile operation 360 may undo any modifications (e.g., reinsertion of a removed component is a modification of the reverse component position).

[0230] The reconversion converter 300 may further support a regeneration option for the mobile layout configuration: when activated, the reconversion converter 300 may instruct the initial layout converter 200 to again operate on the underlying desktop layout configuration (which may have been modified since the initial layout converter 200 last operated on), potentially reapplying changes specific to the mobile layout configuration.

[0231] It will be appreciated that regeneration may apply to pages that are to be regenerated (the current page, a specific page, the entire application) and to mobile layout configuration changes that are to be reapplied after the mobile layout configuration is regenerated. These mobile layout configuration changes may be selected by category (e.g., only hiding components) and may include specific selected modifications or all modifications. It will also be appreciated that if a regeneration operation is performed without reapplying the mobile layout configuration changes to a given page or pages, the associated mobile layout configuration changes are discarded.

[0232] Display 290 may display the new updated mobile layout configuration as described herein above.

[0233] A system such as the one described above may be integrated with an associated website building system in many ways. For example, the system 100 may be fully integrated on the client side, fully integrated on the server side, or integrated both on the client side and the server side within the actual website building system code. The system 100 may also run on the same server platform as the website building system, or on a separate and independent server or set of servers. The system 100 may run online (whenever the website building system is browsed), offline (to form a stored transformed version of the website building system), or a combination thereof (offline at some stages and online at other stages). The system 100 may operate directly on the website building system's data structures, or may provide interfaces (such as web services) that are called by the website building system in a manner that is independent of the website building system's content.

[0234] It will be appreciated that in various embodiments of the present invention, the various elements of the system 100 may divide their functions in various ways and perform the functions described above in a different order than as described above in this specification. Furthermore, it will be appreciated that the system 100 may also be partially implemented using partial functions.

[0235] Thus, a desktop layout configuration of a specific application can be appropriately transformed into a mobile layout configuration (or any other desired layout of various sizes) taking into account the relationships between components. Furthermore, after the initial transformation, modifications can be made to the same page in both the desktop and mobile layout configurations and reflected in the newly generated mobile page.

[0236] Unless specifically stated otherwise, as is evident from the above discussion, discussions of the use of terms such as "processing," "computing," "calculating," "determining," and the like throughout this specification will be understood to refer to the operations and / or processing of a computer, computing system, client / server system, or similar electronic computing device that manipulates and / or transforms data represented in physical quantities, such as electronic quantities in the computing system's registers and / or memory, into other data similarly represented as physical quantities in the computing system's memory, registers, or other device that stores, transfers, or displays such information.

[0237] An embodiment of the present invention may include an apparatus for performing the operations herein. This apparatus may be specially configured for the desired purposes or may comprise a general purpose computer selectively operated or reconfigured by a computer program stored in the computer. The configured apparatus may transform a general purpose computer into the inventive elements described herein when instructed by software. The instructions may define an inventive apparatus that operates in conjunction with a computer platform that is desirable for the inventive apparatus. Such a computer program may be stored on any type of computer readable storage medium, such as, but not limited to, a floppy disk, an optical disk, a magnetic optical disk, a read-only memory (ROM), a compact disk read-only memory (CD-ROM), a random access memory (RAM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a magnetic or optical card, a flash memory, a disk-on-key, or any other type of medium suitable for storing electronic instructions and capable of being coupled to a computer system bus.

[0238] The processes and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs according to the techniques herein, or it may be convenient to configure more specialized apparatus to perform the desired methods. The desired structure for a variety of these systems will appear from the following description. Further, embodiments of the present invention are not described with reference to any particular programming language. It will be understood that a variety of programming languages ​​may be used to implement the techniques of the present invention described herein.

[0239] While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art, and it is to be understood, therefore, that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.

[0240] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 61 / 876,795, filed Sep. 12, 2013, which is incorporated by reference in its entirety.

Claims

1. 1. A method for maintaining and creating a first layout definition and a second layout definition for a website page having multiple components between display areas of different sizes for a given application, comprising: determining geometric and semantic relationships between the components of the website page on a first display area; collecting edit session history information and detecting component changes and component relationships of the plurality of components; analyzing anchors between displayed components of the first display area, the anchors controlling layout changes made to the components; generating at least one display order for the plurality of components according to the determining, collecting, analyzing and content of the plurality of components; Transforming a display of components from the first display area to a second display area and maintaining the display order according to generating the at least one display order; A method for providing the above.

2. The method of claim 1 , wherein the components are at least one of an atomic, a single-page container, and a multi-page container.

3. The method of claim 1 , further comprising at least one of removing and hiding components that are not suitable for display in the second display area.

4. The converting step comprises: Modifying instances of the plurality of components in a template; fitting the plurality of components to the second display area; and arranging the plurality of components within the second display area based on at least one of the determined sequence, reserved space, and grouping of the plurality of components; adjusting the arranged plurality of components prior to displaying them in the second display area; and The method of claim 1 , comprising:

5. 5. The method of claim 4, wherein the grouping comprises locating a group from among the plurality of components that should remain together on the second display area, and creating a hierarchy of supernodes having sub-elements to represent the components of the located group according to the positions of the components and content relationships of the components.

6. The grouping may include: locating groups of the plurality of components having significant component overlap and replacing the groups with virtual elements according to a grouping criterion; locating a group of the plurality of components according to at least one hint at a template, application, page, or component level and replacing the group with a virtual element according to a grouping criterion; Locating components that are text components that overlap a particular background image and replacing said groups with virtual elements according to said grouping criteria; determining a confidence score for the accuracy of locating the group of components where the components significantly overlap, locating the group of components according to at least one of template, application, page, or component level hints, and locating components that are text components that overlap the particular background image; creating a hierarchy of supernodes according to the plurality of components and virtual elements; The method of claim 4 , comprising at least one of:

7. The placing comprises: interpreting hints attached to elements of said supernode; creating line breaks according to the hints created by locating the groups and creating a hierarchy of supernodes and content relationships of the plurality of components; applying at least one of a width adjustment and a height adjustment to the elements of the supernode; Resizing the decorative image; The method of claim 6 comprising:

8. 2. The method of claim 1, further comprising: receiving modifications to the first layout definition and the second layout definition; and generating an updated second layout definition for the second display area while maintaining the at least one display order and at least one of the geometric and semantic relationships.

9. Generating the at least one display order includes: determining an order of said sub-elements according to a predefined ordering criterion; analyzing at least one of the semantics, content, attributes, edit history, and geometry of the subelements of the supernode; determining a confidence score for the accuracy of determining the order of the sub-elements of the supernode according to the predefined ordering criteria and analyzing at least one of the semantics, content, attributes, edit history, and geometry of the sub-elements of the supernode; determining an order of the sub-elements of the supernode according to the predefined order criteria and integrating the order determined by determining the confidence score for the accuracy of analyzing at least one of the semantics, content, attributes, edit history, and geometry of the sub-elements of the supernode to create a merged revised order; The method of claim 5 comprising:

10. The adjusting step comprises: Inserting automatically added mobile-related components; performing at least one of creating, modifying, and deleting dynamic layout anchors, the anchors being adjusted in response to the second display area; The method of claim 4 comprising at least one of:

11. The adapting step comprises: Modifying the sizes and widths of the components; Integrating the menu component; Updating the content-related adaptations; and Creating a composite menu; Converting character-based graphics; and The method of claim 4 comprising:

12. generating the updated layout definition, deleting a page from the second layout definition when the page is deleted from the first layout definition; adding a page to the second layout definition when the page is added to the first layout definition; removing a component from the second layout definition if the component is removed from the first layout definition; adding a component to the second layout definition when the component is added to the second layout definition; modifying a component from the second layout definition when the component is modified in the first layout definition; manipulating modifications to the second layout definition, the modifications being independent of modifications to the first layout definition; and The method of claim 8 comprising:

Images