Text-based layout generation and page generation

Abstract

According to the implementation of the invention, a scheme for page generation is provided. In the scheme, a first part of a target page to be generated can be provided, the layout of the first part is determined based on an input sequence corresponding to a first description text, and the input sequence is generated based on the type of at least one layout constraint indicated by the first description text. Further, a second description text for a second portion of the target page can be obtained. Accordingly, a first set of candidate layouts for the second portion may be provided, where the first set of candidate layouts is determined based at least on the second description text. Therefore, the page generation method and device can support the page generation process according to the regions, and therefore the page generation efficiency is improved.

Description

Background Art

[0001] In recent years, layout design is an important task in content creation. For example, people need to determine the size and dimensions of each element to complete the desired page layout. Such a process is a complex task that relies on professional design capabilities.

[0002] This makes it difficult for ordinary users to complete such layout design independently. They usually need to spend a lot of time and communicate with professional designers, which greatly reduces the efficiency of layout design and page production. Summary of the invention

[0003] According to the implementation of the present disclosure, a scheme for page generation is provided. In the scheme, a first part of a target page to be generated can be provided, and the layout of the first part is determined based on an input sequence corresponding to a first description text, and the input sequence is generated based on the type of at least one layout constraint indicated by the first description text. Further, a second description text for a second part of the target page can be obtained. Accordingly, a first group of candidate layouts for the second part can be provided, wherein the first group of candidate layouts is determined based on at least the second description text. Thus, the implementation of the present disclosure can support a page generation process by region, thereby improving the efficiency of page generation. In addition, by constructing an input sequence using the constraint type indicated by the text description, the implementation of the present disclosure can also improve the accuracy of layout generation.

[0004] The Summary is provided to introduce the concepts in a simplified form, which will be further described in the Detailed Description below. The Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A schematic diagram illustrating an example environment according to some implementations of the present disclosure is shown;

[0006] Figure 2 A schematic diagram showing an example page generation system for generating a page layout according to some implementations of the present disclosure is shown;

[0007] Figure 3A and Figure 3B An example interface for page generation according to some implementations of the present disclosure is shown;

[0008] Figure 4 A flowchart illustrating an example method for page generation according to some implementations of the present disclosure; and

[0009] Figure 5A block diagram of an example computing device is shown in accordance with some implementations of the present disclosure.

[0010] In these drawings, the same or similar reference symbols are used to designate the same or similar elements. DETAILED DESCRIPTION

[0011] The present disclosure will now be discussed with reference to several example implementations. It should be understood that these implementations are discussed only to enable those skilled in the art to better understand and thus implement the present disclosure, and do not imply any limitation on the scope of the present subject matter.

[0012] As used herein, the term "including" and variations thereof are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "based at least in part on". The terms "an implementation" and "an implementation" are to be interpreted as "at least one implementation". The term "another implementation" is to be interpreted as "at least one other implementation". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0013] As discussed above, in page production or design, layout design is a complex task that relies on professional knowledge. Therefore, how to improve the efficiency of layout design has become the focus of people's attention.

[0014] According to the implementation of the present disclosure, a scheme for page generation is provided. In the scheme, a first part of a target page to be generated can be provided, and the layout of the first part is determined based on an input sequence corresponding to a first description text, and the input sequence is generated based on the type of at least one layout constraint indicated by the first description text. Further, a second description text for a second part of the target page can be obtained. Accordingly, a first group of candidate layouts for the second part can be provided, wherein the first group of candidate layouts is determined based on at least the second description text. Thus, the implementation of the present disclosure can support a page generation process by region, thereby improving the efficiency of page generation. In addition, by constructing an input sequence using the constraint type indicated by the text description, the implementation of the present disclosure can also improve the accuracy of layout generation.

[0015] The basic principles and several example implementations of the present disclosure are explained below with reference to the accompanying drawings.

[0016] Example Environment

[0017] Figure 1 An example environment 100 is shown according to some implementations of the present disclosure. Figure 1 As shown, the environment 100 may include an electronic device 110. Such an electronic device 110 may be configured to provide an interface 130 for page authoring or page generation, for example.

[0018] like Figure 1 As shown, the electronic device 110 can, for example, use the interface 130 to obtain a description text 140 about the page to be generated, such as "A header". It should be understood that the user can use an appropriate input method to input the description text 140 to the electronic device 110. For example, the user can use an input device to type text, or can input audio information to be converted into the description text 140. The present disclosure is not intended to limit the method for obtaining the description text 140.

[0019] Further, the electronic device 110 may provide a corresponding page layout 150 based on the description text 140. In some implementations, the electronic device 110 may generate the page layout 150 based on processing the description text 140. Alternatively, the electronic device 110 may also send the description text 140 to an appropriate remote device 120 (e.g., a server), and receive from the remote device 120 the page layout 150 determined by the remote device 120 based on the description text 140.

[0020] The specific generation process of the page layout will be referred to below Figure 2 Detailed description.

[0021] Furthermore, the electronic device 110 may provide the user with the generated page layout 150 by using the interface 130. Such a page layout 150 may indicate information such as the position and size of one or more page elements in the page.

[0022] For example, the user can also use the interface 130 to modify, replace, and other operations on the provided page layout 150, and use it to generate the final page. The page generation process will be referred to below. Figure 3A and Figure 3B Detailed description.

[0023] Layout Generation

[0024] Figure 2 FIG. 2 is a schematic diagram of an example layout generation system 200 for generating a page layout according to some implementations of the present disclosure. In some implementations, the generation system 200 may be configured by reference to FIG. Figure 1 The electronic device 110, the remote device 120, and / or a combination thereof are shown to implement.

[0025] like Figure 2 As shown, the layout generation system 200 may include a parsing module 210 and a placement module 220. The parsing module 210 may be configured to obtain the description text 140 and generate an intermediate representation (IR) 215 for the description text 140.

[0026] In some implementations, such description text 140 may be used to indicate at least one layout constraint associated with the page to be generated. For example, “A header” may indicate that the page includes a page element of type “header”.

[0027] Considering that users may have different habits for describing text 140, parsing module 210 may convert the describing text 140 into an intermediate representation 215 represented by a formal language. Formal language is intended to represent a language defined by precise mathematical or machine-processable formulas. In some implementations, the formal language used to represent the intermediate representation may have, for example, a limited character set and predetermined grammatical rules.

[0028] Such an intermediate representation 215 can describe at least one layout constraint indicated by the text 140 in a formal language, thereby improving the standardization of the representation of the layout constraint and improving the accuracy and generation efficiency of the subsequent layout generation process. In some implementations, the parsing module 210 can generate corresponding intermediate representations according to different types of description texts 140.

[0029] In some examples, the description text 140 may indicate the function of the corresponding page element. For example, the description text may include "a news heading". Accordingly, the parsing module 210 may convert it into an intermediate representation "[e:title]".

[0030] As another example, the description text 140 may also indicate the absolute position and / or relative position of the corresponding page element. For example, the description text 140 may include "a news heading at the top". Accordingly, the parsing module 210 may convert it into an intermediate representation "[e:title[prop:position"top"]]".

[0031] As another example, the description text may also indicate the relative size and / or absolute size of the corresponding page element. For example, the description text 140 may include "a brief news summary". Accordingly, the parsing module 210 may convert it into an intermediate representation "[e:description[prop:size"small"]]".

[0032] As another example, the description text may also indicate the hierarchy of the corresponding page element. For example, the description text 140 may include "3news pieces. Each has a title and summary". Accordingly, the parsing module 210 may convert it into an intermediate representation "[group[prop:repeat "3"][item[e:title][e:description]]]".

[0033] Based on the above examples, it can be seen that the parsing module 210 can convert different types of description texts into intermediate representations in a structured style. In this way, on the one hand, the implementation of the present disclosure can increase the degree of freedom of the text description input by the user, making it convenient for the user to input the text description according to habit. On the other hand, by converting the text description expressed in natural language into a structured description, the implementation of the present disclosure can also improve the accuracy of subsequent layout generation.

[0034] In some implementations, the parsing module 210 may be implemented using a suitable language model. Such a language model may include, for example, a pre-trained language model (PLM) based on a transformer, which may include, for example, an encoder and a decoder.

[0035] Specifically, the encoder of the model can take the description text as input, which can be represented as x = {x 1 , …, x n}, where x i Represents the i-th token in the description text. Furthermore, the encoder can convert the input token into an expression with context information Correspondingly, the decoder of the model can autoregressively predict the tokens of the intermediate representation IR, which can be expressed as P θ (z j |z ＜j , x). In a specific example, the decoder of the model can also generate IR tokens z according to the generated IR tokens z. <j and the encoder output h x To get the token of intermediate representation IR.

[0036] In some examples, in the process of training a language model, the parameters θ of the language model can be trained by minimizing the negative log-likelihood of the labeled IR z given the text input x, which can be expressed as:

[0037]

[0038] It should be understood that other appropriate language models may also be used to implement the parsing module 210, and the present disclosure is not intended to limit the specific architecture or specific training process of the language model.

[0039] Further, the placement module 220 may be configured to determine the page layout 150 according to the received intermediate representation 215. In some implementations, the generation process from the intermediate representation 215 to the page layout 150 may be converted into a sequence-to-sequence conversion task.

[0040] To improve the accuracy of layout generation, placement module 220 may determine the input sequence based on intermediate representation 215. In some implementations, placement module 220 may determine the corresponding input sequence according to the type of layout constraints indicated in intermediate representation 215.

[0041] In some implementations, depending on the difference of the page elements targeted by the description text 140 , the corresponding layout constraints may be processed differently.

[0042] In one example, the description text 140 may indicate a first type of constraint, which indicates, for example, a layout constraint for a single page element. Such a layout constraint is also referred to as an element-level constraint or a point-level constraint. For example, the description text "a large text box" is intended to indicate a type layout constraint, a position layout constraint, and a size layout constraint about a single page element.

[0043] In another example, the description text 140 may indicate a second type of constraint, which indicates, for example, a layout constraint between two page elements. Such a layout constraint is also referred to as a pairwise constraint. For example, the description text "an image below the text box" is intended to indicate a relative position layout constraint between two page elements.

[0044] In another example, the description text 140 may indicate a third type of constraint, which indicates, for example, a layout constraint about a group of page elements. Such a layout constraint is also referred to as a groupwise constraint. For example, the description text "a tool bar with 4 icons" is intended to indicate a hierarchical layout constraint of a layout.

[0045] Additionally, the placement module 220 may generate a corresponding input sequence according to the type of layout constraint.

[0046] In one example, the placement module 220 may use a specific token k to represent the first type of constraint, where k∈∑, ∑ represents a vocabulary constructed by the type of layout constraint. For example, the vocabulary of the type layout constraint for the page element may be represented as ∑ type={image, ..., text}. Further, multiple first-type constraints for a single page element can be represented as a token sequence: C po ={k 1 …k l}. For example, "a large image on the left" can be expressed as "image large left".

[0047] Specifically, the placement module 220 may identify the first type of constraint from the intermediate representation IR, and generate an input sequence corresponding to the first type of constraint according to the input sequence representation method corresponding to the first type of constraint.

[0048] Taking the intermediate representation "[e:title[prop:position"top"]]" as an example, the placement module 220 can determine that the intermediate representation IR is a constraint for a single element, which includes a type constraint "title" and a position constraint "top". Therefore, the placement module 220 can extract the corresponding constraint information from the intermediate representation and compare it with the vocabulary set to generate a corresponding token sequence, for example, "title top".

[0049] In another example, the placement module 220 may represent the second type constraint as three tokens, namely, C pa ={si, k, gi}, where k∈Σ and indicates a relationship, si and gi represent the indexes of the source page element and the target page element in the final input sequence. For example, “an image below a text box” can be represented as “text-1bottom image-1”.

[0050] Specifically, the placement module 220 may identify the second type constraint from the intermediate representation IR, and generate an input sequence corresponding to the second type constraint according to an input sequence representation method corresponding to the second type constraint.

[0051] For example, the placement module 220 may identify a specific token (e.g., bottom) representing a positional relationship from the intermediate representation IR and determine that it corresponds to the second type constraint. Further, the placement module 220 may extract the corresponding page element and determine its index information, thereby converting the intermediate representation IR into a corresponding input sequence.

[0052] In another example, the placement module 220 may use special tokens to indicate the third type of constraint. For example, the placement module 220 may use brackets to indicate an element group C. gpFor example, "a toolbar with 4 icons" can be expressed as "[icon|icon|icon|icon]".

[0053] Furthermore, the placement module 220 may, for example, use a separator “|” to cascade the subsequences of the multiple page elements corresponding to the intermediate representation. This process may, for example, be expressed as:

[0054]

[0055] Specifically, the placement module 220 may identify the third type constraint from the intermediate representation IR, and generate an input sequence corresponding to the third type constraint according to the input sequence representation method corresponding to the third type constraint.

[0056] Taking the intermediate representation "[group[prop:repeat"3"][item[e:title][e:description]]]" as an example, the placement module 220 can determine that the intermediate representation corresponds to the third type constraint by detecting the group class token (e.g., group). Further, the placement module 220 can extract the corresponding page element from the intermediate representation and generate an input sequence as shown in formula (2).

[0057] Thus, the placement module 220 can convert the intermediate representation 215 into an input sequence s for use in a subsequent generation process.

[0058] In some implementations, the placement module 220 can convert the input sequence s into an output sequence y to indicate the generated page layout.

[0059] In some implementations, the output sequence y may include an element representation corresponding to a set of page elements. Where m represents the number of page elements in the layout. In some implementations, the element representation corresponding to each page element may include multiple tags (also referred to as attributes).

[0060] For example, the element represents e i Can include marker c i , to indicate the type of the page element. Additionally or alternatively, the element represents e i You can include the tag l i , to indicate the left coordinate of the page element. Additionally or alternatively, the element represents e i You can include the tag t i , to indicate the top coordinate of the page element. It should be understood that l i and t i Can be used to indicate the position of the page element in the page.

[0061] Additionally or alternatively, the element represents e i Can include the tag w i , to indicate the width of the page element. Additionally or alternatively, the element representation ei may include a tag h i , to indicate the height of the page element. It should be understood that w i and h i Can be used to indicate the size of the page element on the page.

[0062] In some implementations, to facilitate placement module 220 to complete page elements not indicated in description text 140, element representation e i Can include marker a i , to indicate whether the corresponding page element is indicated in the description text. For example, the tag a i The value may be {indication, supplement} to indicate whether the page element is additionally added by the placement module 220 .

[0063] Furthermore, the placement module 220 may, for example, use a separator “|” to concatenate the element representations of multiple elements. Thus, the output sequence y for indicating the page layout may be expressed as:

[0064] y={a 1 c 1 l 1 t 1 w 1 h 1 |…|a m c m l m t m w m h m} (3)

[0065] In some implementations, the placement module 220 may utilize an appropriate machine learning model to implement the conversion from the input sequence s to the output sequence y. For example, the placement module 220 may be implemented using a transformer-based model, which may include an encoder and a decoder.

[0066] In some implementations, such as Figure 2 As shown, the machine learning model for implementing the placement module 220 can be trained using the training data set 230. In some implementations, such a training data set 230 may include a labeled data portion, which may be generated by, for example, labeling a set of training pages. For example, a labeler may label a set of training pages to generate a set of description texts corresponding thereto, and the output sequence corresponding to the set of training pages may be obtained based on, for example, automatic analysis of the set of training pages.

[0067] In such an annotation process, the annotator may omit the description of some page elements in the page according to personal habits. Thus, the annotation data provided by the annotator may include a description text for the training page, and such a description text may correspond to only a subset of all page elements in the corresponding training page. For example, if a page includes four buttons and a top picture, the annotator may provide a description text of "a page including four buttons" and omit the description of the page element "top picture".

[0068] By using such a training data set 230 to train the machine learning model, the placement module 220 can learn the ability to expand other appropriate page elements based on the description of limited page elements. For example, in the case where the text description input by the user only includes four buttons, the placement module 220 can provide a layout that also includes a top picture in addition to the four buttons. Through such an annotation method, the embodiments of the present disclosure can also reduce the annotation cost of the annotation personnel, so that the annotation personnel do not have to pay attention to all the page elements in the page, thereby improving the efficiency of annotation.

[0069] In some implementations, the training data set 230 may also include an unlabeled data portion, for example. Such an unlabeled data portion may be automatically generated based on a set of training pages, for example.

[0070] Specifically, the unlabeled data portion can be generated by an appropriate electronic device. For example, the electronic device can obtain a set of training pages, and generate a training intermediate representation corresponding to the set of training pages based on the page layout of the set of training pages. Exemplarily, the electronic device can first extract the page elements in the set of training pages, and further extract the third type of constraints (group level constraints), the second type of constraints (pair level constraints) and the first type of constraints (point level constraints) between the page elements at one time, so as to construct an intermediate representation corresponding to the set of training pages. On the other hand, the electronic device can also determine the training output sequence corresponding to the set of training pages. Thus, the electronic device can construct the unlabeled data portion in the training data set 230 based on the training intermediate representation and the training output sequence.

[0071] Based on such a process, the embodiments of the present disclosure can greatly reduce the annotation cost and utilize a large number of existing pages to automatically generate annotation data, thereby reducing the production cost of the annotation data and improving the efficiency of model training.

[0072] Based on the process discussed above, the embodiments of the present disclosure can automatically create a page layout based on the description text, thereby improving the efficiency of layout generation. In addition, by constructing an input sequence based on the type of layout constraints, the embodiments of the present disclosure can improve the accuracy of layout generation.

[0073] Page Generation

[0074] In some implementations, such as reference Figure 1 As discussed, the electronic device 110 also supports the user to design or create pages through an interface.

[0075] Figure 3A An example interface 300A is shown according to some example implementations of the present disclosure. Figure 3A As shown, the interface 300A may include an input area 302 for describing text, a preview area 304 for page layouts, and a selection area 306 for candidate layouts.

[0076] For example, the input area 302 may include a text input box for receiving a description text 320 input by a user, such as "A header". Figure 1 As discussed, the user may, for example, utilize any suitable means to input the description text 320, including but not limited to: text entry utilizing an input device, voice-based audio transcription, and the like.

[0077] Furthermore, the interface 300A may include, for example, a generation control 325. Upon receiving a trigger for the generation control 325, the electronic device 110 may trigger the generation and provision of at least one page layout. The generation process of the page layout may refer to the above description, which will not be repeated here.

[0078] Exemplarily, the electronic device 110 may provide the candidate layout 310 in the preview area. Additionally, the electronic device 110 may also provide one or more other selectable candidate layouts 335 in the selection area 306. The user may replace the selected candidate layout 310 in the preview area 304 by clicking on the candidate layout 335, for example.

[0079] The candidate layout 310 and one or more other candidate layouts 335 may be generated, for example, using the single or multiple layout generation processes discussed above, and may have a certain degree of generation randomness, thereby presenting different layout styles.

[0080] Additionally, the user may also trigger, for example, the generation of a new set of candidate layouts based on the description text 320. For example, if the user is not satisfied with the provided candidate layouts 310 and 335, the generation and provision of a new set of candidate layouts may also be triggered.

[0081] Furthermore, the user may modify the provided candidate layout 310. For example, the user may use the interface 300A to adjust the type, size, position, and other information of one or more page elements in the candidate layout 310.

[0082] In some implementations, the electronic device 110 may, for example, support the user to generate pages in stages. For example, the description text 320 may, for example, be the first part for generating a page. After the user determines the page layout of the first part, the user may, for example, continue to design other parts of the page.

[0083] For example, Figure 3A As shown, the interface 300A may include an add control 330 for adding the second part of the page. After receiving a selection for the add control 330, the electronic device 110 may present, for example, Figure 3B Interface 300B is shown.

[0084] like Figure 3B As shown, the electronic device 110 can receive new description text 340 input by the user, such as "A toolbar with four icons". For example, the user can use the text box of the input area 302 to input the new description text 340, and the input process can be similar to the generation process of the description text 320. Such description text 340 can be, for example, the second part of the page to be generated.

[0085] Further, the electronic device 110 may provide a second portion of the candidate layout 345 in the preview area accordingly. Additionally, the electronic device 110 may also provide one or more other candidate layouts 350 in the selection area 306. The user may replace the selected candidate layout 345 in the preview area 304 by, for example, clicking on the candidate layout 355.

[0086] Such candidate layouts 345 and 350 may be generated, for example, using the single or multiple layout generation processes discussed above, and may have a certain degree of generation randomness, thereby presenting different layout styles.

[0087] In some embodiments, the generation process of the candidate layouts 310 and 335 may be different, and the generation of the candidate layouts 345 and 350 may also be determined based on the layout information associated with the first portion, for example.

[0088] As an example, when the user modifies the candidate layout 310 of the first part, the electronic device 110 can also obtain the final layout information of the first part. For example, the user can adjust the left coordinates of the leftmost page element in the first part and the right coordinates of the rightmost page element.

[0089] Accordingly, such layout information may also be provided to a device for layout generation to take into account the layout information of the existing portion of the page. For example, the generation process of the candidate layouts 345 and 350 of the second portion may take into account the left and right coordinates of the first portion, so that the left coordinate of the leftmost page element in the candidate layout generated for the second portion is aligned with the left coordinate of the first portion, and / or the right coordinate of the rightmost page element in the candidate layout generated for the second portion is aligned with the right coordinate of the first portion.

[0090] In some implementations, the layout information associated with the first part may also include other appropriate types of information, such as style information, size information, orientation information, other position information, etc. Such layout information can be provided to the generation process to assist in the generation of the page layout of the second part, thereby improving the style consistency of the generated page.

[0091] Similarly, the user may also trigger the generation of a new set of candidate layouts based on the description text 340. For example, if the user is not satisfied with the provided candidate layouts 345 and 350, the generation and provision of a new set of candidate layouts may also be triggered.

[0092] Furthermore, the user may modify the provided candidate layout 340. For example, the user may use the interface 300B to adjust the type, size, position, and other information of one or more page elements in the candidate layout 340.

[0093] Additionally, the user may also utilize the add control 330 to add one or more other parts in the page to be generated, and may complete the generation or publishing of the page based on the added page parts.

[0094] Therefore, the embodiments of the present disclosure can support a page generation process based on regions, thereby improving the efficiency of page generation.

[0095] Example Process

[0096] Figure 4 FIG. 4 is a flowchart of an example process 400 for page generation according to some implementations of the present disclosure. The process 400 may be performed by, for example, Figure 1 The electronic device 110 or other suitable device (for example, combining Figure 5 The device 500 discussed above is implemented.

[0097] like Figure 4 As shown, in box 410, the electronic device 110 provides a first part of the target page to be generated, and the layout of the first part is determined based on an input sequence corresponding to the first description text, and the input sequence is generated based on the type of at least one layout constraint indicated by the first description text.

[0098] In block 420 , the electronic device 110 obtains a second description text for a second portion of the target page.

[0099] In block 430 , the electronic device 110 provides a first set of candidate layouts for the second portion, the first set of candidate layouts being determined based on at least the second description text.

[0100] In some implementations, the input sequence is generated by converting an intermediate representation corresponding to the first description text, where the intermediate representation describes at least one layout constraint indicated by the first description text in a formal language.

[0101] In some implementations, the type of at least one constraint includes at least one of the following:

[0102] The first constraint type indicates an element-level constraint for a single page element; the second constraint type indicates a pair-level constraint between two page elements; and the third constraint type indicates a group-level constraint on a group of page elements.

[0103] In some implementations, providing the first part of the target page to be generated includes: obtaining an output sequence determined by a target model based on an input sequence, the output sequence indicating a distribution of a set of page elements in the first part; and providing the first part of the target page according to the output sequence.

[0104] In some implementations, the target model is trained using a training dataset, where the training dataset includes at least a first data portion, the first data portion including unlabeled data generated based on a first set of training pages.

[0105] In some implementations, the first data portion includes: a first set of training input sequences generated based on a page layout of a first set of training pages; and a first set of training output sequences corresponding to the first set of training pages.

[0106] In some implementations, the training data set also includes a second data portion, and the second data portion includes annotated data corresponding to the second set of training pages.

[0107] In some implementations, the second data portion includes: a second set of training input sequences generated based on the annotation data, the annotation data including a set of training description texts corresponding to the second set of training pages; and a second set of training output sequences corresponding to the second set of training pages.

[0108] In some implementations, at least one of the second training input sequences corresponds to a subset of all page elements in the corresponding training page.

[0109] In some implementations, the output sequence includes element representations corresponding to a set of page elements, wherein the element representations include at least a first flag indicating whether the corresponding page element is indicated in the description text.

[0110] In some implementations, the element representation further includes at least one of the following: a second mark indicating the type of the corresponding page element; a third mark indicating the position of the corresponding page element; and a fourth mark indicating the size of the corresponding page element.

[0111] In some implementations, the first set of candidate layouts is also determined based on layout information associated with the first portion.

[0112] In some implementations, providing a first portion of a target page to be generated includes: providing a second set of candidate layouts for the first portion, the second set of candidate layouts being determined based on a first description text; and providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts.

[0113] In some implementations, providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts includes generating the first portion of the target page based on a modification operation on the selected target layout.

[0114] In some implementations, providing a second set of candidate layouts for the first portion includes: providing a third set of candidate layouts determined based on the first description text; and providing the second set of candidate layouts for the first portion based on the regeneration request.

[0115] In some implementations, obtaining second description text for the second portion of the target page includes: providing an add control for adding the second portion in association with the first portion; and obtaining second description text for the second portion of the target page based on a selection of the add control.

[0116] In some implementations, process 400 also includes: determining a second portion of the target page based on the first set of candidate layouts; and generating the target page based on at least the first portion and the second portion.

[0117] Example Device

[0118] Figure 5 Schematic block diagram of an example device 500 that can be used to implement embodiments of the present disclosure is shown. It should be understood that Figure 5 The device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the implementations described in the present disclosure. Figure 5As shown, components of device 500 may include, but are not limited to, one or more processors or processing units 510 , memory 520 , storage devices 530 , one or more communication units 540 , one or more input devices 550 , and one or more output devices 55560 .

[0119] In some implementations, the device 500 can be implemented as various user terminals or service terminals. The service terminal can be a server, a large computing device, etc. provided by various service providers. The user terminal is such as any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. It is also foreseeable that the device 500 can support any type of interface for the user (such as a "wearable" circuit, etc.).

[0120] Processing unit 510 may be a real or virtual processor and is capable of performing various processes according to a program stored in memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to increase the parallel processing capabilities of device 500. Processing unit 510 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0121] The device 500 generally includes a plurality of computer storage media. Such media may be any available media accessible to the device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., register, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof. The memory 520 may include one or more generation modules and / or generation modules 525, which are configured to perform layout generation and / or page generation functions of various implementations described herein. The generation module 525 may be accessed and run by the processing unit 510 to implement the corresponding functions. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium that can be used to store information and / or data and can be accessed within the device 500.

[0122] The function of the component of device 500 can be realized with a single computing cluster or multiple computing machines, and these computing machines can communicate by communication connection. Therefore, device 500 can use the logical connection with one or more other servers, personal computers (PC) or another general network node to operate in a networked environment. Device 500 can also communicate with one or more external devices (not shown) by communication unit 540 as needed, and external devices such as databases, other storage devices, servers, display devices, etc. communicate with one or more devices that make users interact with device 500, or communicate with any device (for example, network card, modem, etc.) that makes device 500 communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interface (not shown).

[0123] The input device 550 may be one or more various input devices, such as a mouse, a keyboard, a tracking ball, a voice input device, a camera, etc. The output device 55560 may be one or more output devices, such as a display, a speaker, a printer, etc.

[0124] Example Implementation

[0125] Some example implementations of the present disclosure are listed below.

[0126] In a first aspect of the present disclosure, a method for page generation is provided. The method includes: providing a first part of a target page to be generated, the layout of the first part being determined based on an input sequence corresponding to a first description text, the input sequence being generated based on a type of at least one layout constraint indicated by the first description text; acquiring a second description text for a second part of the target page; and providing a first set of candidate layouts for the second part, the first set of candidate layouts being determined based at least on the second description text.

[0127] In some implementations, the input sequence is generated by converting an intermediate representation corresponding to the first description text, where the intermediate representation describes at least one layout constraint indicated by the first description text in a formal language.

[0128] In some implementations, the type of at least one constraint includes at least one of the following:

[0129] The first constraint type indicates an element-level constraint for a single page element; the second constraint type indicates a pair-level constraint between two page elements; and the third constraint type indicates a group-level constraint on a group of page elements.

[0130] In some implementations, providing the first part of the target page to be generated includes: obtaining an output sequence determined by a target model based on an input sequence, the output sequence indicating a distribution of a set of page elements in the first part; and providing the first part of the target page according to the output sequence.

[0131] In some implementations, the target model is trained using a training dataset, where the training dataset includes at least a first data portion, the first data portion including unlabeled data generated based on a first set of training pages.

[0132] In some implementations, the first data portion includes: a first set of training input sequences generated based on a page layout of a first set of training pages; and a first set of training output sequences corresponding to the first set of training pages.

[0133] In some implementations, the training data set also includes a second data portion, and the second data portion includes annotated data corresponding to the second set of training pages.

[0134] In some implementations, the second data portion includes: a second set of training input sequences generated based on the annotation data, the annotation data including a set of training description texts corresponding to the second set of training pages; and a second set of training output sequences corresponding to the second set of training pages.

[0135] In some implementations, at least one of the second training input sequences corresponds to a subset of all page elements in the corresponding training page.

[0136] In some implementations, the output sequence includes element representations corresponding to a set of page elements, wherein the element representations include at least a first flag indicating whether the corresponding page element is indicated in the description text.

[0137] In some implementations, the element representation further includes at least one of the following: a second mark indicating the type of the corresponding page element; a third mark indicating the position of the corresponding page element; and a fourth mark indicating the size of the corresponding page element.

[0138] In some implementations, the first set of candidate layouts is also determined based on layout information associated with the first portion.

[0139] In some implementations, providing a first portion of a target page to be generated includes: providing a second set of candidate layouts for the first portion, the second set of candidate layouts being determined based on a first description text; and providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts.

[0140] In some implementations, providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts includes generating the first portion of the target page based on a modification operation on the selected target layout.

[0141] In some implementations, providing a second set of candidate layouts for the first portion includes: providing a third set of candidate layouts determined based on the first description text; and providing the second set of candidate layouts for the first portion based on the regeneration request.

[0142] In some implementations, obtaining second description text for the second portion of the target page includes: providing an add control for adding the second portion in association with the first portion; and obtaining second description text for the second portion of the target page based on a selection of the add control.

[0143] In some implementations, the method further includes: determining a second portion of the target page based on the first set of candidate layouts; and generating the target page based on at least the first portion and the second portion.

[0144] In a second aspect of the present disclosure, an electronic device is provided. The device includes: a processing unit; and a memory, coupled to the processing unit and containing instructions stored thereon, the instructions causing the device to perform the following actions when executed by the processing unit: providing a first portion of a target page to be generated, the layout of the first portion being determined based on an input sequence corresponding to a first description text, the input sequence being generated based on the type of at least one layout constraint indicated by the first description text; obtaining a second description text for a second portion of the target page; and providing a first set of candidate layouts for the second portion, the first set of candidate layouts being determined based at least on the second description text.

[0145] In some implementations, the input sequence is generated by converting an intermediate representation corresponding to the first description text, where the intermediate representation describes at least one layout constraint indicated by the first description text in a formal language.

[0146] In some implementations, the type of at least one constraint includes at least one of the following:

[0147] The first constraint type indicates an element-level constraint for a single page element; the second constraint type indicates a pair-level constraint between two page elements; and the third constraint type indicates a group-level constraint on a group of page elements.

[0148] In some implementations, providing the first part of the target page to be generated includes: obtaining an output sequence determined by a target model based on an input sequence, the output sequence indicating a distribution of a set of page elements in the first part; and providing the first part of the target page according to the output sequence.

[0149] In some implementations, the target model is trained using a training dataset, where the training dataset includes at least a first data portion, the first data portion including unlabeled data generated based on a first set of training pages.

[0150] In some implementations, the first data portion includes: a first set of training input sequences generated based on a page layout of a first set of training pages; and a first set of training output sequences corresponding to the first set of training pages.

[0151] In some implementations, the training data set also includes a second data portion, and the second data portion includes annotated data corresponding to the second set of training pages.

[0152] In some implementations, the second data portion includes: a second set of training input sequences generated based on the annotation data, the annotation data including a set of training description texts corresponding to the second set of training pages; and a second set of training output sequences corresponding to the second set of training pages.

[0153] In some implementations, at least one of the second training input sequences corresponds to a subset of all page elements in the corresponding training page.

[0154] In some implementations, the output sequence includes element representations corresponding to a set of page elements, wherein the element representations include at least a first flag indicating whether the corresponding page element is indicated in the description text.

[0155] In some implementations, the element representation further includes at least one of the following: a second mark indicating the type of the corresponding page element; a third mark indicating the position of the corresponding page element; and a fourth mark indicating the size of the corresponding page element.

[0156] In some implementations, the first set of candidate layouts is also determined based on layout information associated with the first portion.

[0157] In some implementations, providing a first portion of a target page to be generated includes: providing a second set of candidate layouts for the first portion, the second set of candidate layouts being determined based on a first description text; and providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts.

[0158] In some implementations, providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts includes generating the first portion of the target page based on a modification operation on the selected target layout.

[0159] In some implementations, providing a second set of candidate layouts for the first portion includes: providing a third set of candidate layouts determined based on the first description text; and providing the second set of candidate layouts for the first portion based on the regeneration request.

[0160] In some implementations, obtaining second description text for the second portion of the target page includes: providing an add control for adding the second portion in association with the first portion; and obtaining second description text for the second portion of the target page based on a selection of the add control.

[0161] In some implementations, the actions further include: determining a second portion of the target page based on the first set of candidate layouts; and generating the target page based on at least the first portion and the second portion.

[0162] In a sixth aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine executable instructions, which, when executed by a device, cause the device to perform the following actions: provide a first portion of a target page to be generated, the layout of the first portion being determined based on an input sequence corresponding to a first description text, the input sequence being generated based on a type of at least one layout constraint indicated by the first description text; obtain a second description text for a second portion of the target page; and provide a first set of candidate layouts for the second portion, the first set of candidate layouts being determined based at least on the second description text.

[0163] In some implementations, the input sequence is generated by converting an intermediate representation corresponding to the first description text, where the intermediate representation describes at least one layout constraint indicated by the first description text in a formal language.

[0164] In some implementations, the type of at least one constraint includes at least one of the following:

[0165] The first constraint type indicates an element-level constraint for a single page element; the second constraint type indicates a pair-level constraint between two page elements; and the third constraint type indicates a group-level constraint on a group of page elements.

[0166] In some implementations, providing the first part of the target page to be generated includes: obtaining an output sequence determined by a target model based on an input sequence, the output sequence indicating a distribution of a set of page elements in the first part; and providing the first part of the target page according to the output sequence.

[0167] In some implementations, the target model is trained using a training dataset, where the training dataset includes at least a first data portion, the first data portion including unlabeled data generated based on a first set of training pages.

[0168] In some implementations, the first data portion includes: a first set of training input sequences generated based on a page layout of a first set of training pages; and a first set of training output sequences corresponding to the first set of training pages.

[0169] In some implementations, the training data set also includes a second data portion, and the second data portion includes annotated data corresponding to the second set of training pages.

[0170] In some implementations, the second data portion includes: a second set of training input sequences generated based on the annotation data, the annotation data including a set of training description texts corresponding to the second set of training pages; and a second set of training output sequences corresponding to the second set of training pages.

[0171] In some implementations, at least one of the second training input sequences corresponds to a subset of all page elements in the corresponding training page.

[0172] In some implementations, the output sequence includes element representations corresponding to a set of page elements, wherein the element representations include at least a first flag indicating whether the corresponding page element is indicated in the description text.

[0173] In some implementations, the element representation further includes at least one of the following: a second mark indicating the type of the corresponding page element; a third mark indicating the position of the corresponding page element; and a fourth mark indicating the size of the corresponding page element.

[0174] In some implementations, the first set of candidate layouts is also determined based on layout information associated with the first portion.

[0175] In some implementations, providing a first portion of a target page to be generated includes: providing a second set of candidate layouts for the first portion, the second set of candidate layouts being determined based on a first description text; and providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts.

[0176] In some implementations, providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts includes generating the first portion of the target page based on a modification operation on the selected target layout.

[0177] In some implementations, providing a second set of candidate layouts for the first portion includes: providing a third set of candidate layouts determined based on the first description text; and providing the second set of candidate layouts for the first portion based on the regeneration request.

[0178] In some implementations, obtaining second description text for the second portion of the target page includes: providing an add control for adding the second portion in association with the first portion; and obtaining second description text for the second portion of the target page based on a selection of the add control.

[0179] In some implementations, the actions further include: determining a second portion of the target page based on the first set of candidate layouts; and generating the target page based on at least the first portion and the second portion.

[0180] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0181] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0182] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0183] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain the desired result. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of a separate implementation can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination.

[0184] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A computer-implemented method comprising: providing a first portion of a target page to be generated, wherein a layout of the first portion is determined based on an input sequence corresponding to a first description text, wherein the input sequence is generated based on a type of at least one layout constraint indicated by the first description text; Obtaining a second description text for a second part of the target page; as well as A first set of candidate layouts for the second portion is provided, the first set of candidate layouts being determined based on at least the second description text. 2 . The method according to claim 1 , wherein the input sequence is generated by converting an intermediate representation corresponding to the first description text, wherein the intermediate representation describes at least one layout constraint indicated by the first description text in a formal language.

3. The method of claim 1 , wherein the type of the at least one constraint comprises at least one of the following: The first constraint type indicates an element-level constraint for a single page element; The second constraint type indicates a pair-level constraint between two page elements; The third constraint type indicates a group-level constraint on a group of page elements.

4. The method according to claim 1, wherein providing the first portion of the target page to be generated comprises: Obtaining an output sequence determined by a target model based on the input sequence, the output sequence indicating distribution of a group of page elements in the first portion; as well as According to the output sequence, the first portion of the target page is provided.

5. The method according to claim 4, wherein the target model is trained using a training dataset, wherein the training dataset includes at least a first data portion, and the first data portion includes unlabeled data generated based on a first set of training pages.

6. The method of claim 5, wherein the first data portion comprises: A first set of training input sequences generated based on the page layout of the first set of training pages; as well as A first set of training output sequences corresponding to the first set of training pages.

7. The method according to claim 5, wherein the training data set further includes a second data portion, wherein the second data portion includes annotated data corresponding to a second set of training pages.

8. The method of claim 7, wherein the second data portion comprises: a second set of training input sequences generated based on the annotated data, the annotated data comprising a set of training description texts corresponding to the second set of training pages; as well as A second set of training output sequences corresponding to the second set of training pages.

9. The method of claim 8, wherein at least one of the second training input sequences corresponds to a subset of all page elements in a corresponding training page.

10. The method according to claim 4, wherein the output sequence includes element representations corresponding to the set of page elements, wherein the element representations include at least a first mark indicating whether the corresponding page element is indicated in the description text.

11. The method according to claim 10, wherein the element representation further comprises at least one of the following: A second tag indicates the type of the corresponding page element; A third mark indicates the position of the corresponding page element; The fourth mark indicates the size of the corresponding page element.

12. The method of claim 1, wherein the first set of candidate layouts is further determined based on layout information associated with the first portion.

13. The method according to claim 1, wherein providing the first portion of the target page to be generated comprises: providing a second set of candidate layouts for the first portion, wherein the second set of candidate layouts is determined based on the first description text; as well as Based at least on a selection of a target layout from the second set of candidate layouts, the first portion of the target page is provided.

14. The method of claim 13, wherein providing the first portion of the target page based at least on a selection of a target layout from the second set of candidate layouts comprises: Based on the modification operation on the selected target layout, the first part of the target page is generated.

15. The method of claim 13, wherein providing a second set of candidate layouts for the first portion comprises: providing a third candidate layout determined based on the first description text; as well as Based on the regeneration request, a second set of candidate layouts for the first portion is provided.

16. The method according to claim 1, wherein obtaining a second description text for a second part of the target page comprises: Associated with the first part, providing an adding control for adding the second part; as well as Based on the selection of the add control, the second description text for the second part of the target page is obtained.

17. The method according to claim 1, further comprising: Based on the first set of candidate layouts, determining the second portion of the target page; as well as The target page is generated based on at least the first part and the second part.

18. An electronic device comprising: Processing unit; as well as a memory coupled to the processing unit and containing instructions stored thereon, the instructions, when executed by the processing unit, causing the electronic device to perform actions, the actions comprising: providing a first portion of a target page to be generated, wherein a layout of the first portion is determined based on an input sequence corresponding to a first description text, wherein the input sequence is generated based on a type of at least one layout constraint indicated by the first description text; Obtaining a second description text for a second part of the target page; and A first set of candidate layouts for the second portion is provided, the first set of candidate layouts being determined based on at least the second description text.

19. The electronic device of claim 18, wherein the type of the at least one constraint comprises at least one of the following: The first constraint type indicates an element-level constraint for a single page element; The second constraint type indicates a pair-level constraint between two page elements; The third constraint type indicates a group-level constraint on a group of page elements.

20. A computer program product tangibly stored in a non-transitory computer storage medium and comprising machine-executable instructions that, when executed by a device, cause the device to perform actions comprising: providing a first portion of a target page to be generated, wherein a layout of the first portion is determined based on an input sequence corresponding to a first description text, wherein the input sequence is generated based on a type of at least one layout constraint indicated by the first description text; Obtaining a second description text for a second part of the target page; as well as A first set of candidate layouts for the second portion is provided, the first set of candidate layouts being determined based on at least the second description text.

Images