A page layout method and apparatus

Abstract

The application relates to a page layout method and device. The method comprises the following steps: acquiring page information, element information of at least one page element to be laid out, and a layout rule; based on the page information, the element information of the at least one page element to be laid out, and the layout rule, at least one candidate page layout strategy is acquired by using a reinforcement learning algorithm; and a target page layout strategy is determined from the at least one candidate page layout strategy by using an imitation learning algorithm.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a page layout method and apparatus. Background Technology

[0002] Page layout is a core part of the design process and also the most time-consuming part for designers when creating pages. Page layout requires deploying a varying number of page elements of different shapes and sizes within a limited space. Furthermore, the layout must adhere to specific rules and maintain a certain level of aesthetic appeal between page elements and between page elements themselves.

[0003] In recent years, with the development of artificial intelligence technology, using machine learning to replace manual labor has gradually become a key focus in page layout development. Automated layout utilizes computer technology to replace human designers in page layout, using artificial intelligence to achieve the required page layout. Automated layout typically does not require trained personnel to operate, thus greatly saving designers' manpower and training time. In related technologies, multiple page elements can be deployed on a canvas using random layout, and then the completed page layout is converted into image data. An evaluation model is then used to score the image data, and finally, the user can select the page layout with the highest score. As can be seen from the automatic page layout generation methods in related technologies, random layout is difficult to generate pages that conform to rules, especially as the number of page elements increases and the page size increases, it is easy to generate irregular page layouts with overlapping page elements and page elements deployed in boundary areas.

[0004] Therefore, there is an urgent need in related technologies for an automated page layout method that conforms to layout rules. Summary of the Invention

[0005] In view of this, a page layout method and apparatus are proposed.

[0006] In a first aspect, embodiments of this application provide a page layout method, the method comprising:

[0007] Obtain page information, element information of at least one page element to be laid out, and layout rules;

[0008] Based on page information, element information of at least one page element to be laid out, and layout rules, a reinforcement learning algorithm is used to obtain at least one candidate page layout strategy.

[0009] The target page layout strategy is determined from the at least one candidate page layout strategy using an imitation learning algorithm.

[0010] The above method first obtains candidate page layout strategies through reinforcement learning algorithm based on layout rules, and then uses imitation learning algorithm to determine the target page layout strategy, which can obtain a page layout that is more in line with layout rules and user aesthetic habits.

[0011] According to the first possible implementation of the first aspect, the layout rules include a first priority layout rule and a second priority layout rule, the at least one candidate page layout strategy satisfies the first priority layout rule, and the reward obtained under the constraint of the second priority layout rule is greater than a preset reward threshold.

[0012] In this embodiment, the layout rules are divided into a first priority layout rule and a second priority layout rule. The first priority layout rule has a higher priority than the second priority layout rule. The first priority layout rule can be used as a constraint condition when the agent performs actions, and the second priority layout rule can be used as the basis for the environment to determine the reward value. Through this method, the generated candidate page layout strategy can fully satisfy the hard rules and, as far as possible, satisfy the soft rules.

[0013] According to the second possible implementation of the first aspect, the step of obtaining at least one candidate page layout strategy using a reinforcement learning algorithm based on the page information, the element information of the at least one page element to be laid out, and the layout rules includes:

[0014] Step 1: Based on the page information and the element information of the first page element to be laid out, lay out the first page element to be laid out on the page according to the layout rules, and generate the page state of the page.

[0015] Step 2: Based on the page state and the element information of the next page element to be laid out, determine the layout position of the next page element to be laid out in the page according to the layout rules.

[0016] Step 3: Determine the reward value corresponding to the page state and the layout position, and update the page state;

[0017] Iterate through steps 2 and 3 until at least one page element to be laid out is placed on the page, and generate a page layout strategy.

[0018] The reward value determined in each iteration is accumulated, and the page layout strategy that satisfies the preset conditions with the accumulated reward value is taken as the candidate page layout strategy.

[0019] In this embodiment of the application, the decision-making process of page layout is modeled as a reinforcement learning decision-making process. In each iteration, the layout rules constrain the layout position of only one page element to be laid out, so that the final candidate page layout strategy can fully satisfy the layout rules.

[0020] According to the third possible implementation of the first aspect, the iterative steps 2 and 3, until at least one page element to be laid out is placed on the page and a page layout strategy is generated, include:

[0021] Calculate the cumulative value of the reward for each page state and each layout position during the iteration process, and determine the strategy value of the current page layout strategy based on the cumulative value of the reward.

[0022] If the value of the strategy is determined to be no less than a preset value threshold, steps 2 and 3 are iterated until at least one page element to be laid out is placed on the page and a page layout strategy is generated.

[0023] In this embodiment, the Monte Carlo tree search algorithm is used during reinforcement learning to reduce the search space for page layout, which can significantly reduce search complexity when there are many elements on the page to be laid out. Furthermore, by predicting the long-term rewards of intermediate nodes using a value function, page layout strategies with higher reward values ​​can be obtained while reducing the search space.

[0024] According to the fourth possible implementation of the first aspect, determining the target page layout strategy from the at least one candidate page layout strategy using an imitation learning algorithm includes:

[0025] The score of each candidate page layout strategy is determined based on the reward function obtained from the pre-trained imitation learner, wherein the reward function is determined by the ranking loss of positive and negative samples used to train the imitation learner, and the positive samples include page layout strategies that conform to user aesthetics.

[0026] The target page layout strategy is determined from the at least one page layout strategy layout based on the score.

[0027] In this embodiment, the page layout strategy of human experts is learned by imitation learning algorithm, so that the aesthetics of the imitation learner are more similar to those of human experts.

[0028] According to the fifth possible implementation of the first aspect, after determining the target page layout strategy from the at least one page layout strategy layout based on the score, the method further includes:

[0029] Send the target page layout strategy;

[0030] Receive user adjustments to the target page layout strategy;

[0031] The adjusted target page layout strategy is used as a positive sample to train the imitation learner, thereby obtaining an optimized reward function.

[0032] In this embodiment, a mechanism for interaction between the imitation learner and the user is provided. If the user is dissatisfied with the optimal page layout strategy output by the imitation learner and makes adjustments, the imitation learner can learn from the adjusted page layout strategy to optimize the reward function. This allows the learner to continuously optimize its performance based on user preferences, enhancing its ability to continuously improve its learning capabilities.

[0033] According to the sixth possible implementation of the first aspect, obtaining page information, element information of at least one page element to be laid out, and layout rules includes:

[0034] It receives page information configured by the user in the user interface, element information of at least one page element to be laid out, and layout rules.

[0035] In this embodiment of the application, the interaction interface between the page layout and the user allows the user to configure various information in the interaction interface.

[0036] According to the seventh possible implementation of the first aspect, the at least one page element to be laid out is set to be laid out on the page in descending order of size.

[0037] In this embodiment of the application, arranging the at least one page element to be laid out in descending order of size can improve layout efficiency and accelerate the convergence speed of the reinforcement learning algorithm.

[0038] According to the eighth possible implementation of the first aspect, the step of laying out the first page element to be laid out on the page according to the layout rules includes:

[0039] The page is divided into multiple grids, and the size of the grids is set according to the size of the smallest element among the at least one page element;

[0040] The first page element to be laid out is placed on the page according to the layout rules, and at least one vertex of the smallest bounding box of the first page element to be laid out coincides with a vertex of the grid.

[0041] In this embodiment of the application, the grid layout method makes it easier to align the page elements to be laid out, and also speeds up the efficiency of page layout.

[0042] According to the ninth possible implementation of the first aspect, the page state includes non-layoutable areas and layable areas in the page, wherein the grid corresponding to the non-layoutable area is set with a masking mark.

[0043] In this embodiment of the application, the page state is described by combining gridding and matrix methods, which not only simplifies the description of the page state, but also reduces the search space for subsequent layout, speeds up the search, and improves layout efficiency.

[0044] According to the tenth possible implementation of the first aspect, determining the reward value corresponding to the page state and the layout position includes:

[0045] Determine the distance values ​​of each of the laid-out page elements from the four sides of the page;

[0046] Count the number of unique distance values ​​along the four sides of the page;

[0047] The reward value is determined based on the number of items.

[0048] In this embodiment, soft rules are quantified into reward values ​​so that the generated layout strategy satisfies the soft rules as much as possible.

[0049] According to the eleventh possible implementation of the first aspect, determining the reward value corresponding to the page state and the layout position includes:

[0050] Determine the sets of distances from each of the laid-out page elements to the four sides of the canvas, as well as the sets of horizontal and vertical spacing between adjacent laid-out page elements;

[0051] Determine the standard deviation of the distance values ​​in the distance set, the horizontal spacing set, and the vertical spacing set, respectively;

[0052] The reward value is determined based on the standard deviation.

[0053] In this embodiment, soft rules are quantified into reward values ​​so that the generated layout strategy satisfies the soft rules as much as possible.

[0054] Secondly, embodiments of this application provide a page layout apparatus, characterized in that it includes:

[0055] The initial information acquisition module is used to acquire page information, element information of at least one page element to be laid out, and layout rules;

[0056] The reinforcement learning module is used to obtain at least one candidate page layout strategy based on the page information, the element information of the at least one page element to be laid out, and the layout rules, using a reinforcement learning algorithm.

[0057] The imitation learning module is used to determine the target page layout strategy from the at least one candidate page layout strategy using an imitation learning algorithm.

[0058] According to the first possible implementation of the first aspect, the layout rules include a first priority layout rule and a second priority layout rule, the at least one candidate page layout strategy satisfies the first priority layout rule, and the reward obtained under the constraint of the second priority layout rule is greater than a preset reward threshold.

[0059] According to the second possible implementation of the first aspect, the reinforcement learning module is specifically used for:

[0060] Step 1: Based on the page information and the element information of the first page element to be laid out, lay out the first page element to be laid out on the page according to the layout rules, and generate the page state of the page.

[0061] Step 2: Based on the page state and the element information of the next page element to be laid out, determine the layout position of the next page element to be laid out in the page according to the layout rules.

[0062] Step 3: Determine the reward value corresponding to the page state and the layout position, and update the page state;

[0063] Iterate through steps 2 and 3 until at least one page element to be laid out is placed on the page, and generate a page layout strategy.

[0064] The reward value determined in each iteration is accumulated, and the page layout strategy that satisfies the preset conditions with the accumulated reward value is taken as the candidate page layout strategy.

[0065] According to the third possible implementation of the first aspect, the reinforcement learning module is further used for:

[0066] Calculate the cumulative value of the reward for each page state and each layout position during the iteration process, and determine the strategy value of the current page layout strategy based on the cumulative value of the reward.

[0067] If the value of the strategy is determined to be no less than a preset value threshold, steps 2 and 3 are iterated until at least one page element to be laid out is placed on the page and a page layout strategy is generated.

[0068] According to the fourth possible implementation of the first aspect, the imitation learning module is specifically used for:

[0069] The score of each candidate page layout strategy is determined based on the reward function obtained from the pre-trained imitation learner, wherein the reward function is determined by the ranking loss of positive and negative samples used to train the imitation learner, and the positive samples include page layout strategies that conform to user aesthetics.

[0070] The target page layout strategy is determined from the at least one page layout strategy layout based on the score.

[0071] According to the fifth possible implementation of the first aspect, the imitation learning module is further used for:

[0072] Send the target page layout strategy;

[0073] Receive user adjustments to the target page layout strategy;

[0074] The adjusted target page layout strategy is used as a positive sample to train the imitation learner, thereby obtaining an optimized reward function.

[0075] According to the sixth possible implementation of the first aspect, the initial information acquisition module is specifically used for:

[0076] It receives page information configured by the user in the user interface, element information of at least one page element to be laid out, and layout rules.

[0077] Thirdly, embodiments of this application provide a computing device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement one or more page layout methods of the first aspect or multiple possible implementations of the first aspect when executing the instructions.

[0078] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, when the computer program instructions are executed by a processor, they implement one or more of the page layout methods described in the first aspect or various possible implementations of the first aspect.

[0079] Fifthly, embodiments of this application provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in an electronic device, the processor in the electronic device executes one or more of the page layout methods described in the first aspect or various possible implementations of the first aspect.

[0080] These and other aspects of this application will become more apparent in the description of the following embodiments(s). Attached Figure Description

[0081] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0082] Figure 1 A schematic diagram of the structure of a page layout apparatus 100 according to an embodiment of this application is shown.

[0083] Figure 2 This is a schematic diagram of a page layout scenario according to an embodiment of the present application.

[0084] Figure 3 A schematic flowchart of a page layout method according to an embodiment of this application is shown.

[0085] Figure 4 A schematic diagram of the structure of a Markov model according to an embodiment of this application is shown.

[0086] Figure 5 This diagram illustrates a process for generating page layout strategies using a reinforcement learning algorithm according to an embodiment of this application.

[0087] Figure 6 A schematic diagram of a scenario according to an embodiment of this application is shown.

[0088] Figure 7 A schematic diagram of a scenario according to an embodiment of this application is shown.

[0089] Figure 8 A schematic diagram of a scenario according to an embodiment of this application is shown.

[0090] Figure 9 A schematic diagram of a scenario according to an embodiment of this application is shown.

[0091] Figure 10 This diagram illustrates a process for generating page layout strategies using a reinforcement learning algorithm according to an embodiment of this application.

[0092] Figure 11 This diagram illustrates the structure of another page layout device 100 provided according to an embodiment of this application.

[0093] Figure 12 A schematic diagram of the module structure of a computing device 1200 according to an embodiment of this application is shown. Detailed Implementation

[0094] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0095] The term "exemplary" as used herein means "serving as an example, embodiment, or illustration." Any embodiment illustrated herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments. The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate, and this is merely a way of distinguishing objects with the same properties in the embodiments of this application.

[0096] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented even without certain specific details. In some instances, methods and means well-known to those skilled in the art have not been described in detail, in order to highlight the main points of this application.

[0097] To facilitate understanding of the technical solutions provided in the embodiments of this application by those skilled in the art, the technical environment for implementing the technical solutions will be described below.

[0098] The core of page layout is arranging multiple page elements of varying numbers, shapes, and sizes within a limited space. To evaluate the quality of a page layout, it's necessary to determine if it meets the following conditions: First, does it adhere to hard rules? Second, does it meet soft rules? Third, does it conform to human aesthetics or user habits? Hard rules can include fundamental principles that page layout must follow, such as avoiding overlapping of different page elements and excluding elements from the edge areas of the page. Soft rules can include less strictly enforced rules that enhance the aesthetics of the layout, such as horizontal and vertical alignment of page elements and uniform spacing between elements.

[0099] Based on practical technical needs similar to those described above, this application provides a page layout method. The method uses a reinforcement learning algorithm to obtain at least one candidate page layout strategy based on page information, element information of at least one page element to be laid out, and layout rules. Then, an imitation learning algorithm is used to determine a target page layout strategy from the at least one candidate page layout strategy. The layout rules may include a first priority layout rule and a second priority layout rule. The first priority layout rule may include hard rules, and the second priority rule may include soft rules. In the reinforcement learning stage, on the one hand, the first priority layout rule can be used as an action constraint condition for reinforcement learning, ensuring that the generated candidate page layout strategy fully satisfies the hard rules; on the other hand, the second priority layout rule can be used as a reward function for reinforcement learning, ensuring that the generated candidate page layout strategy satisfies the soft rules as much as possible. In the imitation learning stage, the imitation learner can be trained to continuously interact with the user, learning from the user's experience and knowledge, making the page layout determined by the imitation learner more in line with the user's aesthetics and habits. Therefore, the target page layout strategy obtained using this method not only fully satisfies the hard rules and satisfies the soft rules as much as possible, but also better conforms to the user's aesthetics and habits, meeting the aforementioned requirements for an excellent page layout.

[0100] The page layout method provided in this application can be applied to various application scenarios such as graphic design, print layout, web design, industrial design drawing layout, PPT layout, social media layout, and chip back-end design.

[0101] like Figure 1 As shown, this application provides an exemplary application scenario, which may include a page layout device 100, which may include a reinforcement learning module 101 and an imitation learning module 103.

[0102] In practice, users can configure page information, element information of at least one page element to be laid out, and layout rules in the user interface. The page information may include the page size, background color, background image, etc. The page element to be laid out may include images, videos, text, or any combination of these elements. The element information may include the size of the page element to be laid out, the RGBA channel values ​​of image pixels (for bitmaps), the description of vector elements (for vector graphics), the source of the image or video, text content, etc. The layout rules may include first priority rules (i.e., hard rules) and second priority rules (i.e., soft rules). Furthermore, for different application scenarios, first priority rules and second priority rules matching the application scenario can be set; this application does not impose any limitations on this. Users can also configure other information such as the cumulative reward threshold and policy value threshold for reinforcement learning in the user interface.

[0103] Figure 2 The internal execution flow of the reinforcement learning module 101 and the imitation learning module 103 is illustrated. Regarding the execution flow of the reinforcement learning module 101, in step 201, a reinforcement learning model environment can be constructed first, for example, a Markov decision model can be constructed, specifically including the definition of parameters such as the model's state, action, reward, and state transition. In step 203, based on the constructed reinforcement learning model environment, a page layout strategy can be generated according to the page information, element information, and layout rules provided by the user. In step 205, it can be determined whether the cumulative reward of the page layout strategy meets a reward threshold. If the determination result indicates that the cumulative reward meets the reward threshold, then in step 209, the page layout strategy that meets the reward threshold is selected as a candidate page layout strategy; otherwise, in step 207, the next page layout strategy can be generated using the constraints of layout rules (including hard rules and soft rules). Figure 1 As shown, through multiple iterations of steps 203, 205 and 207, at least one candidate page layout strategy 105 that satisfies the reward threshold can be generated.

[0104] Regarding the internal workflow of the imitation learning module 103, in step 211, a reward function can be used to determine the score of the at least one candidate page layout strategy 105, and based on the score, at least one page layout strategy that best conforms to human aesthetics can be determined. Then, in step 213, the determined at least one page layout strategy can be sent to the user to determine whether it conforms to the user's aesthetics. If it does not conform to the user's aesthetics, the user can adjust the page layout strategy so that the adjusted page layout strategy conforms to the user's aesthetics. In step 215, the imitation learning module 103 can receive the user's adjustment of the page layout strategy, and in step 217, learn the adjusted page layout strategy, making the imitation learning module 103 more in line with the user's aesthetics. Through iteration of steps 211-217, a page layout strategy 107 that conforms to the user's aesthetics can finally be generated.

[0105] The page layout method described in this application will be explained in detail below with reference to the accompanying drawings. Figure 3 This is a schematic flowchart illustrating an embodiment of the page layout method provided in this application. While this application provides method operation steps as shown in the following embodiments or figures, the method may include more or fewer operation steps based on conventional or non-inventive effort. For steps that do not logically have a necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual page layout processes or when the device executes the method, it may be executed in the order shown in the embodiments or figures, or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0106] Specifically, one implementation of the page layout method provided in this application is as follows: Figure 3 As shown, the method can be executed by the aforementioned page layout device 100, and the method may include:

[0107] S301: Obtain page information, element information of at least one page element to be laid out, and layout rules.

[0108] During the page layout preparation stage, page information can be obtained. This page information may include the page's dimensions (including width and height), descriptions of the layoutable and non-layoutable areas, and background information (including background color, transparency, and image). The page elements to be laid out may include images, videos, text, page controls, or any combination of these elements. Element information may include the dimensions of the page elements to be laid out, for example, an image with dimensions of 640 pixels × 480 pixels. If the page elements to be laid out are irregularly shaped, the dimensions of the smallest bounding rectangle of the page element can be used as the dimensions of the page element. For images or videos, the element information may also include the RGBA channel values ​​of the image pixels (for bitmaps), descriptions of vector elements (for vector graphics), the style of the video playback window, and the source of the image or video. For text, the element information may also include the size and style of the text box, the text content, and the text style.

[0109] In this embodiment, the layout rules can serve as the rules followed when using reinforcement learning for page layout. Since specific reinforcement learning algorithms are involved, layout rules cannot be applied to page layout simply using text. In this embodiment, the layout rules can be quantified. Specifically, for hard rules, the coordinate values ​​of each side of the smallest bounding rectangle of the page elements can be used to determine whether two page elements overlap or whether a page element exceeds the layoutable area. For soft rules, the alignment of page elements can be determined by the distance values ​​of each page element from the four sides of the page, and then by the number of repeated distance values ​​along the four directions on the page.

[0110] S303: Based on the page information, the element information of the at least one page element to be laid out, and the layout rules, at least one candidate page layout strategy is obtained using a reinforcement learning algorithm.

[0111] To more clearly illustrate how reinforcement learning is used in page layout, we will first introduce the basic theoretical model of reinforcement learning: the Markov Decision Process (MDP).

[0112] Figure 4 This is a schematic diagram of the MDP model provided in this application, such as... Figure 4 As shown, MDP involves two interacting entities: the agent and the environment. The agent makes decisions, while the environment provides feedback. MDP can be represented by a quadruple.<S,A,R,T> It means that, among them,

[0113] (1) S is the state space, which can contain the set of environmental states that the agent may perceive;

[0114] (2) A is the Action Space, which can contain the set of actions that the Agent can take in each environmental state;

[0115] (3) R is the reward function. R(s,a,s') can represent the reward that the Agent receives from the environment when it performs action a in state s and transitions to state s'.

[0116] (4) T is the environment state transition function. T(s,a,s') can represent the probability of performing action a in state s and transitioning to state s'.

[0117] like Figure 4 The interaction process between the Agent and the environment in the MDP shown is such that the Agent perceives the environmental state as s at time t. t Based on the environmental state s t The agent can select action a from action space A. t Execution; After receiving the action selected by the Agent, the environment provides the Agent with a corresponding reward signal as feedback. t+1 and transition to a new environmental state. t+1 The agent waits for the environment to make a new decision. In the process of interacting with the environment, the agent's goal is to find an optimal policy π. * , so that π * The maximum long-term cumulative reward can be obtained at any state s and any time step t. In one example, π * It can be defined as formula (1):

[0118]

[0119] Where π represents a certain policy of the Agent (i.e., the probability distribution from state to action), E π Let γ represent the expected value under strategy π, γ be the discount rate, k be the future time step, and r be the expected value. t+k This represents the immediate reward that the Agent receives at time step (t+k).

[0120] In this embodiment of the application, the reinforcement learning model environment can be constructed based on the above MDP model. Specifically, it can be set as follows:

[0121] state st The page state at time t, which may include a description of the location of layable and non-layable areas;

[0122] Action a t The layout position of the page element to be laid out at time t can be represented as the position of the top-left vertex of the smallest outermost matrix of the page element on the page (xt). t ,y t );

[0123] State transition matrix p(s) t+1 |s t ): After completing action a t Then, the next state s t+1 The probability of occurrence;

[0124] Reward r t Transform selected soft rules into rewards through quantification.

[0125] In the reinforcement learning model environment, the agent can act as the object implementing page layout decisions. Upon perceiving the page state, the agent can execute corresponding actions based on the page state, constrained by layout rules (mainly hard rules). These actions can include the placement of the page elements to be laid out within the page. After receiving the agent's actions, the environment can provide a reward signal to the agent according to layout rules (mainly soft rules) and transition to the new page state.

[0126] The following specific example illustrates a method for generating candidate page layout strategies using reinforcement learning. Figure 5 A flowchart illustrating the method is shown. (For example...) Figure 5 As shown, the step of obtaining at least one candidate page layout strategy using a reinforcement learning algorithm based on the page information, the element information of the at least one page element to be laid out, and the layout rules includes:

[0127] S501: Based on the page information and the element information of the first page element to be laid out, the first page element to be laid out is laid out on the page according to the layout rules, and the page state of the page is generated.

[0128] In this embodiment, the page can be laid out one element at a time. This method of placing the at least one element on the page sequentially allows for constraints on the position of only one element in each iteration using layout rules, ensuring that the final candidate page layout strategy fully satisfies the layout rules. In one embodiment, the at least one element can be laid out in descending order of size. Figure 6 As shown in the image above, there are 5 page elements to be laid out on the page. These 5 elements are arranged in descending order of size as shown below. Figure 6 The layout order shown in the lower figure indicates that the five page elements to be laid out can be arranged in the order of 4-3-1-5-2. Arranging the at least one page element to be laid out in descending order of size can improve layout efficiency and accelerate the convergence speed of the reinforcement learning algorithm. Of course, in other embodiments, the at least one page element to be laid out can be arranged in any other order, and this application does not impose any restrictions on this.

[0129] In this embodiment of the application, after determining the first page element to be laid out, the first page element to be laid out can be placed on the page according to the layout rules based on the page information and the element information of the first page element to be laid out, and the page state of the page can be generated. In a specific example, such as Figure 7 As shown, page 701 was obtained, and... Figure 6 The five page elements to be laid out are shown in page 701. Page element 4 is selected as the first element to be laid out, and its placement on page 701 must adhere to the layout rules. These rules may include, for example, that different page elements do not overlap, that page elements do not occupy unlayable areas on the page, and that the spacing between page elements is uniform. For instance, the layout rules may include maintaining 3cm margins on the left and right sides and 5cm margins on the top and bottom of page 701. Therefore, when setting the page elements to be laid out on page 701, it is necessary to obtain the distances of the page elements to be laid out from the top, bottom, left, and right edges of page 701. Figure 7 The diagram illustrates the distances between the page element 4 to be laid out and its four sides. Specifically, given that d1 and d3 are both greater than or equal to 3cm, and d2 and d4 are both greater than or equal to 5cm, the layout of the page element 4 to be laid out satisfies the layout rules.

[0130] It should be noted that, to improve page layout efficiency, page 701 can be divided into multiple grids 703. The size of each grid 703 is set based on the size of the smallest element among the at least one page element to be laid out. For example, among the five page elements to be laid out, element 2 has the smallest size; therefore, the side length of grid 703 can be set to be equal to the smallest side length of element 2. During the layout of the page elements, at least one vertex of the outer frame of each element can coincide with a vertex of the grid. Figure 7 As shown, the top-left vertex of the page element 4 to be laid out is set to coincide with one of the grid vertices. This grid-based layout method makes it easier to align the page elements and also speeds up the page layout process. Furthermore, in other embodiments, the size of the grid 703 can be set to any other value, such as a fixed value, minimum margin, etc., which is not limited in this application.

[0131] In this embodiment, after the first page element to be laid out is placed on the page, a page state of page 701 can be generated. The page state can include non-layoutable areas and layable areas in page 701. In one embodiment, the non-layoutable areas and the layable areas can be described using coordinates, for example, using multiple key points. In another embodiment of this application, the non-layoutable areas in page 701 can be described by setting masking markers. Specifically, after dividing page 701 into multiple grids 703, the number of grids can be determined to be M×N, such as... Figure 7 In this embodiment, M = 8 and N = 10. A masking identifier can be set on the non-layoutable areas to indicate that page elements cannot be laid out at the corresponding locations. For example, the masking identifier can be set to "1", while other layoutable areas are set to "0" by default. This allows the page state of page 701 to be described using an M×N matrix. By using a combination of gridding and matrix methods to describe the page state, the description of the page state is simplified, the search space for subsequent layout is reduced, the search speed is accelerated, and layout efficiency is improved.

[0132] S503: Based on the page state and the element information of the next page element to be laid out, determine the layout position of the next page element to be laid out in the page according to the layout rules.

[0133] In this embodiment of the application, the agent in the MDP model can obtain the aforementioned page state and, based on the page state and the element information of the next page element to be laid out, perform the following action: determine the layout position of the next page element to be laid out on the page according to the layout rules. For example, in Figure 6In the middle, the next page element to be laid out is page element 3, according to... Figure 7 The page state and the page element 3 to be laid out can be laid out in page 701 in the same way as in the above embodiments.

[0134] S505: Determine the page state and the reward value corresponding to the layout position, and update the page state.

[0135] In this embodiment of the application, the environmental agent in the MDP model can obtain the action performed by the agent (i.e., the layout position) and determine the page state and the reward value corresponding to the action. The reward value can be determined according to layout rules; for example, soft rules can be quantified into reward values. Figure 8 In the example shown, after placing page element 3 on page 701, the distance values ​​of each of the laid-out page elements from the four edges of page 701 can be obtained. For example, the distance values ​​of page element 4 from the four edges are d1, d2, d3, and d4, and the distance values ​​of page element 3 from the four edges are d5, d6, d7, and d8. The number of unique distance values ​​is counted. Based on the number of unique distance values, the reward value corresponding to the page state and the action is determined. The more repeated distance values, the more aligned the page layout. The reward value can be calculated using the following formula:

[0136] reward align =4×nn u -n d -n l -n r

[0137] Where n is the number of elements already laid out on the page, n u ,n d ,n l ,n r This represents the number of unique distance values ​​along the top, bottom, left, and right directions in page 701. In the example above, except for d2 and d6 which are repeated distance values, all others are unique distance values. Therefore, the reward value = 8 - 0 - 2 - 2 - 2 = 2.

[0138] In another embodiment of this application, the soft rule of uniform spacing between page elements can also be quantified as a reward value. Specifically, the set of spacings (A0) between each laid-out page element and its adjacent laid-out page elements in both the horizontal and vertical directions can first be obtained. h A v ), and obtain the set of distances from each laid-out page element to the four sides of page 701 (A u A d Al A r Then, the standard deviation of each spacing set and each distance set can be calculated. The larger the standard deviation, the more uneven the spacing of the laid-out page elements. The reward value can then be calculated using the following formula:

[0139] reward gap =-std(A h )-std(A v )-std(A h )-std(A v )-std(A h )-std(A v )

[0140] Here, std(*) represents the standard deviation operation.

[0141] It should be noted that the soft rules that can be used in page layout are not limited to the examples mentioned above. For different application scenarios, soft rules related to the application scenario can be set. For example, in a page layout with touch components, the reward value obtained by placing the touch component on the right side of the page can be set to be higher. Furthermore, different soft rules contribute different reward values. Based on this, in one embodiment of this application, weights can be set for the reward values ​​quantified by different soft rules, and the final reward value can be determined according to the weights. For example, the determined final reward value can be expressed as:

[0142] r = α × reward align +β×reward gap

[0143] Where α is the weight of the reward value quantified based on the alignment of page elements, and β is the weight of the reward value quantified based on the uniform spacing of page elements.

[0144] Furthermore, after the next page element to be laid out is placed on page 701, the layable and non-layable areas in page 701 change. Based on this, the environment subject can also update the page state and send the calculated reward value and the updated page state to the intelligent agent.

[0145] S507: Iterate through S503 and S505 until at least one page element to be laid out is placed on the page, and generate a page layout strategy.

[0146] By iterating through S503 and S505 above, all page elements to be laid out are placed on the page, and a page layout strategy is generated. The page layout strategy may include the layout position of each page element to be laid out on page 701. It should be noted that the number of page elements to be laid out in each iteration is not limited to one. When there are a large number of page elements to be laid out, two or more page elements to be laid out can be laid out in each iteration, and this application does not impose any restrictions here.

[0147] S509: Accumulate the reward value determined in each iteration, and take the page layout strategy corresponding to the accumulated reward value that meets the preset conditions as the candidate page layout strategy.

[0148] After all page elements to be laid out are placed in page 701, the reward value determined in each iteration can be accumulated. The page layout strategy corresponding to the accumulated reward value meeting a preset condition is used as a candidate page layout strategy. The preset condition may include, for example, the accumulated reward value being greater than a preset reward threshold.

[0149] S305: Determine the target page layout strategy from the at least one candidate page layout strategy using an imitation learning algorithm.

[0150] Based on the above overview of the internal process of the imitation learning module 103, the following specific embodiments illustrate how the imitation learning module 103 determines the target page layout strategy 107 from the at least one candidate page layout strategy 105. Figure 9 A flowchart of an implementation scenario is shown, such as Figure 9 As shown, the imitation learner 901 can be trained using a positive sample set 903 and a negative sample set 905. The positive sample set 903 may include multiple page layout strategy samples that conform to user aesthetics or habits. The users may include designers, ordinary consumers, etc. The negative sample set 905 may include page layout strategy samples obtained using reinforcement learning algorithms corresponding to the positive samples. In this embodiment, the reward function of the imitation learner 901 can be determined using the ranking loss between positive and negative samples.

[0151] In one embodiment of this application, during the training of the imitation learner 901, a set of state-action pairs D can be obtained from the positive samples. human ={(s i ,a i This state action applies to set D. human This can include a complete historical record of the states and actions of human experts during the generation of page layout strategies. On the other hand, a set of state-action pairs D can also be obtained from negative samples. agent ={(s j,a j This state action applies to set D. agent This includes a complete history of the agent's states and actions during the process of generating page layout strategies using reinforcement learning algorithms. Then, the imitation learner 901 can utilize D... human and D agent Training yields a reward function r(s,a) such that D human The reward obtained is greater than or equal to D agent The reward obtained, that is, the reward for obtaining a positive sample, always takes precedence over the reward for obtaining a negative sample. Specifically, the imitation learner 901 can assign a value to state s. i As input data, s i The corresponding action a i Imitation learning is performed using labels, with each state-action pair (s) i ,a i Both can obtain a reward, so D human The reward obtained is the cumulative reward for the state-action pair. Based on D human The reward obtained is greater than or equal to D agent The prerequisite for obtaining the reward is that r(s,a) can ultimately be obtained. It should be noted that the imitation learner 901 can include multi-layer feedforward neural networks, such as convolutional neural networks, etc., and this application does not impose any restrictions here.

[0152] In the embodiments of this application, such as Figure 9 As shown, based on the reward function of the trained imitation learner 901, the at least one candidate page layout strategy 105 can be scored, and the one or more target page layout strategies 107 with the highest scores can be determined. Then, the one or more target page layout strategies 107 can be displayed to the user 907. If the user 907 is satisfied with the displayed one or more target page layout strategies 107, they can directly use them. If the user 907 is dissatisfied with the one or more target page layout strategies 107 and adjusts them, the adjusted target page layout strategy can be used as a positive sample to train the imitation learner 901 to learn the adjusted target page layout strategy, obtaining an optimized reward function, making the aesthetics of the imitation learner 901 closer to those of the user 907.

[0153] In this embodiment, since machine learning techniques for image processing are relatively mature, both the positive sample set 903 and the negative sample set 905 can be in image format during the imitation learning stage. Correspondingly, before inputting the at least one candidate page layout strategy 105 into the imitation learner 901, the candidate page layout strategy 105 can be converted into an image format to maintain consistency with the format of the training data.

[0154] In practical applications, page layout involves numerous layout rules, which may differ across scenarios. However, these rules vary in importance. Therefore, layout rules can be categorized into first-priority layout rules and second-priority layout rules, with the first-priority rules having a higher priority than the second-priority rules. For example, the first-priority layout rules may include hard rules, while the second-priority layout rules may include soft rules. In this embodiment, at least one candidate page layout strategy satisfies the first-priority layout rule, and the reward obtained under the constraint of the second-priority layout rule is greater than a preset reward threshold. In a specific embodiment, the first-priority layout rule can be used as a constraint condition when the agent performs an action, and the second-priority layout rule can be used as the basis for the environment to determine the reward value. This approach ensures that the generated candidate page layout strategies fully satisfy the hard rules and, as far as possible, satisfy the soft rules.

[0155] Considering that in practical application scenarios, if the number of page elements to be laid out is large, the number of possible page layout strategies may increase exponentially. In this embodiment, the Monte Carlo tree search algorithm can be used to reduce the search space of page layout strategies and improve layout efficiency. Specifically, as... Figure 10 As shown, after step 505 and before step 507, the following may be included:

[0156] S1001: Determine the current cumulative reward value, and determine the strategy value of the current page layout strategy based on the cumulative reward value.

[0157] S1003: Determine whether the strategy value of the current page layout strategy is greater than or equal to a preset value threshold.

[0158] In this embodiment of the application, during the reinforcement learning phase, a reward value is obtained for each iteration. Based on the obtained reward value, the cumulative reward value for this iteration can be determined. Based on the cumulative reward value, the strategy value of the current page layout strategy (or the current page state) can be determined. In one example, the relationship between the strategy value and the cumulative reward can be expressed as:

[0159]

[0160] Where m represents the current iteration number. express The reward obtained from a state action.

[0161] If the strategy value of the current page layout strategy is higher, it indicates that the current page layout strategy is more valuable. Based on this, it can be determined whether the strategy value of the current page layout strategy is greater than or equal to a preset value threshold. If it is determined that the strategy value is greater than or equal to the preset value threshold, step 507 can be executed. In another embodiment of this application, such as Figure 10 As shown, if the value of the strategy is determined to be less than the preset value threshold, the current page layout strategy can be excluded, and the process can return to step 501 to start a new layout.

[0162] In this embodiment, the Monte Carlo tree search algorithm is used during reinforcement learning to reduce the search space for page layout, which can significantly reduce search complexity when there are many elements on the page to be laid out. Furthermore, by predicting the long-term rewards of intermediate nodes using a value function, page layout strategies with higher reward values ​​can be obtained while reducing the search space.

[0163] This application also provides another embodiment of the page layout device 100, such as... Figure 11 As shown, the page layout device 100 may include:

[0164] The initial information acquisition module 1101 is used to acquire page information, element information of at least one page element to be laid out, and layout rules;

[0165] The reinforcement learning module 101 is used to obtain at least one candidate page layout strategy based on the page information, the element information of the at least one page element to be laid out, and the layout rules, using a reinforcement learning algorithm.

[0166] The imitation learning module 103 is used to determine the target page layout strategy from the at least one candidate page layout strategy using an imitation learning algorithm.

[0167] Optionally, in one embodiment of this application, the layout rules include a first priority layout rule and a second priority layout rule, wherein the at least one candidate page layout strategy satisfies the first priority layout rule, and the reward obtained under the constraint of the second priority layout rule is greater than a preset reward threshold.

[0168] Optionally, in one embodiment of this application, the reinforcement learning module is specifically used for:

[0169] Step 1: Based on the page information and the element information of the first page element to be laid out, lay out the first page element to be laid out on the page according to the layout rules, and generate the page state of the page.

[0170] Step 2: Based on the page state and the element information of the next page element to be laid out, determine the layout position of the next page element to be laid out in the page according to the layout rules.

[0171] Step 3: Determine the reward value corresponding to the page state and the layout position, and update the page state;

[0172] Iterate through steps 2 and 3 until at least one page element to be laid out is placed on the page, and generate a page layout strategy.

[0173] The reward value determined in each iteration is accumulated, and the page layout strategy that satisfies the preset conditions with the accumulated reward value is taken as the candidate page layout strategy.

[0174] Optionally, in one embodiment of this application, the reinforcement learning module is further configured to:

[0175] Calculate the cumulative value of the reward for each page state and each layout position during the iteration process, and determine the strategy value of the current page layout strategy based on the cumulative value of the reward.

[0176] If the value of the strategy is determined to be no less than a preset value threshold, steps 2 and 3 are iterated until at least one page element to be laid out is placed on the page and a page layout strategy is generated.

[0177] Optionally, in one embodiment of this application, the imitation learning module is specifically used for:

[0178] The score of each candidate page layout strategy is determined based on the reward function obtained from the pre-trained imitation learner, wherein the reward function is determined by the ranking loss of positive and negative samples used to train the imitation learner, and the positive samples include page layout strategies that conform to user aesthetics.

[0179] The target page layout strategy is determined from the at least one page layout strategy layout based on the score.

[0180] Optionally, in one embodiment of this application, the imitation learning module is further configured to:

[0181] Send the target page layout strategy;

[0182] Receive user adjustments to the target page layout strategy;

[0183] The adjusted target page layout strategy is used as a positive sample to train the imitation learner, thereby obtaining an optimized reward function.

[0184] Optionally, in one embodiment of this application, the initial information acquisition module is specifically used for:

[0185] It receives page information configured by the user in the user interface, element information of at least one page element to be laid out, and layout rules.

[0186] The page layout apparatus 100 according to the embodiments of this application can correspondingly execute the methods described in the embodiments of this application, and the above and other operations and / or functions of each module in the page layout apparatus 100 are respectively for implementing Figure 2 , Figure 3 , Figure 5 , Figure 10 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0187] It should also be noted that the embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0188] This application embodiment also provides a device 1200 for implementing the above. Figure 1 The system architecture diagram shown illustrates the function of the page layout device 100. Device 1200 can be a physical device or a cluster of physical devices, or a virtualized device, such as at least one cloud virtual machine in a cloud computing cluster. For ease of understanding, this application provides an example illustrating the structure of device 1200.

[0189] Figure 12 A structural schematic diagram of a device 1200 is provided, as follows: Figure 12As shown, device 1200 includes a bus 1201, a processor 1202, a communication interface 1203, and a memory 1204. The processor 1202, memory 1204, and communication interface 1203 communicate via bus 1201. Bus 1201 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 12 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Communication interface 1203 is used for external communication. For example, obtaining page information or information about at least one page element to be laid out, etc.

[0190] The processor 1202 may be a central processing unit (CPU). The memory 1204 may include volatile memory, such as random access memory (RAM). The memory 1204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, HDD, or SSD.

[0191] The memory 1204 stores executable code, and the processor 1202 executes the executable code to perform the aforementioned page layout method.

[0192] Specifically, in achieving Figure 1 In the case of the illustrated embodiment, and Figure 1 When the modules of the page layout device 100 described in the embodiment are implemented by software, the execution... Figure 1 The software or program code required for the functions of the reinforcement learning module 101 and the imitation learning module 103 is stored in the memory 1204. The processor 1202 executes the program code corresponding to each module stored in the memory 1204, such as the program code corresponding to the reinforcement learning module 101 and the imitation learning module 103, to obtain at least one candidate page layout strategy and determine the target page layout strategy from the at least one candidate page layout strategy. The processor 1202 can also execute, for example, Figure 11 The program code corresponding to the initial information acquisition module 1101.

[0193] Embodiments of this application provide a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0194] Embodiments of this application provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0195] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital video disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing.

[0196] The computer-readable program instructions or code described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0197] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of this application.

[0198] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0199] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0200] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0201] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0202] It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented using hardware (such as circuits or ASICs (Application Specific Integrated Circuits)) that performs the corresponding function or action, or using a combination of hardware and software, such as firmware.

[0203] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0204] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A page layout method, characterized in that, include: Obtain page information, element information of at least one page element to be laid out, and layout rules; Based on the page information, the element information of the at least one page element to be laid out, and the layout rules, at least one candidate page layout strategy is obtained using a reinforcement learning algorithm. The target page layout strategy is determined from the at least one candidate page layout strategy using an imitation learning algorithm; The step of determining the target page layout strategy from the at least one candidate page layout strategy using an imitation learning algorithm includes: The score of each candidate page layout strategy is determined based on the reward function obtained from the pre-trained imitation learner, wherein the reward function is determined by the ranking loss of positive and negative samples used to train the imitation learner, and the positive samples include page layout strategies that conform to user aesthetics. The target page layout strategy is determined from the at least one page layout strategy based on the score.

2. The method according to claim 1, characterized in that, The layout rules include a first priority layout rule and a second priority layout rule. The at least one candidate page layout strategy satisfies the first priority layout rule, and the reward obtained under the constraint of the second priority layout rule is greater than a preset reward threshold.

3. The method according to claim 1, characterized in that, The step of obtaining at least one candidate page layout strategy using a reinforcement learning algorithm based on the page information, the element information of the at least one page element to be laid out, and the layout rules includes: Step 1: Based on the page information and the element information of the first page element to be laid out, lay out the first page element to be laid out on the page according to the layout rules, and generate the page state of the page. Step 2: Based on the page state and the element information of the next page element to be laid out, determine the layout position of the next page element to be laid out in the page according to the layout rules. Step 3: Determine the reward value corresponding to the page state and the layout position, and update the page state; Iterate through steps 2 and 3 until at least one page element to be laid out is placed on the page, and generate a page layout strategy. The reward value determined in each iteration is accumulated, and the page layout strategy that satisfies the preset conditions with the accumulated reward value is taken as the candidate page layout strategy.

4. The method according to claim 3, characterized in that, The iterative steps 2 and 3, until at least one page element to be laid out is placed on the page and a page layout strategy is generated, include: Calculate the cumulative value of the reward for each page state and each layout position during the iteration process, and determine the strategy value of the current page layout strategy based on the cumulative value of the reward. If the value of the strategy is determined to be no less than a preset value threshold, steps 2 and 3 are iterated until at least one page element to be laid out is placed on the page and a page layout strategy is generated.

5. The method according to any one of claims 1-4, characterized in that, After determining the target page layout strategy from the at least one page layout strategy based on the score, the method further includes: Send the target page layout strategy; Receive user adjustments to the target page layout strategy; The adjusted target page layout strategy is used as a positive sample to train the imitation learner, thereby obtaining an optimized reward function.

6. The method according to any one of claims 1-4, characterized in that, The acquisition of page information, element information of at least one page element to be laid out, and layout rules includes: It receives page information configured by the user in the user interface, element information of at least one page element to be laid out, and layout rules.

7. A page layout device, characterized in that, include: The initial information acquisition module is used to acquire page information, element information of at least one page element to be laid out, and layout rules; The reinforcement learning module is used to obtain at least one candidate page layout strategy based on the page information, the element information of the at least one page element to be laid out, and the layout rules, using a reinforcement learning algorithm. The imitation learning module is used to determine the target page layout strategy from the at least one candidate page layout strategy using an imitation learning algorithm. Specifically, the imitation learning module is used for: The score of each candidate page layout strategy is determined based on the reward function obtained from the pre-trained imitation learner, wherein the reward function is determined by the ranking loss of positive and negative samples used to train the imitation learner, and the positive samples include page layout strategies that conform to user aesthetics. The target page layout strategy is determined from the at least one page layout strategy based on the score.

8. The apparatus according to claim 7, characterized in that, The layout rules include a first priority layout rule and a second priority layout rule. The at least one candidate page layout strategy satisfies the first priority layout rule, and the reward obtained under the constraint of the second priority layout rule is greater than a preset reward threshold.

9. The apparatus according to claim 7, characterized in that, The reinforcement learning module is specifically used for: Step 1: Based on the page information and the element information of the first page element to be laid out, lay out the first page element to be laid out on the page according to the layout rules, and generate the page state of the page. Step 2: Based on the page state and the element information of the next page element to be laid out, determine the layout position of the next page element to be laid out in the page according to the layout rules. Step 3: Determine the reward value corresponding to the page state and the layout position, and update the page state; Iterate through steps 2 and 3 until at least one page element to be laid out is placed on the page, and generate a page layout strategy. The reward value determined in each iteration is accumulated, and the page layout strategy that satisfies the preset conditions with the accumulated reward value is taken as the candidate page layout strategy.

10. The apparatus according to claim 9, characterized in that, The reinforcement learning module is also used for: Calculate the cumulative value of the reward for each page state and each layout position during the iteration process, and determine the strategy value of the current page layout strategy based on the cumulative value of the reward. If the value of the strategy is determined to be no less than a preset value threshold, steps 2 and 3 are iterated until at least one page element to be laid out is placed on the page and a page layout strategy is generated.

11. The apparatus according to any one of claims 7-10, characterized in that, The imitation learning module is also used for: Send the target page layout strategy; Receive user adjustments to the target page layout strategy; The adjusted target page layout strategy is used as a positive sample to train the imitation learner, thereby obtaining an optimized reward function.

12. The apparatus according to any one of claims 7-10, characterized in that, The initial information acquisition module is specifically used for: It receives page information configured by the user in the user interface, element information of at least one page element to be laid out, and layout rules.

13. A computing device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1-6 when executing the instructions.

14. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-6.

15. A computer program product, characterized in that, The method includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method described in any one of claims 1-6.

Images