Display layout conversion method and device, computer device and storage medium

Abstract

The application relates to a display layout conversion method and device, computer equipment and a storage medium, which can be applied to the information display, centralized management and the like in the fields of safety and intelligent traffic. The method comprises the following steps: acquiring a unit component set in a to-be-converted layout scheme and unit component information corresponding to unit components in the unit component set; according to the unit component information, merging the unit components in the same region to obtain a basic component and corresponding basic component information; generating a layout diagram corresponding to the basic component according to the basic component information; performing layout segmentation on the layout diagram to obtain a layout segmentation multi-ary tree; wherein each node in the layout segmentation multi-ary tree represents a different layout region; determining a scaling ratio of the basic component, and outputting a target layout conversion scheme according to the layout segmentation multi-ary tree and the scaling ratio. According to the method, the basic components can be arranged according to a certain display order, the layout conversion can be realized, and the method has good generalization and high feasibility.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for converting display layout. Background Technology

[0002] With the development of computer technology, display layout conversion technology has emerged, which is used to convert layout schemes between screens of different sizes. For example, display layout conversion technology can be used to automatically adapt a large-screen visual layout scheme to a small-screen visual layout scheme.

[0003] In traditional technologies, the problem of display layout conversion can be broken down into two sub-problems: layout analysis and automated layout. Common layout analysis methods include comprehensively obtaining document layout information through binarization, contour projection, morphological processing, and connected component analysis, or implementing layout analysis based on neural network models.

[0004] However, the method of comprehensively obtaining document layout information through binarization, contour projection, morphological processing, and connected component analysis relies heavily on manually designed algorithm processes and has poor generalization. Neural network model-based layout analysis methods require a large amount of training data through methods such as annotating visualization screen data. However, the amount of visualization screen data available for annotation is small and the annotation cost is high, making it less feasible. Summary of the Invention

[0005] Therefore, it is necessary to provide a display layout conversion method, apparatus, computer equipment, computer-readable storage medium, and computer program product that has good generalizability and high feasibility to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for converting display layout. The method includes:

[0007] Obtain the set of unit components in the layout scheme to be converted, as well as the unit component information corresponding to the unit components in the set of unit components;

[0008] Based on the unit component information, unit components in the same area are merged to obtain the basic component and its corresponding basic component information;

[0009] Generate a layout diagram corresponding to the basic components based on the basic component information;

[0010] The layout image is divided into sections to obtain a multi-branch tree of layout sections; each node in the multi-branch tree of layout sections represents a different layout area;

[0011] The scaling ratio of the basic components is determined based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme.

[0012] Output the target layout conversion scheme based on the page layout split into a multi-branch tree and the scaling ratio.

[0013] Secondly, this application also provides a display layout conversion device. The device includes:

[0014] The acquisition module is used to acquire the set of unit components in the layout scheme to be converted, as well as the unit component information corresponding to the unit components in the set of unit components;

[0015] The merging module is used to merge unit components within the same area based on the unit component information, to obtain the basic component and its corresponding information.

[0016] The layout generation module is used to generate a layout image corresponding to the basic components based on the basic component information.

[0017] The page layout segmentation module is used to segment the page layout image to obtain a page layout segmentation multi-branch tree; each node in the page layout segmentation multi-branch tree represents a different page area;

[0018] The scaling module is used to determine the scaling ratio of the basic components based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme.

[0019] The processing module is used to output the target layout conversion scheme based on the page layout, which is divided into a multi-branch tree and the scaling ratio.

[0020] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0021] Obtain the set of unit components in the layout scheme to be converted, as well as the unit component information corresponding to the unit components in the set of unit components;

[0022] Based on the unit component information, unit components in the same area are merged to obtain the basic component and its corresponding basic component information;

[0023] Generate a layout diagram corresponding to the basic components based on the basic component information;

[0024] The layout image is divided into sections to obtain a multi-branch tree of layout sections; each node in the multi-branch tree of layout sections represents a different layout area;

[0025] The scaling ratio of the basic components is determined based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme.

[0026] Output the target layout conversion scheme based on the page layout split into a multi-branch tree and the scaling ratio.

[0027] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0028] Obtain the set of unit components in the layout scheme to be converted, as well as the unit component information corresponding to the unit components in the set of unit components;

[0029] Based on the unit component information, unit components in the same area are merged to obtain the basic component and its corresponding basic component information;

[0030] Generate a layout diagram corresponding to the basic components based on the basic component information;

[0031] The layout image is divided into sections to obtain a multi-branch tree of layout sections; each node in the multi-branch tree of layout sections represents a different layout area;

[0032] The scaling ratio of the basic components is determined based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme.

[0033] Output the target layout conversion scheme based on the page layout split into a multi-branch tree and the scaling ratio.

[0034] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0035] Obtain the set of unit components in the layout scheme to be converted, as well as the unit component information corresponding to the unit components in the set of unit components;

[0036] Based on the unit component information, unit components in the same area are merged to obtain the basic component and its corresponding basic component information;

[0037] Generate a layout diagram corresponding to the basic components based on the basic component information;

[0038] The layout image is divided into sections to obtain a multi-branch tree of layout sections; each node in the multi-branch tree of layout sections represents a different layout area;

[0039] The scaling ratio of the basic components is determined based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme.

[0040] Output the target layout conversion scheme based on the page layout split into a multi-branch tree and the scaling ratio.

[0041] The aforementioned display layout conversion method, apparatus, computer equipment, storage medium, and computer program product acquire the set of unit components in the layout scheme to be converted, as well as the unit component information corresponding to the unit components in the set. Based on the unit component information, they merge unit components within the same area to obtain basic components and their corresponding information. This enables analysis of the distribution of unit components in the layout scheme to be converted. By generating a layout diagram corresponding to the basic components based on the basic component information, and segmenting the layout diagram to obtain a multi-branch tree, the layout scheme to be converted can be divided into a set of basic components in conjunction with the display effect. By determining the scaling ratio of the basic components based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme, the target layout conversion scheme is output based on the multi-branch tree and the scaling ratio. The basic components can be arranged in a certain display order to achieve layout conversion. The entire process is applicable to various display layout conversion situations because it does not rely on algorithmic flow, exhibiting good generalization. Furthermore, it is highly feasible because it does not require obtaining a large amount of training data for annotation. Attached Figure Description

[0042] Figure 1 This is a diagram illustrating the application environment of the layout conversion method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating a layout conversion method in one embodiment;

[0044] Figure 3 This is a schematic diagram of components in one embodiment;

[0045] Figure 4 This is a schematic diagram of the layout in one embodiment;

[0046] Figure 5 This is a schematic diagram of the second direction jump point in one embodiment;

[0047] Figure 6 This is a schematic diagram of a multi-branch tree for page segmentation in one embodiment;

[0048] Figure 7 This is a schematic diagram of the layout scheme to be converted in one embodiment;

[0049] Figure 8 This is a schematic diagram of a layout scheme to be evaluated in one embodiment;

[0050] Figure 9 This is a schematic diagram of a layout scheme to be evaluated in another embodiment;

[0051] Figure 10 This is a schematic diagram of the page layout result in one embodiment;

[0052] Figure 11 This is a flowchart illustrating the layout conversion method in another embodiment;

[0053] Figure 12 This is a flowchart illustrating the layout conversion method in yet another embodiment;

[0054] Figure 13 This is a structural block diagram of the layout conversion device shown in one embodiment;

[0055] Figure 14 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] The display layout conversion method provided in this application embodiment can be applied to industries such as security and intelligent transportation for information display and centralized management. Figure 1 Taking the application environment shown as an example, the first terminal 102 is a terminal including a large visual screen, and the second terminal 104 is a terminal including a small visual screen. It is necessary to automatically adapt the large visual screen layout scheme to a small visual screen layout scheme through display layout conversion, thus achieving large-screen to small-screen conversion. At this time, the first terminal 102 will obtain the set of unit components in the layout scheme to be converted, as well as the unit component information corresponding to the unit components in the set; based on the unit component information, it will merge unit components within the same area to obtain basic components and their corresponding information; based on the basic component information, it will generate a layout diagram corresponding to the basic components, and perform layout segmentation on the layout diagram to obtain a layout segmentation multi-branch tree; where each node in the layout segmentation multi-branch tree represents a different layout area; based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme, it will determine the scaling ratio of the basic components, and output the target layout conversion scheme based on the layout segmentation multi-branch tree and the scaling ratio; after obtaining the target layout conversion scheme, the content in the large visual screen can be converted and displayed on the small visual screen of the second terminal 104 according to the target layout conversion scheme. The first terminal 102 can be, but is not limited to, various personal computers, laptops, and IoT devices, such as smart TVs. The second terminal 104 can be, but is not limited to, various smartphones, tablets, IoT devices, and portable wearable devices, such as smart speakers, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can be smartwatches, smart bracelets, and head-mounted devices.

[0058] In one embodiment, such as Figure 2 As shown, a method for converting display layout is provided. This embodiment illustrates the application of this method to a first terminal 102. It is understood that this method can also be applied to a server, or to a system including both a first terminal and a server, and is implemented through the interaction between the first terminal and the server. The server can be a standalone server, a server cluster consisting of multiple servers, or a node on a blockchain. In this embodiment, the method includes the following steps:

[0059] Step 202: Obtain the set of unit components in the layout scheme to be converted, and the unit component information corresponding to the unit components in the set of unit components.

[0060] The "layout scheme to be converted" refers to the layout scheme that needs to be converted. For example, in the conversion of the display layout from a large visualization screen to a small visualization screen, the layout scheme to be converted refers to the layout scheme of the large visualization screen. Specifically, the layout scheme to be converted could refer to the layout scheme displayed on a large visualization screen such as a laptop. The "unit component collection" is a collection of unit components existing in the layout scheme to be converted. A unit component refers to the interface element displayed in the layout scheme to be converted. For example, a unit component could specifically refer to interface elements such as text boxes, buttons, and drop-down menus. Another example is that a unit component could specifically refer to interface elements such as text and numerical values. For instance, such as... Figure 3 As shown in the component diagram, if each unit component is represented by its smallest bounding box, then in Figure 3-1 It includes unit component 1, unit component 2, unit component 3, unit component 4, and unit component 5. Figure 3-2 It includes unit component 6, unit component 7 and unit component 8.

[0061] The unit component information refers to information representing the position of a unit component within the layout scheme to be converted. Specifically, unit component information can refer to the coordinates of the smallest bounding box surrounding the unit component within the layout scheme. For example, it can be the coordinates of the two vertex positions of the smallest bounding box surrounding the unit component in the coordinate system corresponding to the layout scheme. Further, the two vertex positions can specifically refer to the top-left and bottom-right corners. It should be noted that the coordinate system corresponding to the layout scheme to be converted is determined based on a pre-defined origin, which can be set as needed. For example, the origin can specifically refer to the top-left vertex in the entire layout scheme to be converted.

[0062] Specifically, when a display layout conversion is required, the first terminal obtains the set of unit components in the layout scheme to be converted, as well as the corresponding unit component attributes. Based on the component size and position in the unit component attributes, the unit component information is obtained. The unit component attributes describe the unit component, including the component identifier, the corresponding parent component information, and the component style. The component style includes component size, component position, component text content, and component text attribute settings. The component identifier distinguishes different unit components, and the corresponding parent component information determines the hierarchical relationship between unit components. The component size includes the component length *w* and the component width *h*. It should be noted that the component length and width in this application are in pixels. For example, the component size could be a component length of 5 pixels and a component width of 4 pixels. The component position refers to the coordinates (x, y) of the top-left corner vertex of the smallest bounding box surrounding the unit component. It should be noted that the component text content may be empty; that is, if there is no text within the unit component, the component text content will be empty.

[0063] Specifically, obtaining unit component information based on component size and position means determining the component length w of unit component i. i Component width h i and component position (x) i ,y i ), calculate the unit component information [x i ,y i x i +w i ,y i +h i Furthermore, while acquiring unit component information, the first terminal also performs other preprocessing on the unit component attributes. Taking the visual large-screen layout solution as an example, since the current visual large-screen layout solution includes multiple categories, each category includes multiple components, and the keywords of all components are not uniform, preprocessing is required. In the preprocessing stage, keywords are unified and component attributes are standardized. Here, keywords refer to dictionary attribute values. The display of unit components on the visual large screen is transmitted via JSON (JavaScript Object Notation) files, which are dictionary data types. Standardizing component attributes means filling in the missing keys in the dictionary values ​​of all unit components.

[0064] Step 204: Based on the unit component information, merge the unit components in the same area to obtain the basic components and their corresponding basic component information.

[0065] Since the layout scheme to be converted typically contains hundreds of unit components, which are stacked on top of each other, box information is needed to group unit components within the same area into a compact bounding box, that is, the smallest bounding box within that area that does not overlap with other components. A basic component refers to the smallest bounding box within any area that overlaps with other components. Basic component information refers to information representing the position of the basic component in the layout scheme to be converted. For example, basic component information can specifically refer to the coordinates of the basic component in the layout scheme to be converted. For instance, basic component information can specifically refer to the coordinates of the two vertices of the smallest bounding box surrounding the basic component in the coordinate system corresponding to the layout scheme to be converted. Furthermore, the two vertices can specifically refer to the top-left and bottom-right corners.

[0066] Specifically, the first terminal determines the area of ​​each unit component based on the unit component information, sorts the unit components according to their area, obtains the corresponding arrangement order, determines the initial basic component based on the arrangement order, calculates the intersection-union ratio (IUR) between the bounding box of the initial basic component and the unit components in the unit component set, and updates the bounding box of the component based on the calculated IUR and a preset merging condition. That is, after calculating the IUR, it determines whether the corresponding unit component can be merged with the bounding box of the initial basic component, until the IUR between the initial basic component and the unit components in the unit component set has been calculated, and the latest bounding box of the component is used as the target component. The preset merging condition can be set as needed, and this embodiment does not impose specific limitations here. For example, the preset merging condition can specifically refer to the IUR between the bounding box of the initial basic component and the unit components being greater than a preset IUR threshold. Further, the IUR threshold can be 0.9.

[0067] Specifically, after obtaining the target component, the first terminal will update the set of unit components and their arrangement order based on each unit component in the target component, return the steps of determining the initial basic component based on the arrangement order, calculating the component bounding box corresponding to the initial basic component and the intersection-union ratio of the unit components in the set of unit components, until there are no unit components in the latest set of unit components, and take each obtained target component as the basic component, and obtain the basic component information based on the unit component information of the unit components in the target component.

[0068] Step 206: Generate a layout diagram corresponding to the basic components based on the basic component information.

[0069] In this context, a layout diagram refers to a page design obtained by representing the basic components on a canvas based on the basic component information. For example, a layout diagram can be like this: Figure 4As shown, basic components are represented by rectangles. To further illustrate, when converting a large visualization screen to a smaller one, the layout diagram specifically refers to the design drawing that represents the basic components of the large visualization screen on the canvas. In this layout diagram, the basic components of the large visualization screen are represented by rectangles. Similarly, when converting a small visualization screen to a large visualization screen, the layout diagram specifically refers to the design drawing that represents the basic components of the small visualization screen on the canvas. In this layout diagram, the basic components of the small visualization screen are represented by rectangles.

[0070] Specifically, the first terminal will create a canvas with the same resolution and size as the layout scheme to be converted. Based on the basic component information, the basic components are represented on the canvas in sequence using rectangles, generating a layout diagram corresponding to the basic components.

[0071] Step 208: Perform page segmentation on the page layout to obtain a page segmentation multi-branch tree; wherein each node in the page segmentation multi-branch tree represents a different page area.

[0072] Page partitioning refers to dividing a page layout into at least two page regions. A page partitioning multi-way tree is a tree composed of nodes representing different page regions. In a page partitioning multi-way tree, different nodes represent different page regions, thus determining the relationships between each page region. For example, when two nodes are sibling nodes, it means that the two page regions they represent are of the same level and have an equal relationship. As another example, when two nodes are parent and child nodes, it means that the page region represented by the parent node includes the page region represented by the child node. A page region is a portion of the page layout obtained after page partitioning, and a page region may contain one or more basic components.

[0073] Specifically, the first terminal will segment the page layout into at least two page regions, using nodes to represent each segmented page region, thus obtaining a page segmentation multi-branch tree. During page segmentation, the first terminal first performs a first-direction segmentation on the page layout to obtain the page title dividing line, thereby separating the page title area from the page content area. Then, the page content area is segmented, and the page segmentation multi-branch tree is obtained based on the segmentation results. In the process of segmenting the content area of ​​the page, the first terminal first performs a second-direction segmentation on the content area based on the pixel values ​​of the pixels in the content area to obtain a second-direction segmentation line. The content area is then segmented according to the second-direction segmentation line to obtain at least two second-direction regions to be segmented. Each second-direction region to be segmented is then segmented according to the first direction to obtain a first-direction segmentation line corresponding to the second-direction region to be segmented. The corresponding second-direction region to be segmented is then segmented according to the first-direction segmentation line to obtain at least two first-direction regions to be segmented. Finally, each first-direction region to be segmented is segmented a second time until the page area to be segmented within the first-direction region meets the preset segmentation cutoff condition. Based on the results of each segmentation, a page segmentation multi-branch tree is obtained.

[0074] In this design, the first direction and the second direction are perpendicular to each other, and the first direction is the same as the layout direction of the page title. For example, when the layout direction of the page title is horizontal, the first direction is horizontal. Or, when the layout direction of the page title is vertical, the first direction is vertical. The preset segmentation cutoff condition can be set as needed; this embodiment does not impose specific limitations here. For example, the preset segmentation cutoff condition can be that the width of the indivisible and / or the page area to be segmented is less than the target width, where the target width can be set as needed.

[0075] Step 210: Determine the scaling ratio of the basic components based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme.

[0076] The scaling ratio of the basic components refers to the scaling ratio of the basic components, which is determined by the first screen size corresponding to the layout scheme to be converted and the second screen size corresponding to the target layout conversion scheme.

[0077] Specifically, the first terminal determines the scaling ratio of the basic components based on the first screen size corresponding to the layout scheme to be converted and the second screen size corresponding to the target layout conversion scheme. It should be noted that this scaling ratio applies to all unit components within the basic components. For example, the first terminal can determine the scaling ratio of the basic components based on the first screen width (tg_width) in the first screen size and the second screen width (base_width) in the second screen size. The scaling ratio of the basic components is scale = tg_width / base_width.

[0078] Step 212: Output the target layout conversion scheme based on the page layout multi-branch tree and scaling ratio.

[0079] Specifically, after determining the scaling ratio of the basic components, the first terminal obtains the traversal order of the layout segmentation multi-branch tree. Based on the traversal order and scaling ratio, it scales and arranges the unit components in all basic components from top to bottom, outputting the target layout conversion scheme. Furthermore, the traversal order of the layout segmentation multi-branch tree can be at least two. Therefore, when obtaining the target layout conversion scheme, the first terminal obtains the layout scheme to be evaluated corresponding to each traversal order and scaling ratio. It then counts the original layout component values ​​and the converted layout component values ​​corresponding to the layout scheme to be evaluated. Based on the original and converted layout component values, the layout schemes to be evaluated are sorted, and the target layout conversion scheme is output based on the sorting result. Here, the original layout component value refers to the number of basic components counted in each segmented layout area in the selected direction after the layout image corresponding to the layout scheme to be converted is segmented. The converted layout component value refers to the number of basic components counted in each layout area in the selected direction within the layout scheme to be evaluated. The selected direction can be set as needed. For example, for the layout scheme to be converted and the layout scheme to be evaluated, the selected direction can be determined according to the layout length and layout width corresponding to the layout scheme. When the layout length is greater than the layout width, the selected direction is vertical, and when the layout width is greater than the layout length, the selected direction is horizontal.

[0080] The aforementioned display layout conversion method obtains the set of unit components in the layout scheme to be converted, as well as the corresponding unit component information within that set. Based on this information, it merges unit components within the same area to obtain basic components and their corresponding information. This allows for analysis of the distribution of unit components in the layout scheme. By generating a layout diagram corresponding to the basic components based on this information, and then segmenting the diagram to obtain a multi-branch tree, the method can divide the layout scheme to be converted into a set of basic components, taking into account display effects. The scaling ratio of the basic components is determined based on the screen size of the layout scheme to be converted and the target layout conversion scheme. Finally, the target layout conversion scheme is output based on the multi-branch tree and the scaling ratio, arranging the basic components in a specific display order to achieve the display layout conversion. The entire process is applicable to various display layout conversion scenarios because it does not rely on a specific algorithm, demonstrating good generalization. Furthermore, it is highly feasible because it does not require extensive training data for annotation.

[0081] In one embodiment, dividing the layout image into a multi-branch tree of layout division includes:

[0082] The layout is divided in the first direction to obtain the layout title dividing line, the layout title area and the layout content area. The layout title dividing line is used to separate the layout title area and the layout content area.

[0083] The content area of ​​the page is divided, and a page segmentation multi-branch tree is obtained based on the segmentation results.

[0084] The first direction is the same as the layout direction of the page title. The page title dividing line is the dividing line used to separate the page title area from the page content area. The page title area refers to the area where the page title is located, and the page content area refers to the area where the page content is located. For example, in... Figure 4 In the layout diagram shown, the page title dividing line 402 is used to separate the page title area 404 and the page content area 406.

[0085] Specifically, when performing page layout segmentation, the first terminal first uses the pixel values ​​of the pixels in the page layout image to perform first-direction segmentation on the page layout image, determines the first-direction jump point, obtains the page title segmentation line based on the first-direction jump point, and then divides the page title area and the page content area. The page content area is then further segmented, and the page segmentation multi-branch tree is obtained based on the segmentation results.

[0086] In this embodiment, by first segmenting the layout image in the first direction to obtain the layout title dividing line, the layout title area and the layout content area, and then segmenting the layout content area, a layout segmentation multi-branch tree can be obtained based on the segmentation results.

[0087] In one embodiment, dividing the page layout in a first direction to obtain a page title dividing line, a page title area, and a page content area includes:

[0088] Determine the average value of the pixels in the first direction of a single row based on the pixel values ​​of the pixels in the layout image;

[0089] Based on the average value of the first direction in a single row, the pixel values ​​of the pixels in the layout image are binarized. The jump point in the first direction is determined based on the binarization result. The jump point in the first direction is the point where the binarized pixel value changes in the first direction.

[0090] Based on the jump point in the first direction, we obtain the page title dividing line, the page title area, and the page content area.

[0091] Here, "single row first direction" refers to a single row direction within the first direction. For example, when the first direction is horizontal, the single row first direction is the row direction; similarly, when the first direction is vertical, the single row first direction is the column direction. The single row first direction average value refers to the average pixel value of each pixel in a single row along that direction. For example, when the single row first direction is a row direction, the formula for calculating the average value is:

[0092]

[0093] Among them, (x j ,y i ) refers to the pixel in the layout image, I(x) j ,y i ) refers to the pixel value of a pixel, w is the page length, and h is the page width.

[0094] The first direction jump point refers to the point where the binary pixel value changes in the first direction. For example, when the pixel value is binary converted to "0" or "1", the first direction jump point is the point where "0" changes to "1" and "1" changes to "0" in the first direction.

[0095] Specifically, the first terminal determines the average value of the pixels in the first direction of a single row based on the pixel values ​​of the pixels in the layout image. Based on the average value in the first direction of a single row, the pixel values ​​of the pixels in the layout image are binarized. Based on the binarization result, the first and second jump points in the first direction are determined. The jump points in the first direction are the points where the binarized pixel values ​​change in the first direction. The first and second jump points in the first direction are used as the page title dividing points. Based on the page title dividing points, the corresponding page title dividing lines are obtained.

[0096] Specifically, binarizing the pixel values ​​of the pixels in the layout image based on the average value of the first direction of a single row involves comparing the average value of the first direction of a single row with a predetermined binarization threshold. When the average value of the first direction of a single row is less than the binarization threshold, the pixel value of the pixel in the row corresponding to the average value of the first direction of a single row is binarized to 0. When the average value of the first direction of a single row is greater than or equal to the binarization threshold, the pixel value of the pixel in the row corresponding to the average value of the first direction of a single row is binarized to 1. For example, when the first direction of a single row is a row direction, the corresponding judgment formula is as follows:

[0097]

[0098] Where row_thresh(i) is the pixel value after binarization, and thresh is the binarization threshold. The binarization threshold can be calculated. For example, the formula for calculating the binarization threshold is:

[0099]

[0100] The first terminal determines pixel value change points by traversing the binarized pixel values. Based on the first and second jump points in the first direction, it obtains the page title segmentation points. Using these segmentation points, the corresponding page title segmentation lines are derived. For example, when the first direction is horizontal, the page title area and the page content area typically contain two jump points, row_left and row_right. Therefore, the page title segmentation points are s-p1 = row_left / 2 + row_right / 2. The page title segmentation lines are the lines connecting these segmentation points.

[0101] In this embodiment, the average value of a single row of pixels in the first direction is determined based on the pixel values ​​of the pixels in the layout image. Based on the average value of the single row in the first direction, the pixel values ​​of the pixels in the layout image are binarized. The first direction jump point can be determined based on the binarization result. Thus, the layout title dividing line, the layout title area, and the layout content area are obtained using the first direction jump point.

[0102] In one embodiment, the page content area is segmented, and a page segmentation multi-branch tree is obtained based on the segmentation result, including:

[0103] Based on the pixel values ​​of the pixels in the content area of ​​the page, the content area of ​​the page is divided in a second direction to obtain a second direction dividing line, which is perpendicular to the first direction.

[0104] Divide the content area of ​​the page according to the second direction dividing line to obtain at least two second direction areas to be divided;

[0105] The second-direction region to be segmented is segmented in the first direction to obtain the first-direction segmentation line corresponding to the second-direction region to be segmented.

[0106] Based on the first direction dividing line, the corresponding second direction region to be divided is divided to obtain at least two first direction regions to be divided.

[0107] The first direction region to be segmented is further segmented until the page area to be segmented within the first direction region meets the preset segmentation cutoff condition.

[0108] Based on the results of each segmentation, a multi-way tree for page segmentation is obtained.

[0109] The second direction refers to the direction perpendicular to the first direction. For example, when the first direction is horizontal, the second direction is vertical. Or, when the first direction is vertical, the second direction is horizontal. A second-direction dividing line is a dividing line used to divide the content area of ​​a page in a second direction, resulting in the area to be divided in the second direction. For example, in... Figure 4 In the layout diagram shown, when the second direction is vertical, the second direction dividing line 408 is used to divide the content area 406 of the layout in the second direction, resulting in the second direction areas 410, 412, and 414 to be divided. The second direction area to be divided refers to the area obtained after second direction division, which still needs further segmentation. The first direction dividing line refers to the dividing line used to divide the second direction area to be divided in the first direction, resulting in the corresponding first direction area to be divided. For example, in... Figure 4 In the layout diagram shown, when the first direction is horizontal, the first direction dividing line 416 is used to perform first direction division on the second direction region 412 to be divided, resulting in three first direction regions to be divided. The first direction regions to be divided refer to the regions obtained after first direction division that still need to be further divided.

[0110] Specifically, when performing second-direction segmentation on the content area of ​​the page, the first terminal first determines the average value of a single row of pixels in the second direction based on the pixel values ​​of the pixels in the content area. Then, it performs binarization on the pixel values ​​based on the average value of a single row of pixels in the second direction. Based on the binarization result, it obtains the second-direction segmentation line. Next, it segments the content area based on the second-direction segmentation line, resulting in at least two second-direction regions to be segmented. Then, it performs first-direction segmentation on these regions, obtaining the corresponding first-direction segmentation line. Based on the first-direction segmentation line, it segments the corresponding second-direction regions to be segmented, resulting in at least two first-direction regions to be segmented. Finally, it performs secondary segmentation on each first-direction region until the page area to be segmented within the first-direction region meets the preset segmentation cutoff condition. Based on each segmentation result, it obtains a page segmentation multi-branch tree. It should be noted that in this application, when performing second-direction segmentation and first-direction segmentation, the pixel values ​​used in calculating the average value of a single row of pixels in the second direction and the average value of a single row of pixels in the first direction refer to the pixel values ​​on the page image, not the binarized pixel values.

[0111] The preset segmentation cutoff condition can be set as needed, and this embodiment does not impose specific limitations here. For example, the preset segmentation cutoff condition can be that the area to be segmented is indivisible and / or the width of the area to be segmented is less than the target width, where the target width can be set as needed. It should be noted that, except during the secondary segmentation, it is also necessary to determine whether the area to be segmented meets the preset segmentation cutoff condition each time the second direction segmentation and the first direction segmentation are performed, and to stop segmenting the area to be segmented when the preset segmentation cutoff condition is met.

[0112] Furthermore, the method for segmenting the second-direction region to be segmented in the first direction to obtain the first-direction segmentation line corresponding to the second-direction region can be as follows: Based on the pixel values ​​of the pixels in the second-direction region to be segmented, determine the average value of a single row of pixels in the first direction. Based on the average value of a single row of pixels in the first direction, perform binarization processing on the pixel values. Based on the binarization result, determine the first-direction jump points in the second-direction region to be segmented. Use the first-direction jump points as the first-direction segmentation points. Based on the first-direction segmentation points, obtain the first-direction segmentation line. For example, the first-direction segmentation line can be a line connecting the first-direction segmentation points.

[0113] Furthermore, when obtaining the multi-branch tree of page segmentation based on each segmentation result, to prevent the page segmentation from being too fine and causing layout chaos, it is also necessary to filter the segmentation lines. At this time, the first terminal will calculate the distance between every two segmentation lines in the same direction, and filter the segmentation lines by comparing the segmentation line distance with the segmentation distance threshold, filtering out segmentation lines whose distance is less than the segmentation distance threshold. The segmentation distance threshold can be set as needed.

[0114] In this embodiment, by first performing second-direction segmentation, then first-direction segmentation, and finally secondary segmentation, the complete segmentation of the page content area can be achieved, resulting in a page segmentation multi-branch tree.

[0115] In one embodiment, the second-direction segmentation of the content area of ​​the page is performed based on the pixel values ​​of the pixels in the content area to obtain the second-direction segmentation line, including:

[0116] Determine the average value of the pixels in the second direction of a single row based on the pixel values ​​of the pixels in the content area of ​​the page.

[0117] Based on the average value of the second direction in a single row, the pixel values ​​of the pixels in the content area of ​​the page are binarized. The second direction jump point is determined based on the binarization result. The second direction jump point is the point where the binarized pixel value changes in the second direction.

[0118] Determine the second direction split point based on the second direction jump point;

[0119] Based on the dividing point in the second direction, the dividing line in the second direction is obtained.

[0120] In this context, "single-row second direction" refers to a single-row direction within the second direction. For example, when the second direction is horizontal, the single-row second direction is the row direction; similarly, when the second direction is vertical, the single-row second direction is the column direction. The average value of the single-row second direction refers to the average pixel value of each single-row pixel point within that direction. The second-direction jump point refers to the point where the binarized pixel value changes within the second direction. For instance, after binarizing a pixel value to "0" or "1", the second-direction jump point is the point where the value changes from "0" to "1" or from "1" to "0" within the second direction. For example, when the second direction is vertical... Figure 5 As shown, for Figure 3 The two basic components in the middle contain two jump points (left jump point and right jump point), then the second direction split point is s-p2 = left jump point / 2 + right jump point / 2.

[0121] Specifically, the first terminal determines the average value of the pixels in a single row in the second direction based on the pixel values ​​of the pixels in the content area of ​​the page. Then, it performs binarization processing on the pixel values ​​of the pixels in the content area based on this average value. The second-direction jump points are determined based on the binarization result and used as second-direction segmentation points. Finally, second-direction dividing lines are obtained based on these segmentation points. For example, second-direction dividing lines can be obtained by connecting the second-direction segmentation points.

[0122] The average value in the second direction of a single row can be calculated using a formula. For example, when the second direction of a single row is a column direction, the formula for calculating its average value is as follows:

[0123]

[0124] Among them, (x j ,y i ) refers to the pixel in the layout image, I(x) j ,y i ) refers to the pixel value of a pixel, w is the page length, and h is the page width.

[0125] Specifically, binarizing pixel values ​​based on the average value of the second direction in a single row involves comparing the average value of the second direction in a single row with a predetermined binarization threshold. When the average value of the second direction in a single row is less than the binarization threshold, the pixel value of the pixel in the single row corresponding to the average value of the second direction in a single row is binarized to 0. When the average value of the second direction in a single row is greater than or equal to the binarization threshold, the pixel value of the pixel in the single row corresponding to the average value of the second direction in a single row is binarized to 1. For example, when the second direction of a single row is a column direction, the corresponding judgment formula is as follows:

[0126]

[0127] Where line_thresh(i) is the pixel value after binarization, and thresh is the binarization threshold. The binarization threshold can be calculated. For example, the formula for calculating the binarization threshold is:

[0128]

[0129] In this embodiment, the average value of the second direction of a single row of pixels is determined based on the pixel values ​​of the pixels in the content area of ​​the page. Based on the average value of the second direction of a single row, the pixel values ​​of the pixels in the content area of ​​the page are binarized. The second direction jump point can be determined based on the binarization result. Then, the second direction segmentation point is determined using the second direction jump point, and thus the second direction segmentation line is obtained.

[0130] In one embodiment, the output of the target layout conversion scheme based on the page layout multi-branch tree and scaling ratio includes:

[0131] Obtain at least two traversal orders for a page-partitioned multi-way tree;

[0132] Based on the traversal order and scaling ratio, the layout scheme to be evaluated corresponding to the traversal order is obtained;

[0133] Evaluate the proposed layout scheme and output the target layout conversion scheme.

[0134] Traversal refers to visiting each node in a multi-way tree along a search path, one by one. Different traversal methods result in different node visits in different orders. Common traversal methods include preorder traversal, inorder traversal, and postorder traversal. For example, regarding a... Figure 6 The layout shown is a multi-branch tree, and different traversal orders can be obtained by different traversal methods.

[0135] Specifically, the first terminal traverses the layout segmentation multi-branch tree according to a pre-defined traversal method, obtaining at least two traversal orders for the tree. It then scales and arranges the basic components sequentially from top to bottom according to the traversal order, resulting in a layout scheme to be evaluated corresponding to the traversal order. This layout scheme is then evaluated to determine and output the target layout conversion scheme, i.e., the final layout conversion scheme. For example, depending on the traversal order, such as... Figure 7 The layout scheme to be converted shown can be evaluated as follows: Figure 8 and Figure 9 As shown.

[0136] In this embodiment, by obtaining at least two traversal orders of the page segmentation multi-branch tree, and based on the traversal order and scaling ratio, a layout scheme to be evaluated corresponding to the traversal order is obtained. The layout scheme to be evaluated is then evaluated, and a target layout conversion scheme can be output.

[0137] In one embodiment, the evaluation of the typesetting scheme to be evaluated, and the output of the target typesetting conversion scheme, include:

[0138] Statistics on the original and converted page component values ​​corresponding to the layout scheme to be evaluated;

[0139] The first component density is obtained based on the original layout component values, and the second component density is obtained based on the converted layout component values;

[0140] The layout schemes to be evaluated are sorted according to the density of the first component and the density of the second component;

[0141] Output the target layout conversion scheme based on the sorting results.

[0142] The original layout component values ​​refer to the number of basic components within each segmented area of ​​the layout image corresponding to the layout scheme to be converted, counted in the selected direction. The converted layout component values ​​refer to the number of basic components within each layout area of ​​the layout scheme to be evaluated, counted in the selected direction. The selected direction can be set as needed. For example, for the layout scheme to be converted and the layout scheme to be evaluated, the selected direction can be determined based on the layout length and layout width corresponding to the layout scheme. When the layout length is greater than the layout width, the selected direction is vertical; when the layout width is greater than the layout length, the selected direction is horizontal. The first component density refers to the variance between the original layout component values, and the second component density refers to the variance between the converted layout component values.

[0143] Specifically, the first terminal will collect the original layout component values ​​and the converted layout component values ​​corresponding to the layout scheme to be evaluated, calculate the variance between the original layout component values ​​to obtain the first component density, calculate the variance between the converted layout component values ​​to obtain the second component density, calculate the difference between the first component density and the second component density to obtain the relative component density, sort the layout schemes to be evaluated according to the first component density and the relative component density, and determine and output the target layout conversion scheme according to the sorting results.

[0144] In this embodiment, by statistically analyzing the original layout component values ​​and the converted layout component values ​​corresponding to the layout scheme to be evaluated, a first component density is obtained based on the original layout component values, and a second component density is obtained based on the converted layout component values. The first component density and the second component density can be used to sort the layout scheme to be evaluated, and then the target layout conversion scheme can be output based on the sorting results.

[0145] In one embodiment, the statistical analysis of the original layout component values ​​and the transformed layout component values ​​corresponding to the layout scheme to be evaluated includes:

[0146] Determine the first selected direction corresponding to the typesetting scheme to be converted, and determine the second selected direction corresponding to the typesetting scheme to be evaluated;

[0147] In the first selected direction, count the number of basic components in each segmented page area corresponding to the layout scheme to be converted, and obtain the original page component values;

[0148] In the second selected direction, the number of basic components in each page area of ​​the layout scheme to be evaluated is counted to obtain the converted page component value.

[0149] The first selected direction refers to the direction in which the values ​​of the original layout components are statistically analyzed. The second selected direction refers to the direction in which the values ​​of the transformed layout components are statistically analyzed.

[0150] Specifically, the first terminal acquires the layout length and width of the layout scheme to be converted. Based on these dimensions, it determines a first selected direction corresponding to the layout scheme to be converted. It then acquires the layout length and width of the layout scheme to be evaluated. Based on these dimensions, it determines a second selected direction corresponding to the layout scheme to be evaluated. After determining the first and second selected directions, the first terminal counts the number of basic components in each segmented page area corresponding to the layout scheme to be converted along the first selected direction, obtaining the original page component values. Similarly, along the second selected direction, it counts the number of basic components in each page area of ​​the layout scheme to be evaluated, obtaining the converted page component values.

[0151] For example, such as Figure 10 As shown, for the same as Figure 4 The layout of the page diagram shows a page segmentation result. Since the layout length is greater than the layout width, the first selected direction can be determined as vertical. Figure 10 As can be seen, the page is divided vertically into three regions: 1001, 1002, and 1003. Region 1001 contains 7 basic components, region 1002 contains 6, and region 1003 contains 7. Therefore, the corresponding original component counts are 7, 6, and 7, respectively. It should be noted that since the number of basic components within each vertically divided region is counted, both the title and content areas are included in the count. For example, [the text abruptly ends here]. Figure 10 As can be seen, within the segmented page area 1001, the number of basic components in the page title area is 1, and the number of basic components in the page content area is 6.

[0152] In this embodiment, by determining the first selected direction and the second selected direction, it is possible to perform statistics on the original layout component values ​​in the first selected direction and to perform statistics on the converted layout component values ​​in the second selected direction.

[0153] In one embodiment, determining the first selected direction corresponding to the typesetting scheme to be converted includes:

[0154] Obtain the layout length and width of the layout scheme to be converted;

[0155] When the layout length is greater than the layout width, the first selected direction is determined to be vertical;

[0156] When the layout length is less than the layout width, the first selected direction is determined to be horizontal.

[0157] Specifically, the first terminal will obtain the layout length and layout width of the layout scheme to be converted, compare the layout length and layout width, and determine the first selected direction as vertical when the layout length is greater than the layout width, and determine the first selected direction as horizontal when the layout length is less than the layout width.

[0158] In this embodiment, by obtaining the layout length and layout width and comparing the layout length and layout width, the first selected direction can be determined.

[0159] In one embodiment, ranking the layout schemes to be evaluated based on the density of the first component and the density of the second component includes:

[0160] Calculate the difference between the density of the first component and the density of the second component to obtain the relative component density;

[0161] Based on the density of the first component and the relative component density, the initial score corresponding to the layout scheme to be evaluated is deducted;

[0162] The layout schemes to be evaluated are sorted according to the deduction results.

[0163] Specifically, the first terminal calculates the difference between the density of the first component and the density of the second component to obtain the relative component density. Based on the first component density and the relative component density, the initial score corresponding to the layout scheme to be evaluated is deducted, that is, the first component density and the relative component density are deducted from the initial score. The layout schemes to be evaluated are sorted according to the deduction result, and the layout scheme corresponding to the highest score is selected as the target layout conversion scheme. The initial score can be set as needed, and all layout schemes to be evaluated have the same initial score.

[0164] In this embodiment, by calculating the relative component density based on the first component density and the second component density, the initial score corresponding to the layout scheme to be evaluated can be deducted based on the first component density and the relative component density, and the layout scheme to be evaluated can be sorted according to the deduction result.

[0165] In one embodiment, based on the unit component information, unit components within the same region are merged to obtain the basic component and its corresponding basic component information, including:

[0166] Based on the unit component information, the unit components are sorted to obtain the corresponding arrangement order of the unit components;

[0167] Determine the initial basic components based on the arrangement order;

[0168] Based on the component bounding box and unit component set corresponding to the initial basic component, merge the unit components in the same area as the initial basic component to obtain the target component;

[0169] Remove the unit components from the target component set and their order, and return to the steps of determining the initial basic components based on the order, until no unit components remain in the latest set of unit components;

[0170] Each target component is used as a base component, and the base component information is obtained based on the unit component information of the unit components in the target component.

[0171] In this context, the initial basic component refers to the unit component with the largest area in the arrangement order. The bounding box algorithm is an algorithm for finding the optimal bounding space. Its basic idea is to approximate a complex geometric object with a slightly larger volume and simpler characteristics (called a bounding box). In this embodiment, component bounding boxes are used to approximate unit components within the same region. The component bounding box is the smallest bounding box within that region that does not overlap with other unit components. For example, the component bounding box in this embodiment can be a rectangular bounding box.

[0172] Specifically, the first terminal determines the area of ​​the unit component based on the unit component information, sorts the unit components according to their area, obtains the arrangement order of the unit components, selects the unit component with the largest area as the initial basic component according to the arrangement order, calculates the intersection-union ratio of the bounding box of the initial basic component and the unit components in the unit component set, and merges the unit components in the same area as the initial basic component according to the intersection-union ratio to obtain the target component. Obtaining a target component indicates that one component merging has been completed. At this point, the first terminal needs to remove the unit components in the target component from the unit component set and the arrangement order, and then return to the step of determining the initial base component according to the arrangement order. Select the unit component with the largest current area from the updated arrangement order as the new initial base component. Based on the bounding box of the new initial base component and the updated unit component set, merge the unit components in the same area as the initial base component to obtain the target component. Remove the unit components in the target component from the unit component set and the arrangement order, and return to the step of determining the initial base component according to the arrangement order again, until there are no unit components in the latest unit component set, that is, all unit components have been merged. Use the target component obtained each time as the base component, and obtain the base component information based on the unit component information of the unit components in the target component.

[0173] Specifically, obtaining basic component information based on the unit component information of the target component refers to determining the boundary coordinates of the target component by comparing the unit component information, and using these boundary coordinates as the basic component information. For example, the basic component information could specifically be:

[0174]

[0175] Among them, boxi (i is from 0 to m) is used to represent a unit component in the target component, box i [0] represents the x-coordinate of the top-left corner in the unit component information, box i [1] represents the vertical coordinate of the top left corner of the unit component information, box i [2] indicates the horizontal coordinate of the upper right corner of the unit component information, box i [3] represents the vertical coordinate of the upper right corner of the unit component information.

[0176] In this embodiment, by determining the arrangement order of the unit components based on the unit component information, and determining the initial basic component based on the arrangement order, it is possible to merge the unit components in the same area as the initial basic component based on the component bounding box and the set of unit components corresponding to the initial basic component, thereby obtaining the target component. Then, the unit components in the target component can be removed from the set of unit components and the arrangement order, and the step of determining the initial basic component based on the arrangement order can be returned until there are no unit components in the latest set of unit components. Each target component obtained is used as the basic component, and the basic component information is obtained based on the unit component information of the unit components in the target component, thus completing the merging of all unit components.

[0177] In one embodiment, based on the component bounding box and unit component set corresponding to the initial basic component, unit components within the same region as the initial basic component are grouped to obtain the target component, which includes:

[0178] Calculate the bounding box of the initial basic component and the intersection-union ratio of the unit components in the unit component set;

[0179] Based on the calculated intersection-union ratio and preset merging conditions, update the component bounding box and use the latest component bounding box as the target component.

[0180] The intersection-union ratio refers to the overlap rate between the initial basic component and the unit component, that is, the ratio of their intersection to their union.

[0181] Specifically, the first terminal calculates the intersection-union ratio (IUR) of the bounding box of the initial basic component and the unit components in the unit component set. Based on the calculated IUR and the preset merging conditions, the bounding box of the component is updated. That is, after calculating the IUR, it determines whether the corresponding unit component can be merged with the bounding box of the initial basic component. This process continues until the IUR between the initial basic component and the unit components in the unit component set has been calculated. The latest bounding box of the component is then used as the target component, which includes at least one unit component.

[0182] The following is a pair Figure 3-1 The merging of five unit components is used as an example to illustrate the component merging involved in this embodiment.

[0183] like Figure 3-1 As shown, based on the unit component information, the five unit components are sorted, resulting in the following arrangement order: Unit Component 1 > Unit Component 5 > Unit Component 2 = Unit Component 3 = Unit Component 4. Therefore, the initial base component is determined to be Unit Component 1 based on this arrangement order. First, the intersection-union ratio (IUU) of the bounding box of Unit Component 1 (i.e., the smallest bounding box containing only Unit Component 1) and Unit Component 2 is calculated. If, according to the preset merging condition, the bounding box of Unit Component 1 and Unit Component 2 can be merged, the bounding box is updated. At this point, the bounding box is the smallest bounding box encompassing both Unit Component 1 and Unit Component 2. Then, the IUU of the updated bounding box and Unit Component 3 is calculated. If, according to the preset merging condition, the updated bounding box of Unit Component 1 and Unit Component 2 can be merged, then... The component bounding box is updated next. At this point, the bounding box is the smallest bounding box surrounding unit components 1, 2, and 3. The intersection-union ratio (IUU) of the latest bounding box with unit component 4 is then calculated. If, according to the preset merge condition, the updated bounding box and unit component 4 can be merged, the component bounding box is updated again. At this point, the bounding box is the smallest bounding box surrounding unit components 1, 2, 3, and 4. Finally, the IUU of the latest bounding box with unit component 5 is calculated. If, according to the preset merge condition, the latest bounding box and unit component 5 can be merged, the component bounding box is updated. At this point, the bounding box is the smallest bounding box surrounding unit components 1, 2, 3, 4, and 5. It should be noted that a special case here is... Figure 3-1 In the diagram, the bounding box of unit component 1 can encompass all four other unit components. Therefore, the bounding box after each update is the smallest bounding box of unit component 1. Correspondingly, in Figure 3-2 If unit components 6, 7, and 8 can be merged, then the component bounding box is the smallest bounding box that surrounds unit components 6, 7, and 8.

[0184] In this embodiment, by calculating the intersection-union ratio of the component bounding box corresponding to the initial basic component and the unit components in the unit component set, the component bounding box can be updated using the calculated intersection-union ratio and the preset merging conditions, thereby merging the initial basic component and other unit components, and using the latest component bounding box as the target component.

[0185] In one embodiment, taking the automatic adaptation of a visual large-screen layout scheme to a mobile layout as an example, the display layout conversion method of this application is illustrated. The flowchart of this display layout conversion method can be as follows: Figure 11 As shown, where:

[0186] The first terminal first obtains the set of unit components in the layout scheme to be converted, as well as the unit component attributes corresponding to the unit components in the set. Preprocessing is then performed through unified keywords, standardized component attributes, and calculation of unit component information. Specifically, the first terminal can determine the component length w of unit component i. i Component width h i and component position (x) i ,y i ), calculate the unit component information [x i ,y i x i +w i ,y i +h i ].

[0187] After obtaining the unit component information, the terminal can merge components based on the unit component information. At this time, the first terminal will sort the unit components according to the unit component information to obtain the corresponding arrangement order of the unit components. Based on the arrangement order, the initial basic component is determined. The intersection-union ratio (IUU) of the bounding box of the initial basic component and the unit components in the unit component set is calculated. Based on the calculated IUU and the preset merging conditions, the component bounding box is updated. The latest component bounding box is used as the target component. The unit components in the target component are removed from the unit component set and the arrangement order. The process of determining the initial basic component based on the arrangement order and calculating the IUU of the bounding box of the initial basic component and the unit components in the unit component set is repeated until there are no unit components in the latest unit component set. The target component obtained each time is used as the basic component, and the basic component information is obtained based on the unit component information of the unit components in the target component.

[0188] After obtaining the basic component information, the first terminal can generate a layout diagram corresponding to the basic components, divide the layout diagram into a multi-branch tree, determine the scaling ratio of the basic components, obtain the three traversal orders of the multi-branch tree (i.e., layout division strategy 1, layout division strategy 2, and layout division strategy 3), obtain the layout scheme to be evaluated corresponding to the traversal order based on the traversal order and scaling ratio, evaluate the layout scheme to be evaluated, and obtain and output the target layout conversion scheme.

[0189] The method for segmenting the layout image to obtain a multi-branch tree is as follows: The layout image is segmented in the first direction to obtain the layout title dividing line, the layout title area, and the layout content area. Based on the pixel values ​​of the pixels in the layout content area, the average value of the pixels in a single row in the second direction is determined. Based on the average value of the pixels in a single row in the second direction, the pixel values ​​are binarized. Based on the binarization result, the second direction jump point is determined. The second direction jump point is the point where the binarized pixel value changes. Based on the second direction jump point, the second direction dividing point is determined. Based on the second direction dividing point, the second direction dividing line is obtained. The second direction is perpendicular to the first direction. The layout content area is divided according to the second direction dividing line to obtain at least two second direction areas to be divided. The second direction areas to be divided are then divided according to the first direction to obtain the first direction dividing line corresponding to the second direction areas to be divided. According to the first direction dividing line, the corresponding second direction areas to be divided are divided to obtain at least two first direction areas to be divided. The first direction areas to be divided are then divided a second time until the layout areas to be divided within the first direction areas meet the preset division cutoff condition. Based on the results of each division, a layout division multi-branch tree is obtained.

[0190] The evaluation process for the proposed layout scheme to obtain the target layout conversion scheme involves: statistically analyzing the original and converted page component values ​​corresponding to the proposed layout scheme; obtaining the first component density based on the variance between the original and converted page component values; obtaining the second component density based on the variance between the converted page component values; obtaining the relative component density based on the difference between the first and second component densities; deducting the initial score of the proposed layout scheme based on the first and relative component densities; ranking the proposed layout schemes based on the deduction results; and selecting the proposed layout scheme with the highest score as the target layout conversion scheme, i.e., the mobile layout. For example, regarding... Figure 7 The typesetting scheme to be converted, and the corresponding typesetting scheme to be evaluated can be Figure 8 and Figure 9 As shown, the final mobile layout can be selected through effect evaluation.

[0191] In one embodiment, such as Figure 12 As shown, a flowchart is provided to illustrate the display layout conversion method of this application. The display layout conversion method specifically includes the following steps:

[0192] Step 1202: Obtain the set of unit components in the layout scheme to be converted, and the unit component information corresponding to the unit components in the set of unit components.

[0193] Specifically, when a display layout conversion is required, the first terminal obtains the set of unit components in the layout scheme to be converted, as well as the corresponding unit component attributes. Based on the component size and position in the unit component attributes, the unit component information is obtained. The unit component attributes describe the unit component, including the component identifier, the corresponding parent component information, and the component style. The component style includes component size, component position, component text content, and component text attribute settings.

[0194] Step 1204: Based on the unit component information, sort the unit components to obtain the corresponding arrangement order of the unit components.

[0195] Specifically, the first terminal will determine the size of the unit component based on the unit component information, and sort the unit components according to the size to obtain the corresponding arrangement order of the unit components.

[0196] Step 1206: Determine the initial basic components according to the arrangement order.

[0197] Specifically, the first terminal will select the unit component with the largest area in the arrangement order as the initial basic component.

[0198] Step 1208: Calculate the component bounding box corresponding to the initial basic component and the intersection-union ratio of the unit components in the unit component set.

[0199] The intersection-union ratio refers to the overlap rate between the initial basic components and the unit components, that is, the ratio of their intersection to their union.

[0200] Specifically, the first terminal will determine the component bounding box corresponding to the initial basic component, that is, the smallest bounding box that surrounds the initial basic component, and calculate the intersection-union ratio of the component bounding box corresponding to the initial basic component and the unit components in the unit component set.

[0201] Step 1210: Update the component bounding box according to the calculated intersection-union ratio and the preset merging conditions, and use the latest component bounding box as the target component.

[0202] Specifically, after calculating the intersection-union ratio (CUN), the first terminal determines whether the unit component corresponding to the CUN can be merged with the initial base component based on the calculated CUN and preset merging conditions. When the CUN meets the preset merging conditions, it means that the unit component can be merged with the initial base component. The first terminal merges the unit component corresponding to the CUN and the initial base component and updates the component bounding box. At this time, the component bounding box is the smallest bounding box that simultaneously encloses the unit component corresponding to the CUN and the initial base component. After completing one update, when calculating the CUN with the next unit component, the first terminal will use the updated component bounding box for calculation. That is, after each CUN calculation, it will determine whether merging is possible. If merging is possible, the component bounding box will be updated, and the next calculation will be performed. If merging is not possible, the component bounding box will not be updated, and the next calculation will be performed. In other words, the component bounding box used in each CUN calculation is the one obtained from the most recent update.

[0203] Step 1212: Remove the unit components from the target component set and their order, return to step 1206, and continue until no unit components are found in the latest set of unit components, then proceed to step 1214.

[0204] Specifically, in the target component, the unit component is a unit component that has already been merged. Before the next merging, the first terminal needs to remove the target component from the unit component set and the arrangement order, and then determine the unit component with the largest current area from the updated arrangement order as the new initial basic component, and start the component merging again until there are no unit components in the latest unit component set.

[0205] Step 1214: Take the target component obtained each time as the base component, and obtain the base component information based on the unit component information of the unit component in the target component.

[0206] Specifically, when no unit component exists in the latest set of unit components, it means that all unit components have been merged. At this time, the first terminal will use the target component obtained each time as the base component, and determine the boundary coordinates of the target component by comparing the unit component information of the unit components in the target component. The boundary coordinates are then used as the base component information. Step 1216: Based on the base component information, generate the layout diagram corresponding to the base component.

[0207] Specifically, the first terminal will create a canvas with the same resolution and size as the layout scheme to be converted. Based on the basic component information, the basic components are represented on the canvas in sequence using rectangles, generating a layout diagram corresponding to the basic components.

[0208] Step 1218: Determine the average value of the first direction of a single row of pixels based on the pixel values ​​of the pixels in the layout image.

[0209] Specifically, the first terminal first calculates the total number of pixels in each row along the first direction based on the pixel values ​​of the pixels in the layout image. Then, it divides the total number of pixels by the corresponding layout size to obtain the average pixel value of each row along the first direction, i.e., the average pixel value of a row along the first direction. The corresponding layout size refers to the layout size corresponding to the first direction. When the first direction is horizontal, the layout size is the layout length; when the first direction is vertical, the layout size is the layout width.

[0210] Step 1220: Based on the average value of the first direction in a single row, perform binarization processing on the pixel values ​​of the pixels in the layout image, and determine the first direction jump point based on the binarization processing result. The first direction jump point is the point where the binarized pixel value changes in the first direction.

[0211] Specifically, the first terminal compares the average value of a single row in the first direction with a pre-determined binarization threshold. When the average value of a single row in the first direction is less than the binarization threshold, the pixel value of the pixel corresponding to the average value of the single row in the first direction is binarized to 0. When the average value of a single row in the first direction is greater than or equal to the binarization threshold, the pixel value of the pixel corresponding to the average value of the single row in the first direction is binarized to 1. After completing the binarization process, the terminal iterates through the binarized pixel values ​​of the pixels. By iterating through the pixels, it can determine the pixel value change points, i.e., the jump points in the first direction.

[0212] Step 1222: Based on the first direction jump point, obtain the page title dividing line, page title area, and page content area.

[0213] Specifically, after determining the first direction jump point, the first terminal will obtain the page title dividing point based on the first and second first direction jump points. Using the page title dividing point, the corresponding page title dividing line, page title area, and page content area will be obtained.

[0214] Step 1224: Determine the average value of the second direction of a single row of pixels based on the pixel values ​​of the pixels in the content area of ​​the page.

[0215] Specifically, the first terminal first calculates the total number of pixels in each row of pixels in the content area along the second direction, based on the pixel values ​​of the pixels in that area. Then, it divides the total number of pixels by the corresponding page size to obtain the average number of pixels in each row along the second direction, i.e., the average number of pixels per row along the second direction. The corresponding page size refers to the page size corresponding to the second direction. When the second direction is horizontal, the page size is the page length; when the second direction is vertical, the page size is the page width.

[0216] Step 1226: Based on the average value of the second direction in a single row, perform binarization processing on the pixel values ​​of the pixels in the content area of ​​the page. Determine the second direction jump point based on the binarization processing result. The second direction jump point is the point where the binarized pixel value changes in the second direction. The second direction is perpendicular to the first direction.

[0217] Specifically, the first terminal compares the average value of a single row in the second direction with a pre-determined binarization threshold. When the average value of a single row in the second direction is less than the binarization threshold, the pixel value of the pixel corresponding to the average value of the single row in the second direction is binarized to 0. When the average value of a single row in the second direction is greater than or equal to the binarization threshold, the pixel value of the pixel corresponding to the average value of the single row in the second direction is binarized to 1. After completing the binarization process, the terminal iterates through the binarized pixel values ​​of the pixels. By iterating through the pixels, it can determine the pixel value change points, i.e., the jump points in the second direction.

[0218] Step 1228: Determine the second direction split point based on the second direction jump point.

[0219] Specifically, the first terminal determines a second-direction dividing point based on every two adjacent second-direction jump points. For example, when the second direction is vertical, such as... Figure 5 As shown, for Figure 3 The two basic components in the middle contain two jump points (left jump point and right jump point), then the second direction split point is s-p2 = left jump point / 2 + right jump point / 2.

[0220] Step 1230: Obtain the second direction dividing line based on the second direction dividing point.

[0221] Specifically, the first terminal obtains the second-direction dividing line by connecting the second-direction dividing points in the same single row in the second direction. Step 1232: Divide the content area of ​​the page according to the second-direction dividing line to obtain at least two second-direction areas to be divided.

[0222] Specifically, the first terminal will use the second-direction dividing line to divide the content area of ​​the page, resulting in at least two second-direction areas to be divided.

[0223] Step 1234: Perform first-direction segmentation on the second-direction region to be segmented to obtain the first-direction segmentation line corresponding to the second-direction region to be segmented.

[0224] Specifically, the first terminal determines the average value of a single row of pixels in the first direction based on the pixel values ​​of the pixels in the second direction region to be segmented. Based on the average value of the single row of pixels in the first direction, the pixel values ​​are binarized. Based on the binarization result, the first direction jump point in the second direction region to be segmented is determined. The first direction jump point is used as the first direction segmentation point. Based on the first direction segmentation point, the first direction segmentation line is obtained.

[0225] Step 1236: Based on the first direction dividing line, divide the corresponding second direction region to be divided to obtain at least two first direction regions to be divided.

[0226] Specifically, the first terminal will divide the corresponding second direction region to be divided according to the first direction dividing line to obtain at least two first direction regions to be divided.

[0227] Step 1238: Perform secondary segmentation on the first direction region to be segmented until the page area to be segmented within the first direction region meets the preset segmentation cutoff condition.

[0228] Specifically, after obtaining the first direction region to be segmented, the first terminal will continue to perform secondary segmentation on the first direction region until the page area to be segmented within the first direction region meets the preset segmentation cutoff condition. The preset segmentation cutoff condition can be set as needed; this embodiment does not impose specific limitations here. For example, the preset segmentation cutoff condition can be that the page area to be segmented is indivisible and / or the width of the page area to be segmented is less than the target width, where the target width can be set as needed. It should be noted that, in addition to the secondary segmentation, each time the second direction segmentation and the first direction segmentation are performed, it is also necessary to determine whether the page area to be segmented meets the preset segmentation cutoff condition, and if the preset segmentation cutoff condition is met, the segmentation of the page area to be segmented is stopped.

[0229] Step 1240: Based on the results of each segmentation, obtain the multi-branch tree of the page segmentation.

[0230] It should be noted that when obtaining the multi-branch tree of page segmentation based on each segmentation result, in order to prevent the page segmentation from being too fine and causing layout chaos, it is also necessary to filter the segmentation lines. At this time, the first terminal will calculate the distance between every two segmentation lines in the same direction, and filter the segmentation lines by comparing the segmentation line distance with the segmentation distance threshold, filtering out the segmentation lines whose distance is less than the segmentation distance threshold. The segmentation distance threshold can be set as needed.

[0231] Step 1242: Determine the scaling ratio of the basic components based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme.

[0232] Specifically, the first terminal will use the ratio of the screen size corresponding to the layout scheme to be converted to the screen size corresponding to the target layout conversion scheme as the scaling ratio of the basic components.

[0233] Step 1244: Obtain at least two traversal orders for the page partitioning multi-way tree.

[0234] Specifically, the first terminal will traverse the page segmentation multi-branch tree according to the pre-set traversal method to obtain at least two traversal orders for the page segmentation multi-branch tree.

[0235] Step 1246: Based on the traversal order and scaling ratio, obtain the layout scheme to be evaluated corresponding to the traversal order.

[0236] Specifically, the first terminal will scale the basic components sequentially according to the traversal order and arrange them from top to bottom to obtain the layout scheme to be evaluated corresponding to the traversal order.

[0237] Step 1248: Calculate the original layout component values ​​and the converted layout component values ​​corresponding to the layout scheme to be evaluated.

[0238] Specifically, the first terminal determines a first selected direction corresponding to the layout scheme to be converted, and a second selected direction corresponding to the layout scheme to be evaluated. In the first selected direction, the number of basic components in each segmented page area corresponding to the layout scheme to be converted is counted to obtain the original page component value. In the second selected direction, the number of basic components in each page area of ​​the layout scheme to be evaluated is counted to obtain the converted page component value. The method for determining the first and second selected directions is as follows: the layout length and width of the layout scheme are obtained, and the layout length and width are compared. When the layout length is greater than the layout width, the first selected direction is determined to be vertical; when the layout length is less than the layout width, the first selected direction is determined to be horizontal.

[0239] Step 1250: Obtain the first component density based on the original layout component values, and obtain the second component density based on the converted layout component values.

[0240] Specifically, the first terminal calculates the difference between the original layout component values ​​and uses this difference as the first component density, and calculates the difference between the converted layout component values ​​and uses this difference as the second component density.

[0241] Step 1252: Sort the layout schemes to be evaluated according to the density of the first component and the density of the second component.

[0242] Specifically, the first terminal calculates the difference between the density of the first component and the density of the second component to obtain the relative component density. Based on the density of the first component and the relative component density, the initial score corresponding to the layout scheme to be evaluated is deducted, and the layout scheme to be evaluated is sorted according to the deduction result.

[0243] Step 1254: Output the target layout conversion scheme based on the sorting results.

[0244] Specifically, the first terminal will determine the layout scheme with the highest score to be evaluated as the target layout conversion scheme based on the sorting results.

[0245] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0246] Based on the same inventive concept, this application also provides a display layout conversion device for implementing the display layout conversion method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more display layout conversion device embodiments provided below can be found in the limitations of the display layout conversion method described above, and will not be repeated here.

[0247] In one embodiment, such as Figure 13 As shown, a display layout conversion device is provided, including: an acquisition module 1302, a merging module 1304, a layout generation module 1306, a layout segmentation module 1308, a scaling module 1310, and a processing module 1312, wherein:

[0248] The acquisition module 1302 is used to acquire the set of unit components in the layout scheme to be converted and the unit component information corresponding to the unit components in the set of unit components;

[0249] The merging module 1304 is used to merge unit components in the same area based on the unit component information to obtain the basic component and the corresponding basic component information.

[0250] The layout generation module 1306 is used to generate a layout image corresponding to the basic components based on the basic component information.

[0251] The page segmentation module 1308 is used to segment the page image to obtain a page segmentation multi-branch tree; wherein each node in the page segmentation multi-branch tree represents a different page area;

[0252] The scaling module 1310 is used to determine the scaling ratio of the basic components based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme.

[0253] Processing module 1312 is used to output the target layout conversion scheme based on the page layout splitting into a multi-branch tree and scaling ratio.

[0254] The aforementioned display layout conversion device acquires the set of unit components in the layout scheme to be converted, as well as the corresponding unit component information within that set. Based on this information, it merges unit components within the same area to obtain basic components and their corresponding information. This allows for analysis of the distribution of unit components in the layout scheme. By generating a layout diagram corresponding to the basic components based on this information, the device performs layout segmentation to obtain a multi-branch tree. This tree, combined with the display effect, divides the layout scheme to be converted into a set of basic components. The scaling ratio of the basic components is determined based on the screen size of the layout scheme to be converted and the target layout conversion scheme. Finally, the target layout conversion scheme is output based on the multi-branch tree and the scaling ratio, arranging the basic components in a specific display order to achieve layout conversion. The entire process is applicable to various display layout conversion scenarios because it does not rely on a specific algorithm, demonstrating good generalization. Furthermore, it is highly feasible because it does not require extensive training data for annotation.

[0255] In one embodiment, the page layout segmentation module is further used to perform a first-direction segmentation on the page layout image to obtain a page title segmentation line, a page title area, and a page content area. The page title segmentation line is used to segment the page title area and the page content area. The page content area is segmented, and a page segmentation multi-branch tree is obtained based on the segmentation result.

[0256] In one embodiment, the layout segmentation module is further configured to determine the average value of a single row of pixels in the first direction based on the pixel values ​​of the pixels in the layout image, perform binarization processing on the pixel values ​​of the pixels in the layout image based on the average value of the single row in the first direction, determine the first direction jump point based on the binarization processing result, the first direction jump point is the point where the binarized pixel value changes in the first direction, and obtain the layout title segmentation line, the layout title area and the layout content area based on the first direction jump point.

[0257] In one embodiment, the page layout segmentation module is further configured to perform second-direction segmentation on the page content area based on the pixel values ​​of the pixels in the page content area to obtain second-direction segmentation lines, wherein the second direction is perpendicular to the first direction; segment the page content area according to the second-direction segmentation lines to obtain at least two second-direction areas to be segmented; perform first-direction segmentation on the second-direction areas to be segmented to obtain first-direction segmentation lines corresponding to the second-direction areas to be segmented; segment the corresponding second-direction areas to be segmented according to the first-direction segmentation lines to obtain at least two first-direction areas to be segmented; perform secondary segmentation on the first-direction areas to be segmented until the page areas to be segmented within the first-direction areas to be segmented meet the preset segmentation cutoff conditions; and obtain a page layout segmentation multi-branch tree based on the segmentation results of each segmentation.

[0258] In one embodiment, the page segmentation module is further configured to determine the average value of a single row of pixels in the second direction based on the pixel values ​​of pixels in the page content area, perform binarization processing on the pixel values ​​of pixels in the page content area based on the average value of a single row in the second direction, determine the second direction jump point based on the binarization processing result, the second direction jump point is the point where the binarized pixel value changes in the second direction, determine the second direction segmentation point based on the second direction jump point, and obtain the second direction segmentation line based on the second direction segmentation point.

[0259] In one embodiment, the processing module is further configured to obtain at least two traversal orders of the layout segmentation multi-branch tree, obtain the layout scheme to be evaluated corresponding to the traversal order based on the traversal order and scaling ratio, evaluate the layout scheme to be evaluated, and output the target layout conversion scheme.

[0260] In one embodiment, the processing module is further configured to count the original layout component values ​​and the converted layout component values ​​corresponding to the layout scheme to be evaluated, obtain a first component density based on the original layout component values, obtain a second component density based on the converted layout component values, sort the layout schemes to be evaluated based on the first component density and the second component density, and output the target layout conversion scheme based on the sorting result.

[0261] In one embodiment, the processing module is further configured to determine a first selected direction corresponding to the typesetting scheme to be converted, and to determine a second selected direction corresponding to the typesetting scheme to be evaluated. In the first selected direction, the number of basic components in each segmented page area corresponding to the typesetting scheme to be converted is counted to obtain the original page component value. In the second selected direction, the number of basic components in each page area of ​​the typesetting scheme to be evaluated is counted to obtain the converted page component value.

[0262] In one embodiment, the processing module is further configured to obtain the layout length and layout width of the layout scheme to be converted, and when the layout length is greater than the layout width, the first selected direction is determined to be vertical, and when the layout length is less than the layout width, the first selected direction is determined to be horizontal.

[0263] In one embodiment, the processing module is further configured to calculate the difference between the first component density and the second component density to obtain the relative component density, deduct the initial score corresponding to the layout scheme to be evaluated based on the first component density and the relative component density, and sort the layout scheme to be evaluated based on the deduction result.

[0264] In one embodiment, the merging module is further configured to sort the unit components according to the unit component information to obtain the arrangement order of the unit components, determine the initial basic component according to the arrangement order, merge the unit components in the same area as the initial basic component according to the component bounding box and the unit component set corresponding to the initial basic component to obtain the target component, remove the unit components in the target component from the unit component set and the arrangement order, return to the step of determining the initial basic component according to the arrangement order, until there are no unit components in the latest unit component set, take the target component obtained each time as the basic component, and obtain the basic component information according to the unit component information of the unit components in the target component.

[0265] In one embodiment, the merging module is further configured to calculate the intersection-union ratio of the component bounding box corresponding to the initial basic component and the unit components in the unit component set, update the component bounding box according to the calculated intersection-union ratio and the preset merging conditions, and use the latest component bounding box as the target component.

[0266] Each module in the aforementioned display layout conversion device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0267] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 14As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a display layout conversion method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0268] Those skilled in the art will understand that Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0269] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0270] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0271] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0272] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0273] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0274] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for converting display layout, characterized in that, The method includes: Obtain the set of unit components in the layout scheme to be converted, and the unit component information corresponding to the unit components in the set of unit components; the unit component refers to the interface element displayed in the layout scheme to be converted; the unit component information refers to the information that represents the position of the unit component in the layout scheme to be converted; Based on the unit component information, the unit components in the same area are merged to obtain the basic component and the corresponding basic component information; the basic component information refers to the information that characterizes the position of the basic component in the typesetting scheme to be converted. Based on the basic component information, generate a layout diagram corresponding to the basic component; The layout image is segmented to obtain a multi-branch tree of layout segmentation; wherein each node in the multi-branch tree of layout segmentation represents a different layout area; The scaling ratio of the basic component is determined based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme; The target layout conversion scheme is output based on the multi-branch tree of the page layout and the scaling ratio.

2. The method according to claim 1, characterized in that, The step of segmenting the layout image to obtain a multi-branch tree of layout segmentation includes: The layout image is divided in a first direction to obtain a layout title dividing line, a layout title area, and a layout content area. The layout title dividing line is used to divide the layout title area and the layout content area. The content area of ​​the page is segmented, and a page segmentation multi-branch tree is obtained based on the segmentation results.

3. The method according to claim 2, characterized in that, The step of dividing the page layout in a first direction to obtain the page title dividing line, the page title area, and the page content area includes: Based on the pixel values ​​of the pixels in the layout image, determine the average value of the pixels in the first direction of a single row; Based on the average value of the first direction in the single row, the pixel values ​​of the pixels in the layout image are binarized, and the first direction jump point is determined based on the binarization result. The first direction jump point is the point where the binarized pixel value changes in the first direction. Based on the jump point in the first direction, the page title dividing line, the page title area, and the page content area are obtained.

4. The method according to claim 2, characterized in that, The step of segmenting the content area of ​​the page and obtaining a page segmentation multi-branch tree based on the segmentation result includes: Based on the pixel values ​​of the pixels in the content area of ​​the page, the content area of ​​the page is divided in a second direction to obtain a second direction dividing line, wherein the second direction is perpendicular to the first direction; The layout content area is divided according to the second direction dividing line to obtain at least two second direction areas to be divided; The second-direction region to be segmented is segmented in the first direction to obtain a first-direction segmentation line corresponding to the second-direction region to be segmented. Based on the first direction dividing line, the corresponding second direction region to be divided is divided to obtain at least two first direction regions to be divided. The first direction region to be segmented is further segmented until the page area to be segmented within the first direction region meets the preset segmentation cutoff condition. Based on the results of each segmentation, a multi-way tree for page segmentation is obtained.

5. The method according to claim 4, characterized in that, The step of dividing the content area of ​​the page layout in a second direction based on the pixel values ​​of the pixels in the content area to obtain the second direction dividing line includes: Based on the pixel values ​​of the pixels in the content area of ​​the page, determine the average value of the pixels in the second direction of a single row; Based on the average value of the second direction in the single row, the pixel values ​​of the pixels in the content area of ​​the page are binarized, and the second direction jump point is determined based on the binarization result. The second direction jump point is the point where the binarized pixel value changes in the second direction. Determine the second direction split point based on the second direction jump point; The second direction dividing line is obtained based on the second direction dividing point.

6. The method according to claim 1, characterized in that, The step of outputting the target layout conversion scheme based on the page layout multi-branch tree and the scaling ratio includes: Obtain at least two traversal orders for the page partitioning multi-way tree; Based on the traversal order and the scaling ratio, a layout scheme to be evaluated corresponding to the traversal order is obtained; The proposed layout scheme is evaluated, and the target layout conversion scheme is output.

7. The method according to claim 6, characterized in that, The process of evaluating the layout scheme to be evaluated and outputting the target layout conversion scheme includes: Calculate the original layout component values ​​and the converted layout component values ​​corresponding to the layout scheme to be evaluated; The first component density is obtained based on the original layout component values, and the second component density is obtained based on the converted layout component values; The layout schemes to be evaluated are sorted according to the density of the first component and the density of the second component; Output the target layout conversion scheme based on the sorting results.

8. The method according to claim 7, characterized in that, The statistical analysis of the original layout component values ​​and the converted layout component values ​​corresponding to the layout scheme to be evaluated includes: Determine a first selected direction corresponding to the typesetting scheme to be converted, and determine a second selected direction corresponding to the typesetting scheme to be evaluated; In the first selected direction, the number of basic components in each segmented page area corresponding to the layout scheme to be converted is counted to obtain the original page component value; In the second selected direction, the number of basic components in each page area of ​​the layout scheme to be evaluated is counted to obtain the converted page component value.

9. The method according to claim 8, characterized in that, The determination of the first selected direction corresponding to the typesetting scheme to be converted includes: Obtain the layout length and layout width of the layout scheme to be converted; When the layout length is greater than the layout width, the first selected direction is determined to be vertical; When the layout length is less than the layout width, the first selected direction is determined to be horizontal.

10. The method according to claim 7, characterized in that, The step of sorting the layout schemes to be evaluated based on the density of the first component and the density of the second component includes: Calculate the difference between the density of the first component and the density of the second component to obtain the relative component density; The initial score corresponding to the layout scheme to be evaluated is deducted based on the first component density and the relative component density. The layout schemes to be evaluated are sorted according to the deduction results.

11. The method according to claim 1, characterized in that, The step of merging the unit components within the same region based on the unit component information to obtain the basic component and its corresponding information includes: Based on the unit component information, the unit components are sorted to obtain the arrangement order of the unit components; The initial basic components are determined according to the arrangement order; Based on the component bounding box corresponding to the initial basic component and the set of unit components, the unit components in the same region as the initial basic component are merged to obtain the target component; Remove the unit components from the target component set and the arrangement order, and return to the step of determining the initial basic component according to the arrangement order, until no unit components are found in the latest unit component set; Each target component obtained is used as a base component, and the base component information is obtained based on the unit component information of the unit components in the target component.

12. The method according to claim 11, characterized in that, The step of grouping unit components within the same region as the initial basic component and the set of unit components according to the component bounding box corresponding to the initial basic component to obtain the target component includes: Calculate the component bounding box corresponding to the initial basic component and the intersection-union ratio of the unit components in the unit component set; Based on the calculated crossover / union ratio and the preset merging conditions, the component bounding box is updated, and the latest component bounding box is used as the target component.

13. A display layout conversion device, characterized in that, The device includes: The acquisition module is used to acquire the set of unit components in the layout scheme to be converted and the unit component information corresponding to the unit components in the set of unit components; the unit component refers to the interface element displayed in the layout scheme to be converted; the unit component information refers to the information that represents the position of the unit component in the layout scheme to be converted; The merging module is used to merge the unit components in the same area according to the unit component information to obtain the basic component and the corresponding basic component information; the basic component information refers to information that characterizes the position of the basic component in the layout scheme to be converted. The layout generation module is used to generate a layout image corresponding to the basic components based on the basic component information. The page layout segmentation module is used to segment the page layout image to obtain a page layout segmentation multi-branch tree; wherein, each node in the page layout segmentation multi-branch tree represents a different page layout area; The scaling module is used to determine the scaling ratio of the basic component based on the screen size corresponding to the layout scheme to be converted and the screen size corresponding to the target layout conversion scheme; the processing module is used to output the target layout conversion scheme based on the page division multi-branch tree and the scaling ratio.

14. The apparatus according to claim 13, characterized in that, The page layout segmentation module is also used to perform a first-direction segmentation on the page layout image to obtain a page title segmentation line, a page title area, and a page content area. The page title segmentation line is used to segment the page title area and the page content area. The page content area is segmented, and a page segmentation multi-branch tree is obtained based on the segmentation result.

15. The apparatus according to claim 14, characterized in that, The layout segmentation module is further configured to determine the average value of a single row of pixels in the first direction based on the pixel values ​​of the pixels in the layout image, perform binarization processing on the pixel values ​​of the pixels in the layout image based on the average value of a single row in the first direction, determine the first direction jump point based on the binarization processing result, the first direction jump point is the point where the binarized pixel value changes in the first direction, and obtain the layout title segmentation line, the layout title area, and the layout content area based on the first direction jump point.

16. The apparatus according to claim 14, characterized in that, The layout segmentation module is further configured to perform second-direction segmentation on the layout content area based on the pixel values ​​of the pixels in the layout content area to obtain second-direction segmentation lines, wherein the second direction is perpendicular to the first direction; segment the layout content area according to the second-direction segmentation lines to obtain at least two second-direction areas to be segmented; perform first-direction segmentation on the second-direction areas to be segmented to obtain first-direction segmentation lines corresponding to the second-direction areas to be segmented; segment the corresponding second-direction areas to be segmented according to the first-direction segmentation lines to obtain at least two first-direction areas to be segmented; perform secondary segmentation on the first-direction areas to be segmented until the layout areas to be segmented within the first-direction areas to be segmented meet the preset segmentation cutoff conditions; and obtain a layout segmentation multi-branch tree based on the results of each segmentation.

17. The apparatus according to claim 16, characterized in that, The layout segmentation module is further configured to determine the average value of a single row of pixels in the second direction based on the pixel values ​​of pixels in the layout content area, perform binarization processing on the pixel values ​​of pixels in the layout content area based on the average value of a single row in the second direction, determine the second direction jump point based on the binarization processing result, the second direction jump point is the point where the binarized pixel value changes in the second direction, determine the second direction segmentation point based on the second direction jump point, and obtain the second direction segmentation line based on the second direction segmentation point.

18. The apparatus according to claim 13, characterized in that, The processing module is further configured to obtain at least two traversal orders of the layout segmentation multi-branch tree, obtain a layout scheme to be evaluated corresponding to the traversal order based on the traversal order and the scaling ratio, evaluate the layout scheme to be evaluated, and output the target layout conversion scheme.

19. The apparatus according to claim 18, characterized in that, The processing module is also used to count the original layout component values ​​and the converted layout component values ​​corresponding to the layout scheme to be evaluated, obtain a first component density based on the original layout component values, obtain a second component density based on the converted layout component values, sort the layout scheme to be evaluated based on the first component density and the second component density, and output the target layout conversion scheme based on the sorting result.

20. The apparatus according to claim 19, characterized in that, The processing module is further configured to determine a first selected direction corresponding to the layout scheme to be converted, and a second selected direction corresponding to the layout scheme to be evaluated. In the first selected direction, the number of basic components in each segmented page area corresponding to the layout scheme to be converted is counted to obtain the original page component value. In the second selected direction, the number of basic components in each page area of ​​the layout scheme to be evaluated is counted to obtain the converted page component value.

21. The apparatus according to claim 20, characterized in that, The processing module is also used to obtain the layout length and layout width of the layout scheme to be converted. When the layout length is greater than the layout width, the first selected direction is determined to be vertical. When the layout length is less than the layout width, the first selected direction is determined to be horizontal.

22. The apparatus according to claim 19, characterized in that, The processing module is also used to calculate the difference between the first component density and the second component density to obtain the relative component density, deduct the initial score corresponding to the layout scheme to be evaluated based on the first component density and the relative component density, and sort the layout schemes to be evaluated based on the deduction result.

23. The apparatus according to claim 13, characterized in that, The merging module is further configured to perform component sorting processing on the unit components according to the unit component information to obtain the arrangement order corresponding to the unit components, determine the initial basic component according to the arrangement order, merge the unit components in the same area as the initial basic component according to the component bounding box corresponding to the initial basic component and the unit component set to obtain the target component, remove the unit components in the target component from the unit component set and the arrangement order, return to the step of determining the initial basic component according to the arrangement order, until there are no unit components in the latest unit component set, take the target component obtained each time as the basic component, and obtain the basic component information according to the unit component information of the unit components in the target component.

24. The apparatus according to claim 23, characterized in that, The merging module is also used to calculate the intersection-union ratio of the component bounding box corresponding to the initial basic component and the unit components in the unit component set, update the component bounding box according to the calculated intersection-union ratio and the preset merging conditions, and use the latest component bounding box as the target component.

25. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

26. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

Images