Page layout identification method and device, electronic equipment and storage medium

Abstract

The invention discloses a page layout identification method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a view structure information set of a target page set; view structure information in the view structure information set is in one-to-one correspondence with pages in a target page set, and the target page set comprises pages in a target application program; counting each page element in the target page set based on the view structure information set to obtain the current occurrence frequency of each page element, the current occurrence frequency representing the occurrence frequency of the same page element in different pages in the target page set; the current occurrence frequency is compared with a preset occurrence frequency threshold value, so that the to-be-adjusted target page element with the left-right rudder layout attribute in each page is determined, and automatic identification of the left-right rudder layout of the page is achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a method, apparatus, electronic device, and storage medium for page layout recognition. Background Art

[0002] With the popularization of in-vehicle systems, the adaptation requirements of mobile applications in the in-vehicle environment are increasing day by day. Vehicles in different countries may adopt left-hand drive or right-hand drive configurations. In the prior art, it is impossible to automatically recognize the left-hand drive and right-hand drive layouts of Android system pages. Summary of the Invention

[0003] To solve the problems of the prior art, embodiments of this application provide a method, apparatus, electronic device, and storage medium for page layout recognition. The technical solutions are as follows:

[0004] On the one hand, a method for page layout recognition is provided. The method includes:

[0005] Obtain a set of view structure information of a target page set; the view structure information in the set of view structure information corresponds one-to-one with the pages in the target page set, and the target page set includes pages in a target application;

[0006] Based on the set of view structure information, count each page element in the target page set to obtain the current occurrence frequency of each page element; the current occurrence frequency represents the number of times the same page element appears in different pages in the target page set;

[0007] Based on a comparison between the current occurrence frequency and a preset occurrence frequency threshold, determine the target page elements to be adjusted in each page; the target page elements to be adjusted represent the page elements in each page that are to be adjusted to have left-hand drive and right-hand drive layout attributes.

[0008] In an exemplary embodiment, the obtaining the set of view structure information of the target page set includes:

[0009] Use an automated test framework to click on the target application;

[0010] Based on the click process, obtain and save the view structure information of each page in the target page set to obtain the set of view structure information.

[0011] In an exemplary embodiment, the view structure information includes the layout information of each page element in the page;

[0012] After determining the target page elements to be adjusted in each page, the method further includes:

[0013] For each page in the target page set, search the page layout code file based on the layout information of the target page element to be adjusted in the page, and locate the target layout code of the target page element to be adjusted; the page layout code file represents a code file for defining the layout of the page.

[0014] Modify the target layout code based on the left - right rudder layout attribute to obtain the adjusted target page element in the page.

[0015] In an exemplary embodiment, the method further includes:

[0016] Compare the current occurrence frequency with the preset occurrence frequency threshold to determine the non - target page elements to be adjusted in each page; the non - target page elements to be adjusted represent the page elements in each page to be adjusted to have a non - left - right rudder layout attribute.

[0017] After obtaining the adjusted target page element in the page, the method further includes:

[0018] For each page in the target page set, search the page layout code file based on the view structure information to obtain the current width of the menu bar, where the menu bar represents the area in the page where the adjusted target page element is set.

[0019] For each page in the target page set, search the page layout code file based on the layout information of the non - target page element to be adjusted in the page, and locate the non - target layout code of the non - target page element to be adjusted.

[0020] Modify the non - target layout code based on the current width, the non - left - right rudder layout attribute, the page layout direction of the page, and the left - right rudder layout of the page to obtain the adjusted non - target page element in the page, and the position of the adjusted non - target page element in the page does not overlap with the position of the menu bar in the page.

[0021] In an exemplary embodiment, the method further includes:

[0022] Identify the icons in each page of the target page set to determine the target icons.

[0023] Mirror - process the target icons based on the page layout direction.

[0024] In an exemplary embodiment, the identifying the icons in each page of the target page set to determine the target icons includes:

[0025] Read the image in the icon and grayscale the image;

[0026] Perform contour detection on the grayscaled image;

[0027] Perform convex defect analysis on the result of the contour detection;

[0028] Based on the convex defect analysis, determine the target icon; the target icon includes an arrow.

[0029] On the other hand, a page layout recognition device is provided, and the device includes:

[0030] A view structure information acquisition module, configured to acquire a view structure information set of a target page set; the view structure information in the view structure information set corresponds one-to-one with the pages in the target page set, and the target page set includes pages in a target application;

[0031] A current occurrence frequency statistics module, configured to perform statistics on each page element in the target page set based on the view structure information set to obtain the current occurrence frequency of each page element; the current occurrence frequency represents the number of times the same page element appears in different pages of the target page set;

[0032] A target page element to be adjusted determination module, configured to determine the target page element to be adjusted in each page based on a comparison between the current occurrence frequency and a preset occurrence frequency threshold; the target page element to be adjusted represents the page element in each page to be adjusted to have a left-right rudder layout attribute.

[0033] In an exemplary embodiment, the view structure information acquisition module includes:

[0034] An automated test framework, configured to click on the target application;

[0035] A first acquisition and saving module, configured to acquire and save the view structure information of each page in the target page set based on the click process to obtain the view structure information set.

[0036] In an exemplary embodiment, the view structure information includes the layout information of each page element in the page;

[0037] After determining the target page element to be adjusted in each page, the device further includes:

[0038] A target layout code module for each page in the target page set to search the page layout code file based on the layout information of the target page element to be adjusted in the page, and locate the target layout code of the target page element to be adjusted; the page layout code file represents a layout code file for defining the layout of the page.

[0039] A modified target layout code module for modifying the target layout code based on the left - right rudder layout attribute to obtain the adjusted target page element in the page.

[0040] In an exemplary embodiment, the device further includes:

[0041] A module for determining non - target page elements to be adjusted, which is used to compare the current occurrence frequency with the preset occurrence frequency threshold to determine the non - target page elements to be adjusted in each page; the non - target page elements to be adjusted represent the page elements in each page to be adjusted to have a non - left - right rudder layout attribute.

[0042] After obtaining the adjusted target page element in the page, the device further includes:

[0043] A module for finding the current width of the menu bar, which is used to search the page layout code file for each page in the target page set based on the view structure information to obtain the current width of the menu bar, and the menu bar represents the area in the page where the adjusted target page element is set.

[0044] A non - target layout code module for each page in the target page set to search the page layout code file based on the layout information of the non - target page element to be adjusted in the page, and locate the non - target layout code of the non - target page element to be adjusted.

[0045] A modified non - target layout code module for modifying the non - target layout code based on the current width, the non - left - right rudder layout attribute, the page layout direction of the page, and the left - right rudder layout of the page to obtain the adjusted non - target page element in the page, and the position of the adjusted non - target page element in the page does not overlap with the position of the menu bar in the page.

[0046] In an exemplary embodiment, the device further includes:

[0047] A target icon module for identifying the icons in each page of the target page set to determine the target icon.

[0048] A mirror processing module, configured to perform mirror processing on the target icon based on the page layout direction.

[0049] In an exemplary embodiment, the target icon determination module includes:

[0050] An image processing module, configured to read the image in the icon and perform grayscale processing on the image;

[0051] A contour detection module, configured to perform contour detection on the grayscale processed image;

[0052] A convex defect analysis module, configured to perform convex defect analysis on the result of the contour detection;

[0053] An analysis and determination module, configured to determine the target icon based on the convex defect analysis; the target icon includes an arrow.

[0054] On the other hand, an electronic device is provided, including a processor and a memory. At least one instruction or at least one program segment is stored in the memory, and the at least one instruction or the at least one program segment is loaded and executed by the processor to implement the page layout recognition method in any of the above aspects.

[0055] On the other hand, a computer-readable storage medium is provided. At least one instruction or at least one program segment is stored in the computer-readable storage medium, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the page layout recognition method in any of the above aspects.

[0056] On the other hand, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the page layout recognition method in any of the above aspects.

[0057] In the embodiments of the present application, by obtaining a view structure information set of a target page set; the view structure information in the view structure information set corresponds one-to-one with the pages in the target page set, and the target page set includes pages in a target application; based on the view structure information set, each page element in the target page set is counted to obtain the current occurrence frequency of each page element, and the current occurrence frequency represents the number of times the same page element appears in different pages of the target page set; based on the comparison between the current occurrence frequency and a preset occurrence frequency threshold, the target page elements to be adjusted to have the left-right rudder layout attribute in each page are determined, realizing the automatic recognition of the left-right rudder layout of the page. Description of the Drawings

[0058] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0059] Figure 1 It is a schematic flowchart of a page layout recognition method provided by an embodiment of the present application;

[0060] Figure 2 It is a schematic diagram of different layouts of the same page provided by an embodiment of the present application;

[0061] Figure 3 It is a structural block diagram of a page layout recognition device provided by an embodiment of the present application;

[0062] Figure 4 It is a hardware structural block diagram of an electronic device provided by an embodiment of the present application. Specific embodiments

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0064] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above accompanying drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0065] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other relevant parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of that module or unit.

[0066] It can be understood that in the specific implementation of the present application, when it comes to data related to user information, etc., when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0067] The embodiments of the present application relate to a page layout recognition technology for application program pages in the Android system in the field of computer technology. With the development of globalization, mobile application programs need to adapt to the needs of multiple languages and regions. Generally, the interface layout of application programs has two directions: from left to right (Left to Right, abbreviated as LTR) and from right to left (Right to Left, abbreviated as RTL). The LTR layout is suitable for most Western languages, while the RTL layout is mainly used for Middle Eastern languages such as Arabic and Hebrew. In addition, with the popularization of in-vehicle systems, the adaptation requirements of mobile application programs in the in-vehicle environment are increasing. Vehicles in different countries may adopt left-hand drive or right-hand drive configurations, and the interface layout of the application program needs to dynamically adjust both the layout direction and the left and right steering wheels according to the vehicle configuration to improve the driver's operation convenience and user experience. The prior art cannot automatically adjust the left and right steering wheel layouts.

[0068] The embodiments of the present application address the problem in the prior art that the left and right steering wheel layouts cannot be automatically recognized, and provide a page layout recognition method. This method counts each page element in the target page set based on the obtained view structure information set, and then compares the current occurrence frequency of each page element obtained with a preset occurrence frequency threshold, so as to determine the target page elements to be adjusted to have the left and right steering wheel layout attributes in each page, realizing the automatic recognition of the left and right steering wheel layouts of the page.

[0069] The following introduces a specific embodiment of a page layout recognition method provided by the present application. Please refer to Figure 1, which shows a schematic flowchart of a page layout recognition method provided by an embodiment of the present application. It should be noted that this specification provides method operation steps as described in the embodiments or flowcharts, but based on routine or non-creative labor, there may be more or fewer operation steps. The step order listed in the embodiments is only one way among the execution orders of numerous steps and does not represent the only execution order. When the actual system or product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or executed in parallel (for example, in an environment of parallel processors or multi-threaded processing). Specifically, the method may include the following steps:

[0070] Step S1: Obtain a set of view structure information of the target page set; the view structure information in the set of view structure information corresponds one-to-one with the pages in the target page set, and the target page set includes the pages in the target application.

[0071] Specifically, by obtaining the view structure information of all pages in the target program, and the view structure information in this set of view structure information cannot appear repeatedly, to improve the accuracy of subsequent statistics on the current occurrence frequency of each page element. It should be noted that this view structure information can be obtained through the automatic click script of the Android application interface automation testing tool UIAutomator. The view structure information includes the view hierarchy structure of the page, the types, identity identifiers, display texts, boundary coordinates, touch attributes, and the applications to which they belong, etc. information of each page element in the page.

[0072] In an exemplary embodiment, obtaining the set of view structure information of the target page set includes:

[0073] Use the automatic test framework to click on the target application;

[0074] Based on the click process, obtain and save the view structure information of each page in the target page set to obtain a set of view structure information.

[0075] Specifically, create a folder for saving view structure information. By using the automatic test framework to click on the target application, the current page is obtained after the click. Obtain the view structure information of the current page, compare the view structure information with the folder for saving view structure information for similarity. When the result of the similarity comparison is greater than or equal to the preset similarity threshold, it is considered that the view structure information has been saved, and the view structure information is not saved repeatedly. After traversing each page in the target application based on the click process, a set of view structure information is obtained.

[0076] Step S2: Statistically analyze each page element in the target page set based on the view structure information set to obtain the current occurrence frequency of each page element; the current occurrence frequency represents the number of times the same page element appears in different pages in the target page set.

[0077] Specifically, the same page element in the target page set may appear in multiple pages. The current occurrence frequency actually refers to the number of times each page element appears in multiple different pages in the target page set. Exemplarily, a page element appears in two pages in the target page set, that is, the current occurrence frequency of this page element is 2.

[0078] Step S3: Compare based on the current occurrence frequency and a preset occurrence frequency threshold to determine the target page elements to be adjusted in each page; the target page elements to be adjusted represent the page elements in each page to be adjusted to have the left and right rudder layout attributes.

[0079] Specifically, in the embodiment of the present application, a suitable preset occurrence frequency can be set according to actual needs, and the page elements with the current occurrence frequency greater than or equal to the preset occurrence frequency threshold are determined as the target page elements to be adjusted; Exemplarily, the preset occurrence frequency is set to 4, and the page elements with the current occurrence frequency greater than or equal to 4 are determined as the target page elements to be adjusted. The left and right rudder layout attributes refer to the need to be set in the left or right rudder area of the page, where the right rudder refers to the position on the right side of the page relative to the user, and the left rudder refers to the position on the left side of the page relative to the user.

[0080] In an exemplary implementation, the view structure information includes the layout information of each page element in the page;

[0081] After determining the target page elements to be adjusted in each page, the method further includes:

[0082] For each page in the target page set, search the page layout code file based on the layout information of the target page elements to be adjusted in the page, and locate the target layout code of the target page elements to be adjusted; the page layout code file represents the code file used to define the layout of the page;

[0083] Modify the target layout code based on the left and right rudder layout attributes to obtain the adjusted target page elements in the page.

[0084] Specifically, by finding the same information as in the layout information in the page layout code file, such as the identity identifier corresponding to the page element or the text information in the page element, etc., the target layout code of the target page element to be adjusted is located; according to the left or right rudder layout setting of the target page element to be adjusted in the page, its target layout code is modified. Exemplarily, if the left and right rudder layout attribute of the target page element to be adjusted in the page is the left rudder of the page, that is, its target layout code is modified, so that the adjusted target page element is located at the left rudder of the page and is not affected by the page layout direction of the page. The page layout direction refers to the page layout direction of the page in the system, which is divided into two types: LTR and RTL.

[0085] In an exemplary embodiment, the method further includes:

[0086] Based on the comparison between the current occurrence frequency and the preset occurrence frequency threshold, the non-target page elements to be adjusted in each page are determined; the non-target page elements to be adjusted represent the page elements to be adjusted in each page to have non-left and right rudder layout attributes;

[0087] Specifically, the page elements with the current occurrence frequency less than the preset occurrence frequency threshold are determined as the non-target page elements to be adjusted.

[0088] After obtaining the adjusted target page elements in the page, the method further includes:

[0089] For each page in the target page set, based on the view structure information, the page layout code file is searched to obtain the current width of the menu bar. The menu bar represents the area where the adjusted target page elements are set in the page;

[0090] For each page in the target page set, based on the layout information of the non-target page elements to be adjusted in the page, the page layout code file is searched to locate the non-target layout code of the non-target page elements to be adjusted;

[0091] Based on the current width, non-left and right rudder layout attributes, page layout direction of the page, and left and right rudder layout of the page, the non-target layout code is modified to obtain the adjusted non-target page elements in the page, and the position of the adjusted non-target page elements in the page does not overlap with the position of the menu bar in the page.

[0092] Specifically, the menu bar is the area for setting the target page elements after adjustment in the page; the page layout displayed in the mobile application usually uses a tree - like structure layout, and the view structure information of the page obtained through UIAutomator can be regarded as a state snapshot of each page element including the page tree structure. Since the view structure of the page is a tree - like structure, this menu bar is the parent layout of one of the target page elements after adjustment. By the layout information of this target page element after adjustment included in the view structure information, the layout code information of this target page element after adjustment in the page layout code file can be found, and thus the width setting of this menu bar, that is, the current width of the menu bar, can be found.

[0093] The non - target page element to be adjusted has a non - left - right rudder layout property, that is, there is no left - right rudder setting. In its non - target layout code, set the non - target page element to be adjusted to automatically set the layout direction according to the page layout direction of the page. Among them, the page layout direction of the page is consistent with the current system direction, and this current system direction can be determined according to the system language. Exemplarily, when the current system direction is LTR, that is, the page layout direction is LTR, if the page is set with a right rudder, the menu bar is on the right rudder of the page. By adjusting the non - target layout code of the non - target page element to be adjusted to automatically set the layout direction according to the page layout direction, and ensuring that the starting boundary distance of the non - target page element to be adjusted in the non - target layout code satisfies being greater than or equal to the width of the menu bar, where the starting boundary distance is defaulted to be a fixed right - hand boundary distance; if the current system direction is RTL, that is, the page layout direction is RTL, and when the page is set with a left rudder, the menu bar is on the left rudder of the page. By adjusting the non - target code layout of the non - target page element to be adjusted to automatically set the layout direction according to the page layout direction, that is, the layout direction in the non - target code layout of the non - target page element to be adjusted is automatically switched to RTL, and modifying the starting boundary distance of the non - target page element to be adjusted in the non - target layout code to a fixed left - hand boundary distance and the left - hand boundary distance satisfies being greater than or equal to the width of the menu bar, so as to avoid the position overlap between the non - target page element to be adjusted and the menu bar in the page.

[0094] In an exemplary embodiment, the method further includes:

[0095] Identify the icons in each page of the target page set to determine the target icons;

[0096] Mirror - process the target icons based on the page layout direction.

[0097] Specifically, by directly recognizing the page, the target icon is determined. The target icon is a direction icon with a left-to-right or right-to-left direction. In the embodiments of the present application, it can be set to automatically locate the relevant layout code of the target icon according to the image resource name of the target icon, modify the layout direction of the target icon, and add a mirror attribute. Exemplarily, when the initial default direction of the target icon is left-to-right and the page layout direction is left-to-right, the target icon remains in the left-to-right direction. If the initial default direction of the target icon is left-to-right and the page layout direction is right-to-left, then the target icon is mirrored left and right to obtain a target icon with a right-to-left direction. In specific implementation, since the menu bar is affected by the left and right steering settings, when mirroring the icons in the menu bar, it can be performed according to the layout direction of the parent layout code where the adjusted target page element is located in the menu bar, and is not affected by the page layout direction of the page. When the menu bar is set to right steering, the page layout direction is LTR, and the arrow direction of the icons outside the menu bar area on the page is the default left-to-right direction, and there is no need to mirror the icons outside the menu bar area left and right. Since the menu bar is on the right steering, the layout direction of the parent layout code where the adjusted target page element is located in the menu bar is RTL, so the icons in the menu bar are mirrored left and right to obtain the arrow direction of the icons in the menu bar as right-to-left.

[0098] In an exemplary embodiment, recognizing the icons in each page of the target page set to determine the target icon includes:

[0099] Read the image in the icon and grayscale the image.

[0100] Perform contour detection on the grayscaled image.

[0101] Perform convexity defect analysis on the result of the contour detection.

[0102] Based on the convexity defect analysis, determine the target icon; the target icon includes an arrow.

[0103] Specifically, first, preprocess the image in the icon, that is, read the icon image, convert it to grayscale and binarize it; contour detection is to find the contours in the image; perform convexity defect analysis on the contour: the arrow shape usually produces two obvious convexity defects, that is, two concave parts, at the connection between the arrow head and the arrow shaft. The contour with these two concave parts is determined as an arrow, and the icon including the arrow is the target icon.

[0104] The following introduces a specific embodiment of a page layout recognition method provided by this application. First, in the Android system, the layout code framework for defining the layout, style, menu, etc. of an application provides native RTL support, and RTL layout is supported by enabling the RTL attribute. The two-way layout support for LTR and RTL requires enabling RTL support and modifying the layout direction-related attributes in the layout code. According to the user's location or system language settings, the page layout direction to be used currently is automatically detected. If the system language is an RTL language such as Arabic or Hebrew, the system automatically switches to RTL layout. For non-RTL languages (such as English, Chinese, etc.), the LTR layout is maintained. In the layout code, in order to support both LTR and RTL layouts simultaneously, the input layout code is normalized to ensure that the page layout direction can be automatically switched according to the current language or region settings, and the attributes related to the "Left / Right" direction are modified to achieve the support for switching the page layout direction of the overall page in the system.First, create a folder to save the view structure information. Use the automated test framework to perform click operations on the target application, obtain the view structure information of the current page obtained from each click, and compare the similarity between this view structure information and all the view structure information in the folder that saves the view structure information. Set the preset similarity threshold to 90%. When the similarity comparison result is greater than or equal to 90%, it is determined that the view structure information of the current page has been saved and no repeated saving will be performed. Traverse each page of the target application based on this click process to obtain a set of view structure information. This page set consists of 7 pages. Therefore, the set of view structure information obtained contains 7 view structure information, and the 7 view structure information corresponds one-to-one with the 7 pages. Taking one page as an example: This page contains page elements numbered 1, 2, 3, and 4. Based on the set of view structure information, count the number of times each page element appears on different pages. The result shows that the page element numbered 4 appears in 7 different pages, and the page elements numbered 1, 2, and 3 appear in 2 different pages. That is, the current appearance frequency of the page element numbered 4 is 7, and the current appearance frequencies of the page elements numbered 1, 2, and 3 are all 2. The preset appearance frequency is 5. Therefore, determine that the page element numbered 4 is the target page element to be adjusted, and the page elements numbered 1, 2, and 3 are the non-target page elements to be adjusted. Based on the layout information of the page element numbered 4 included in the view structure information corresponding to the page, search the page layout code file to locate the target layout code of the page element numbered 4. Modify the left and right rudder layout attributes of the page element numbered 4. When the current page is set to the right rudder, that is, after adjustment, the page element numbered 4 is in the right rudder menu bar of this page. The way to set the right rudder menu bar is to set the layout direction of the parent layout code where the page element numbered 4 is located to RTL, so that the menu bar is always on the right rudder and is not affected by the page layout direction of the page. Adjust the layout codes of the page elements numbered 1, 2, and 3 so that the layout direction automatically switches according to the page layout direction of the page, and the page layout direction of the page is automatically adjusted according to the system language. That is, when the system language is Arabic, the default system direction is RTL, that is, the page layout direction of this page is RTL, and the layout directions of the page elements numbered 1, 2, and 3 automatically switch to RTL. On this basis, it is also necessary to find the current width of the menu bar on this page. The menu bar on this page is the parent layout of the page element numbered 4. Regarding the current width of the menu bar, it can be found in the target layout code of the page element numbered 4. Then, combined with the setting of the starting boundary distance of the page elements numbered 1, 2, and 3 in their respective non-target layout codes, and ensure that the starting boundary distance is greater than or equal to the current width of the menu bar to ensure that the positions of the page elements numbered 1, 2, and 3 on this page do not overlap with the menu bar after adjustment. Among them, the starting boundary distance is defaulted to a fixed right boundary distance, such as. Figure 2Right-to-left and right rudder settings; when maintaining the right rudder setting and changing the system language to English, the position of the menu bar remains unchanged. The page element No. 4 is still in the right rudder menu bar, that is, the layout direction of the parent layout code where the page element No. 4 is located is still set to RTL and is not affected by the system language. The layout directions of the page elements No. 1, 2, and 3 are automatically switched to LTR. Find the current width of the menu bar on this page. The menu bar on this page is the parent layout of the page element No. 4. Regarding the current width of the menu bar, it can be found in the target layout code of the page element No. 4. Then, combined with the starting boundary distance settings of the page elements No. 1, 2, and 3 in their respective non-target layout codes, the starting boundary distance should be greater than or equal to the current width of the menu bar to ensure that the positions of the page elements No. 1, 2, and 3 on this page do not overlap with the menu bar after adjustment, as Figure 2 Left-to-right and right rudder settings; when the system language remains English and is modified to the left rudder setting, the left and right rudder attributes of the page element No. 4 are modified to the left rudder setting, that is, the layout direction of the parent layout code where the page element No. 4 is located is adjusted to LTR, so that the position of the menu bar is on the left rudder of the page. Regarding the starting boundary distance settings of the page elements No. 1, 2, and 3, the starting boundary distance setting is modified to a fixed left boundary distance, and the left boundary distance should be greater than or equal to the current width of the menu bar to ensure that the positions of the page elements No. 1, 2, and 3 on this page do not overlap with the menu bar after adjustment, obtaining Figure 2 Left-to-right and left rudder; when the left rudder setting remains unchanged and the system language is switched back to Arabic again, the position of the menu bar remains unchanged. The page element No. 4 is still in the menu bar. The layout directions of the page elements No. 1, 2, and 3 are automatically switched to RTL. Find the current width of the menu bar on this page. The menu bar on this page is the parent layout of the page element No. 4. Regarding the current width of the menu bar, it can be found in the target layout code of the page element No. 4. Then, combined with the left boundary distances of the page elements No. 1, 2, and 3 in their respective non-target layout codes, check to ensure that the left boundary distance is greater than or equal to the current width of the menu bar to ensure that the positions of the page elements No. 1, 2, and 3 on this page do not overlap with the position of the menu bar on this page after adjustment, as Figure 2Right-to-left and left rudder settings. When performing left and right rudder settings and page layout direction adjustments, it also includes identifying the icons on the page and mirroring the icons with arrows based on the page layout direction. When the page layout direction is LTR, the icons are in the default direction. When the page layout direction is RTL, the icons are mirrored left and right. Here, it should be noted that since the menu bar is affected by the left and right rudder settings, when mirroring the icons in the menu bar, it is necessary to follow the layout direction of the parent layout code where the adjusted target page element is located in the menu bar, which has nothing to do with the page layout direction, that is, it is not affected by the system language. When the system language is English and the menu bar is set to the right rudder, the page layout direction is LTR. Therefore, the arrow direction of the icons outside the menu bar area on the page is in the default direction from left to right, and there is no need to mirror the icons outside the menu bar area left and right. However, the menu bar is on the right rudder, that is, the layout direction of the parent layout code where the adjusted target page element is located in the menu bar is RTL. Therefore, after mirroring the icons in the menu bar left and right, we get as Figure 2 Shown in left-to-right and right rudder.

[0105] As can be seen from the above technical solutions of the embodiments of the present application, the embodiments of the present application obtain a set of view structure information of the target page set; the view structure information in the set of view structure information corresponds one by one to the pages in the target page set, and the target page set includes the pages in the target application; based on the set of view structure information, each page element in the target page set is counted to obtain the current occurrence frequency of each page element, and the current occurrence frequency represents the number of times the same page element appears in different pages in the target page set; based on the comparison between the current occurrence frequency and the preset occurrence frequency threshold, the target page elements to be adjusted to have the left and right rudder layout attributes in each page are determined, thereby realizing the automatic recognition of the left and right rudder layout of the page.

[0106] The embodiments of the present application also provide a page layout recognition device, Figure 3 which is a structural block diagram of a page layout recognition device, as Figure 3 shown. The page layout recognition device 300 includes:

[0107] According to the second aspect of the embodiments of the present application, a page layout recognition device 300 is provided. The device includes:

[0108] A view structure information acquisition module 310, configured to acquire a set of view structure information of the target page set; the view structure information in the set of view structure information corresponds one by one to the pages in the target page set, and the target page set includes the pages in the target application;

[0109] The current occurrence frequency statistics module 320 is used to perform statistics on each page element in the target page set based on the view structure information set, and obtain the current occurrence frequency of each page element; the current occurrence frequency represents the number of times the same page element appears in different pages in the target page set.

[0110] The target page element to be adjusted determination module 330 is used to compare the current occurrence frequency with a preset occurrence frequency threshold to determine the target page element to be adjusted in each page; the target page element to be adjusted represents the page element to be adjusted to have the left-right rudder layout attribute in each page.

[0111] In an exemplary embodiment, the view structure information acquisition module 310 includes:

[0112] An automated test framework for clicking on the target application.

[0113] The first acquisition and saving module is used to acquire and save the view structure information of each page in the target page set based on the clicking process, and obtain the view structure information set.

[0114] In an exemplary embodiment, the view structure information includes the layout information of each page element in the page.

[0115] After determining the target page element to be adjusted in each page, the apparatus 300 further includes:

[0116] The target layout code search module is used to search the page layout code file for each page in the target page set based on the layout information of the target page element to be adjusted in the page, and locate the target layout code of the target page element to be adjusted; the page layout code file represents the layout code file used to define the page layout.

[0117] The target layout code modification module is used to modify the target layout code based on the left-right rudder layout attribute to obtain the adjusted target page element in the page.

[0118] In an exemplary embodiment, the apparatus 300 further includes:

[0119] The non-target page element to be adjusted determination module is used to compare the current occurrence frequency with a preset occurrence frequency threshold to determine the non-target page element to be adjusted in each page; the non-target page element to be adjusted represents the page element to be adjusted to have a non-left-right rudder layout attribute in each page.

[0120] After obtaining the adjusted target page element in the page, the apparatus 300 further includes:

[0121] Find the current width module of the menu bar, which is used to search the page layout code file for each page in the target page set based on the view structure information to obtain the current width of the menu bar. The menu bar represents the area of the target page element after adjustment in the page.

[0122] Find the non-target layout code module, which is used to search the page layout code file for each page in the target page set based on the layout information of the non-target page element to be adjusted in the page, and locate the non-target layout code of the non-target page element to be adjusted.

[0123] Modify the non-target layout code module, which is used to modify the non-target layout code based on the current width, non-left and right rudder layout attributes, page layout direction of the page, and left and right rudder layout of the page, to obtain the adjusted non-target page element in the page, and the position of the adjusted non-target page element in the page does not overlap with the position of the menu bar in the page.

[0124] In an exemplary embodiment, the apparatus 300 further includes:

[0125] Determine the target icon module, which is used to identify the icons in each page of the target page set to determine the target icon.

[0126] Mirror processing module, which is used to perform mirror processing on the target icon based on the page layout direction.

[0127] In an exemplary embodiment, the determine target icon module includes:

[0128] Image processing module, which is used to read the image in the icon and perform grayscale processing on the image.

[0129] Contour detection module, which is used to perform contour detection on the grayscale processed image.

[0130] Convex defect analysis module, which is used to perform convex defect analysis on the result of contour detection.

[0131] Analysis and determination module, which is used to determine the target icon based on the convex defect analysis; the target icon includes an arrow.

[0132] It should be noted that for the apparatus provided in the above embodiments, when implementing its functions, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus provided in the above embodiments and the method embodiments belong to the same concept, and the specific implementation process can be seen in the method embodiments, which will not be repeated here.

[0133] An embodiment of the present application further provides an electronic device, including a processor; a memory for storing instructions executable by the processor; wherein, when the processor is configured to execute the instructions stored on the memory, the page layout recognition method provided in any of the above embodiments is implemented.

[0134] The electronic device may be a terminal, a server or a similar computing device. Taking the electronic device being a terminal as an example, Figure 4 is a block diagram of an electronic device running a page layout recognition method shown according to an exemplary embodiment. Specifically:

[0135] The terminal may include an RF (Radio Frequency) circuit 410, a memory 420 including one or more computer-readable storage media, an input unit 430, a display unit 440, a sensor 450, an audio circuit 360, a WiFi (wireless fidelity) module 470, a processor 480 including one or more processing cores, and a power supply 490 and other components. Those skilled in the art can understand that Figure 3 the terminal structure shown in does not constitute a limitation on the terminal, and may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements. Among them:

[0136] The RF circuit 410 can be used for receiving and sending signals during information reception or call processes. Specifically, after receiving the downlink information of the base station, it is handed over to one or more processors 480 for processing; in addition, data related to the uplink is sent to the base station. Usually, the RF circuit 410 includes but is not limited to antennas, at least one amplifier, a tuner, one or more oscillators, a subscriber identity module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. In addition, the RF circuit 410 can also communicate with the network and other terminals through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.

[0137] The memory 420 can be used to store software programs and modules. The processor 480 executes various functional applications and data processing by running the software programs and modules stored in the memory 420. The memory 420 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 420 may further include a memory controller to provide access to the memory 420 by the processor 480 and the input unit 430.

[0138] The input unit 430 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls. Specifically, the input unit 430 may include a touch-sensitive surface 431 and other input devices 432. The touch-sensitive surface 431, also known as a touch display screen or a touchpad, can collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or near the touch-sensitive surface 431), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 431 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch position of the user and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 480, and can receive and execute the commands sent by the processor 480. In addition, the touch-sensitive surface 431 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 431, the input unit 430 may further include other input devices 432. Specifically, the other input devices 432 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0139] The display unit 440 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the terminal. These graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 440 may include a display panel 441. Optionally, the display panel 441 can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 431 can cover the display panel 441. When the touch-sensitive surface 431 detects a touch operation on or near it, it is transmitted to the processor 480 to determine the type of touch event. Subsequently, the processor 480 provides a corresponding visual output on the display panel 441 according to the type of touch event. Among them, the touch-sensitive surface 431 and the display panel 441 can be implemented as two independent components for input and input functions. However, in some embodiments, the touch-sensitive surface 431 and the display panel 441 can also be integrated to implement input and output functions.

[0140] The terminal may further include at least one sensor 450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 441 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 441 and / or the backlight when the terminal is moved to the ear. As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that identify the posture of the terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, tapping), etc. As for other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors that the terminal can also be configured with, they will not be elaborated here.

[0141] The audio circuit 460, the speaker 461, and the microphone 462 can provide an audio interface between the user and the terminal. The audio circuit 460 can transmit the electrical signal converted from the received audio data to the speaker 461, and the speaker 461 converts it into a sound signal for output. On the other hand, the microphone 462 converts the collected sound signal into an electrical signal, which is received by the audio circuit 460 and converted into audio data. After the audio data is output to the processor 480 for processing, it is sent through the RF circuit 410 to, for example, another terminal, or the audio data is output to the memory 420 for further processing. The audio circuit 460 may also include an earphone jack to provide communication between the peripheral earphone and the terminal.

[0142] WiFi belongs to short - range wireless transmission technology. The terminal can help users send and receive emails, browse the web, and access streaming media through the WiFi module 470, providing users with wireless broadband Internet access. Although Figure 4 the WiFi module 470 is shown, it can be understood that it does not belong to the essential components of the terminal and can be completely omitted within the scope of not changing the essence of the invention as needed.

[0143] The processor 480 is the control center of the terminal, connecting various parts of the entire terminal using various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 420, and by calling data stored in the memory 420, it executes various functions of the terminal and processes data. Optionally, the processor 480 may include one or more processing cores; preferably, the processor 480 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interfaces, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above - mentioned modem processor may not be integrated into the processor 480 either.

[0144] The terminal further includes a power source 490 (such as a battery) for powering each component. Preferably, the power source can be logically connected to the processor 480 through a power management system, thereby realizing functions such as management of charging, discharging, and power consumption management through the power management system. The power source 490 may also include any components such as one or more DC or AC power sources, a recharge system, a power - failure detection circuit, a power converter or inverter, and a power - status indicator.

[0145] An embodiment of the present application also provides a computer - readable storage medium. The computer - readable storage medium can be disposed in an electronic device to store at least one instruction or at least one segment of a program related to implementing a page layout recognition method. The at least one instruction or the at least one segment of the program is loaded and executed by the processor to implement any one of the page layout recognition methods provided by the above - mentioned method embodiments.

[0146] An embodiment of the present application also provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer - readable storage medium. The processor of the electronic device reads the computer instructions from the computer - readable storage medium, and the processor executes the computer instructions, causing the electronic device to execute any one of the page layout recognition methods provided by the above - mentioned method embodiments.

[0147] Optionally, in this embodiment, the above storage medium may include, but is not limited to: various media that can store program codes such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), external hard drives, magnetic disks, or optical discs.

[0148] It should be noted that: the above sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of this specification have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0149] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.

[0150] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk, or an optical disc, etc.

[0151] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A page layout recognition method, characterized in that: The method comprises: Acquire a view structure information set of a target page set; the view structure information in the view structure information set corresponds one-to-one to the pages in the target page set, and the target page set includes pages in a target application; Based on the view structure information set, each page element in the target page set is counted to obtain a current occurrence frequency of each page element; the current occurrence frequency represents the number of times the same page element appears in different pages in the target page set; Based on the comparison between the current occurrence frequency and the preset occurrence frequency threshold, the target page element to be adjusted in each page is determined; the target page element to be adjusted represents the page element to be adjusted to have left and right rudder layout attributes in each page.

2. A page layout recognition method according to claim 1, characterized in that: The view structure information set of the target page set is obtained including: Using an automated testing framework, clicking on the target application; Based on the clicking process, the view structure information of each page in the target page set is acquired and saved to obtain the view structure information set.

3. A page layout recognition method according to claim 1, characterized in that: The view structure information includes layout information of each page element in the page; After determining the target page elements to be adjusted in each page, the method further includes: For each page in the target page set, a page layout code file is searched based on the layout information of the target page element to be adjusted in the page, and a target layout code of the target page element to be adjusted is located; the page layout code file represents a layout code file used to define the page; Based on the left and right rudder layout attributes, the target layout code is modified to obtain an adjusted target page element in the page.

4. A page layout recognition method according to claim 1 or 3, characterized in that: The method further comprises: Determine the non-target page elements to be adjusted in each page based on the comparison between the current occurrence frequency and the preset occurrence frequency threshold; the non-target page elements to be adjusted represent the page elements to be adjusted to have non-left-right rudder layout attributes in each page; After obtaining the adjusted target page element in the page, the method further includes: For each page in the target page set, based on the view structure information, the page layout code file is searched to obtain the current width of a menu bar, where the menu bar represents an area in the page where the adjusted target page element is set; For each page in the target page set, searching the page layout code file based on the layout information of the non-target page element to be adjusted in the page, and locating the non-target layout code of the non-target page element to be adjusted; Based on the current width, the non-left and right rudder layout attributes, the page layout direction of the page and the left and right rudder layout of the page, the non-target layout code is modified to obtain an adjusted non-target page element in the page, and the position of the adjusted non-target page element in the page does not overlap with the position of the menu bar in the page.

5. A page layout recognition method according to claim 4, characterized in that: The method further comprises: Identify the icon in each page in the target page set to determine the target icon; Based on the page layout direction, the target icon is mirrored.

6. A page layout recognition method according to claim 5, characterized in that: The identifying the icon in each page in the target page set to determine the target icon comprises: Reading the image in the icon and graying the image; Performing contour detection on the image after the graying process; Performing convex defect analysis on the result of the contour detection; Based on the convex defect analysis, a target icon is determined; the target icon includes an arrow.

7. A page layout recognition device, characterized in that: The device comprises: A view structure information acquisition module, used to acquire a view structure information set of a target page set; the view structure information in the view structure information set corresponds to the pages in the target page set one by one, and the target page set includes pages in a target application; A current occurrence frequency counting module is used to count each page element in the target page set based on the view structure information set to obtain the current occurrence frequency of each page element; the current occurrence frequency represents the number of times the same page element appears in different pages in the target page set; A module for determining target page elements to be adjusted is used to determine the target page elements to be adjusted in each page based on the comparison between the current occurrence frequency and the preset occurrence frequency threshold; the target page elements to be adjusted represent the page elements in each page to be adjusted to have left and right rudder layout attributes.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a page layout recognition method as described in claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a page layout recognition method as described in claims 1 to 6.

10. A computer program, characterized in that When the computer program is executed by a processor, the page layout recognition method described in claims 1 to 6 is implemented.

Images