A page data processing method and an electronic device

Abstract

The application provides a page data processing method and an electronic device. The method can include: determining a node feature of a node when receiving download completion information or version update completion information of an application; determining a candidate node from the node of a to-be-processed page based on a first type of node feature; determining a function node from the candidate node based on a second type of node feature; determining a function node file corresponding to the to-be-processed page based on the function node in the to-be-processed page; the function node file includes part or all data fields of the function node; adjusting the display size and the display position of the function node in a preset display area based on the function node file and a preset layout parameter to determine a page layout corresponding to the to-be-processed page; the first type of node feature includes a node width and a node height of the node; and the second type of node feature includes a structure feature and a node attribute of the node. The application can improve the efficiency and accuracy of function node identification.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a page data processing method and electronic device. Background Technology

[0002] With the continuous development of electronic technology, the types of electronic devices are increasing, and the differences in their displays are becoming more and more apparent. For example, electronic devices can be candybar phones, foldable phones, or tablets, and the display parameters of different electronic devices, such as size or resolution, may differ. Under different display parameters, even for the same page, the content displayed on the screen may be different. For example, the screen size of a foldable phone is different in its folded state and different in its unfolded state. Consequently, the number of application icons that can be displayed on the screen in the folded state is different from the number that can be displayed in the unfolded state.

[0003] To ensure that electronic devices can display page content completely, or to maintain the integrity and consistency of the displayed page content even when the display parameters of the electronic device's screen change, it is usually necessary to manually identify the characteristic information of each application's page. This includes the characteristic information of each display area on the screen, such as the size and position of the display area, and the number, size, and position of the display elements (such as icons and text) included in the display area. Based on this characteristic information, developers can create a page layout scheme and apply it to the electronic device. The electronic device can then control the display screen to display according to the page layout scheme. However, manually identifying the characteristic information of each different page individually increases the workload for developers.

[0004] Therefore, how to quickly and accurately identify the feature information of a page is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides a page data processing method and an electronic device that can improve the efficiency and accuracy of identifying functional nodes in a page.

[0006] In a first aspect, embodiments of this application provide a page data processing method. This method can be executed by an electronic device or by a device compatible with an electronic device, such as a processor, chip, or chip system. The method may include: upon receiving application download completion information or version update completion information, determining node characteristics of nodes in the page to be processed based on the page data of the application's page to be processed; the node characteristics include a first type of node characteristics and a second type of node characteristics; the first type of node characteristics includes the node's width and height; the second type of node characteristics includes the node's structural characteristics and node attributes; determining candidate nodes from the nodes of the page to be processed based on the first type of node characteristics; determining functional nodes from the candidate nodes based on the second type of node characteristics; determining a functional node file corresponding to the page to be processed based on the functional nodes; the functional node file includes some or all of the functional node's data fields; adjusting the display size and display position of the functional nodes in a preset display area based on the functional node file and preset layout parameters to determine the page layout corresponding to the page to be processed; the preset display area is the area on the screen used to display the page to be processed; the preset layout parameters include the width and height of the preset display area.

[0007] As can be seen, this application can automatically identify functional nodes among the nodes of the page to be processed based on the node characteristics, thus improving the accuracy and efficiency of functional node identification. Outputting the data fields of the nodes identified as functional nodes facilitates subsequent adjustments to the page layout using the functional node file.

[0008] In one possible implementation, based on the characteristics of the first type of nodes, candidate nodes are determined from the nodes of the page to be processed, including: determining candidate nodes whose node height is greater than a first threshold and whose node width satisfies a first condition from the nodes of the page to be processed; wherein, candidate node i is any candidate node among the candidate nodes, and the node width of candidate node i satisfies the first condition including one or more of the following: the node width of candidate node i is equal to the screen width; the node width of candidate node i is equal to half of the screen width; the node width of candidate node i is greater than or equal to three-quarters of the screen width.

[0009] As can be seen, based on the node height and node width, candidate nodes that meet the first condition can be determined from the nodes of the page to be processed, which is helpful for accurately identifying functional nodes.

[0010] In one possible implementation, based on the second type of node features, a functional node is determined from the candidate nodes, including: if the structural features of candidate node p indicate that candidate node p has no child nodes, and the node attribute of candidate node p is a text attribute, then candidate node p is determined to be a functional node; candidate node p is any candidate node in the page to be processed.

[0011] As can be seen, when a candidate node has no child nodes, it can be determined whether the candidate node is a functional node based on the node attributes of the candidate node.

[0012] In one possible implementation, based on the second type of node features in the node features, a functional node is determined from the candidate nodes, including: if the structural features of the candidate node p indicate that the candidate node p includes one or more child nodes, the node features of the child nodes of the candidate node p are obtained; the node features of the child nodes of the candidate node p include the node width and the node height of the child nodes; if at least one child node of the candidate node p has a node width that is different from the node width of the candidate node p, and the node width and node height of at least one child node are both greater than a second threshold, the candidate node p is determined to be a functional node; wherein, the candidate node p is any candidate node in the page to be processed.

[0013] As can be seen, when a candidate node includes one or more child nodes, it is possible to determine whether the candidate is a functional node based on features such as the node width of the candidate node, the node width of the child nodes, and the node height.

[0014] In one possible implementation, the non-functional nodes in the page to be processed include the child nodes of the candidate node p. Non-functional nodes are the nodes in the page to be processed that are not functional nodes.

[0015] It is evident that once a node is identified as a functional node, all its child nodes can be determined to be non-functional nodes, thereby improving the efficiency of functional node identification.

[0016] In one possible implementation, the above method may further include: using a preset display effect image of the page to be processed as a background image, marking the display area corresponding to the functional node in the background image to obtain a marker image; the proportion of the marker image is the same as that of the background image; and outputting the marker image on the image display interface according to the proportion of the marker image.

[0017] As can be seen, marking and displaying the identified functional nodes in the background image can intuitively show the recognition results of functional nodes, which is beneficial for adjusting the rules used to identify functional nodes based on the marked image and improving the accuracy of functional node recognition.

[0018] In one possible implementation, within a preset display area, the display area corresponding to the functional node includes the functional node and its N child nodes; N is an integer greater than or equal to 0.

[0019] As can be seen, the display area corresponding to the functional node in the preset display area determined in this application may include the node itself, which is determined to be a functional node, and the N child nodes included in the functional node, thereby facilitating the simultaneous planning of the display effect of the functional node and its child nodes in the preset display area.

[0020] In a second aspect, embodiments of this application provide an electronic device comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in the first aspect or any implementation thereof.

[0021] Thirdly, embodiments of this application provide a chip system including at least one processor, a memory, and an interface circuit. The memory, the interface circuit, and at least one processor are interconnected via circuits. The at least one memory stores program instructions. When the program instructions are executed by the processor, the chip system performs the method described in the first aspect or any implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer program product containing instructions that, when run on an electronic device, cause the electronic device to perform the method described in the first aspect or any implementation thereof.

[0023] Fifthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in the first aspect or any implementation thereof. Attached Figure Description

[0024] Figure 1A This is a schematic diagram of an application interface provided in an embodiment of this application;

[0025] Figure 1B This is a schematic diagram of another application interface provided in an embodiment of this application;

[0026] Figure 1C This is a schematic diagram of another application interface provided in the embodiments of this application;

[0027] Figure 1D This is a schematic diagram of another application interface provided in the embodiments of this application;

[0028] Figure 2 This is a flowchart illustrating a page layout strategy provided in an embodiment of this application;

[0029] Figure 3This is a schematic diagram of the structure of an AI processing system provided in an embodiment of this application;

[0030] Figure 4 This is a flowchart illustrating a page data processing method provided in an embodiment of this application;

[0031] Figure 5A This is a schematic diagram of an input file for a page provided in an embodiment of this application;

[0032] Figure 5B This is a schematic diagram of a view tree structure provided in an embodiment of this application;

[0033] Figure 6 This is a flowchart illustrating a method for determining functional nodes provided in an embodiment of this application;

[0034] Figure 7A This is a schematic diagram of a page layout provided in an embodiment of this application;

[0035] Figure 7B This is a schematic diagram of a view tree structure provided in an embodiment of this application;

[0036] Figure 8 This is a schematic diagram of a functional node file provided in an embodiment of this application;

[0037] Figure 9 This is a schematic diagram of a marker image corresponding to a page to be processed, provided in an embodiment of this application;

[0038] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0039] Figure 11 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0040] The embodiments of this application will now be described with reference to the accompanying drawings.

[0041] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0042] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0043] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0044] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0045] A user interface (UI) generally refers to the interface provided by a display device to a user. The UI can display icons, text, and other display elements. The display device and the user can interact through various means, such as responding to clicks, touches, and swipes on display elements to display information or transmit data. For example, the display device can send commands to a player in an electronic device to play corresponding music. Electronic devices can include mobile phones and tablets, and display devices can include mobile phone screens and tablet screens.

[0046] The user interface may include the user interface in an application (hereinafter referred to as an application), the operating interface of a smart driving device, the home screen interface of a mobile phone, etc. In this embodiment of the application, the user interface in the application is used as an example for illustration.

[0047] The user interface (referred to as a page) for any application may include, but is not limited to, one or more pages such as a loading screen, onboarding screen, home screen, and content page. Typically, an application icon is displayed on an electronic device. When the electronic device detects a trigger action (such as clicking) on ​​the application icon, it responds by displaying the application's loading screen. The loading screen is the first page displayed after opening the application, and it usually consists of a solid color or gradient background, a logo, copyright information, etc. The loading screen is typically displayed for a short period, such as 2-3 seconds. After the loading screen loads, an onboarding screen is usually displayed. The onboarding screen generally consists of several consecutive, styled pages used to showcase the application's main functions and features before actual use, or to introduce the new features of a new version after an update. The onboarding screen is usually displayed when the application is first opened after downloading. If it's not the first time opening the application, the home screen can be displayed directly after the loading screen loads. The application's home screen is usually the most important page, showcasing some or all of the application's functionalities. For example, assuming the application is a shopping application, its home screen might look like this: Figure 1A As shown, the homepage of a shopping app can consist of multiple parts, including a search bar at the top, multiple horizontal product sections, multiple vertical product sections, and a navigation bar at the bottom. The horizontal and vertical orientations can be as follows: Figure 1A As indicated by the annotations in the diagram. For example, multiple horizontal product columns can consist of 8 components such as "Recharge Center" and "Platform Subsidy," and 4 components such as "Limited-Time Offer" and "Live Stream." Multiple vertical product columns can include multiple product images and text information (such as prices) as components. The application page displayed in response to a trigger action on the homepage can be called a content page. Typically, content pages can include various functional types such as list pages, detail pages, personal center pages, settings pages, and chat pages. For example, when a trigger action is detected on a page such as... Figure 1A The trigger action of the icon of any of the multiple product columns shown will display a content page of the details page function in response to the trigger action. This content page can be used to display the product's detailed information, such as product title, product price, product detail image, monthly sales, etc.

[0048] Different electronic devices use different display parameters, such as screen size and aspect ratio, to constrain the display effect of a page. Furthermore, even for the same electronic device, such as a foldable phone, the screen size differs between its folded and unfolded states, leading to different display effects for the same application's page in these two screen states. For example, a page might display complete information on a folded phone screen, but when the phone is unfolded, the amount of information displayed might be reduced.

[0049] Foldable screen phones can be folded in various ways, such as folding inwards or outwards, or folding vertically.

[0050] For example, if the electronic device is a foldable screen phone that folds inwards and outwards, and it is in the folded state, the homepage of the shopping application page displayed on the foldable screen phone can be as follows: Figure 1A As shown. When a foldable phone switches to its unfolded state, the screen size increases, and correspondingly, the sizes of each displayed element on the page also increase proportionally. For example, suppose... Figure 1A The displayed element "Recharge Center" is 50 pixels x 50 pixels. When expanded, it appears as follows on the screen. Figure 1B As shown, the size of the "Recharge Center" can be increased to 80 pixels × 80 pixels. Correspondingly, Figure 1B The sizes of the various display elements shown also increase to some extent. Because the width and height (or length) of each display element on the page increase in the expanded state, while the screen height remains unchanged (only the width changes), the expanded screen may not be able to display all the enlarged display elements, resulting in the page missing some content in the expanded state, such as... Figure 1B As shown, there may be a lack of vertical product columns, resulting in less content displayed on the homepage. If the size of each display element on the page remains unchanged after the foldable phone is switched to the unfolded state, it will result in a lot of blank areas on the left and right sides of the page, wasting display space.

[0051] If the electronic device is a foldable screen phone that folds vertically and horizontally, and is in the unfolded state, it can display something like this. Figure 1A The page shown illustrates this. When a foldable phone switches to its folded state, the screen height decreases while the width remains constant, resulting in a smaller screen size. In one scenario, the size of displayed elements on the page can remain unchanged; however, if switched to the folded state, such as... Figure 1A The sizes of the display elements shown remain unchanged. However, due to the screen size being smaller in the folded state than in the unfolded state, it is impossible to display elements like... Figure 1A The full content of the page shown results in a reduction of the amount of page information in the application, such as... Figure 1C As shown, the page displayed in the collapsed state lacks several vertical product columns.

[0052] For example, if the electronic device is a tablet, the display of the shopping application page on the mobile phone can be as follows: Figure 1A As shown. The same shopping app page, when displayed on a tablet, can be like... Figure 1D As shown. Because the screen size of a mobile phone is smaller than that of a tablet computer, if the size of each displayed element on the page remains unchanged, then the page displayed on a tablet computer may include, for example: Figure 1D The content shown is divided into multiple sections, but there is a lot of blank space on the left and right sides of the tablet, which reduces the content displayed on the homepage and wastes a lot of display space.

[0053] Typically, for any application that needs to be displayed on an electronic device, after the application data has been successfully downloaded to the device, developers can formulate customized page layout strategies based on the application and the display parameters (such as screen size, aspect ratio, etc.) of different types of electronic devices to ensure that the same page displays consistent and complete content on different electronic device displays, and to ensure that the same page displays consistent content in different states (such as folded and unfolded) of the same foldable phone, avoiding information loss. This ensures that the display effect of the electronic device is compatible with the application's page. For example, the same content with the same amount of information can be displayed in both the folded and unfolded states of the foldable phone, improving the user experience.

[0054] For example, please refer to Figure 2 , Figure 2 This diagram illustrates a process for developing page layout strategies for various applications. For example... Figure 2 As shown, firstly, for each application, developers can analyze and understand the structure and size of the page's view based on the application's page data. Based on the information obtained from the analysis, developers can adjust the page layout strategy of each application, such as determining the position and size of each display element on the page. Developers can debug and adjust the page layout strategy to obtain the target page layout strategy. After completing the setting and debugging of the page layout strategy, the configuration can be distributed, that is, the page layout strategy corresponding to each application can be configured in the electronic device, so that the electronic device can display the page corresponding to each application based on the page layout strategy.

[0055] In this context, a view is an object drawn on the screen for user interaction. Typically, an application's page includes multiple views, each representing the same or different content. For example, ... Figure 1A The top navigation bar shown can be composed of a single view; for example, Figure 1A The horizontal product columns shown can form a view, and sections like "Recharge Center," "Platform Subsidy," and "Online Supermarket" can also be views. The structure within a view refers to the structural relationships between the views; for example, a structural relationship could mean that one view is a child or parent element of another view. Figure 1A The bottom navigation bar shown is the parent element of the "Messages" section within the bottom navigation bar; "Messages" is the child element of the bottom navigation bar. The size of a view can be used to indicate the size of a rectangular area on the screen that the view occupies, and the unit of size can be pixels. For example, the size of the bottom navigation bar can be 800×200, indicating that the bottom navigation bar occupies a rectangular area on the screen that is 800 pixels wide and 200 pixels high.

[0056] Typically, smartphones, tablets, and other electronic devices lack third-party access engines or other management tools to constrain and manage the pages of third-party applications. This makes it difficult to require third-party applications to input their data into the device according to a unified standard, and also difficult to require them to independently set and input various constraints applicable to the device. Consequently, developers need to analyze each application separately, identify its constraints, and develop matching layout strategies based on those constraints. This process is cumbersome, has limited universality, and low engineering feasibility. If the electronic device is a foldable phone, even for the same application, two different layout strategies must be developed for the folded and unfolded states. Furthermore, application updates require maintenance by developers, resulting in excessive manpower investment and low practicality.

[0057] Based on this, this application provides a page data processing method that can be applied to electronic devices, enabling electronic devices to identify functional nodes in the application's page based on artificial intelligence (AI) technology, which is beneficial for subsequent configuration of page layout strategies based on functional nodes.

[0058] Please see Figure 3 , Figure 3This is a schematic diagram of an AI processing system provided in an embodiment of this application. The AI ​​processing system may include multiple modules such as a data understanding module, a scene recognition module, a planning algorithm module, and a decision-making strategy module. The AI ​​processing system can receive input data, which may include device information, application data, constraints, and other data from the electronic device. The application data may include data from the page corresponding to the application, such as text and images included on the page. Constraints describe the positional relationships between views on the page. During the display of the page, the electronic device can solve for the constraints in real time to determine the layout scheme for the current display area on the screen.

[0059] The data understanding module can be used to analyze input data and determine information such as location information, relational structure, and element characteristics. Location information can refer to the location information of electronic devices; relational structure can refer to the data structure in the application data; element characteristics can refer to the data features in the application data, such as data attributes and data storage methods.

[0060] The planning algorithm module can be used to determine constraints, perform high-performance solutions, and define evaluation criteria based on the processing results of the data understanding module. Constraints describe the positional relationships between views on the page. High-performance solutions refer to the ability of electronic devices to solve for constraints in real time during page display to determine the layout scheme for the current display area on the screen. Evaluation criteria are the evaluation conditions used to assess the layout scheme for the current display area.

[0061] The scene recognition module can be used to identify modules and functional scenarios in various applications. Based on the processing results of the data understanding module and the recognition algorithm determined by the planning algorithm module, module recognition can identify each functional node in the application; functional scenario recognition refers to identifying the specific functional scenario corresponding to each identified functional node, such as search bars, vertical lists, waterfall layouts, and other functional scenarios.

[0062] The decision strategy module can be used to scale and adjust the display interface of each application, add candidate elements, and determine the layout strategy of each application based on the processing results of the planning algorithm module.

[0063] The AI ​​processing system can output the layout strategies, location information, and other data of each application obtained after processing by the aforementioned modules to the processor in the electronic device. Based on the layout strategies and location information of each application, the processor can perform rendering and drawing, thereby obtaining a display interface adapted to each application.

[0064] Based on such Figure 3The AI ​​processing system shown can use AI technology to adaptively analyze the data of the corresponding page of the application and adaptively identify functional nodes. While improving the accuracy and versatility of functional node identification, it can significantly reduce the workload of developers.

[0065] Please see Figure 4 , Figure 4 This is a flowchart illustrating a page data processing method provided in an embodiment of this application. The method can be executed by an electronic device, a processor within an electronic device, or a chip or chip system with processor functionality. Figure 4 As shown, page data processing methods may include, but are not limited to, the following steps:

[0066] S401, Obtain the input file of the page to be processed for the application.

[0067] The application to be processed can be any type of application, such as a shopping application, a map application, an audio / video playback application, etc., and this application does not limit it. For example, the application to be processed can be such as... Figure 1A The shopping application shown is an example. Optionally, after downloading an application, the electronic device can treat it as an application to be processed and generate download completion information to instruct the electronic device to analyze and understand the pages of the application to be processed. In response to the application download completion information, the electronic device can obtain the input file of the pages to be processed for the application. Optionally, when an existing application on the electronic device completes a version update, the application can be treated as an application to be processed, and version update completion information can be generated to instruct the electronic device to obtain the input file of the pages to be processed for that application.

[0068] The pending pages of an application may include multiple pages such as loading pages, onboarding pages, home pages, and content pages. If there are multiple pending pages for an application, the electronic device can process each page sequentially. For example, it could be as follows: Figure 1A The homepage of the shopping app shown.

[0069] The input file for the page to be processed includes the page data of the page to be processed. Based on the input file, the electronic device can determine the relationship structure, size, and position of each view within the page to be processed, thereby determining the page layout strategy for the page to be processed. The input file for the page to be processed can be an Extensible Markup Language (XML) file. The page layout of the page to be processed can form a tree structure, called a view tree. The "leaves" or "branches" in the view tree can be defined as nodes, and a node can be used to represent a view. In other words, the XML file of the page to be processed can include the data of all views included in the page to be processed, i.e., all nodes. A node can include multiple data fields, each of which can represent an attribute of a page.

[0070] For example, the input file for the page to be processed can be as follows: Figure 5A As shown. Figure 5A The labels ①-④ in the diagram represent four nodes. For example... Figure 5A As shown in the diagram, node ① contains two child nodes, ② and ④. In other words, node ① is the parent node of nodes ② and ④, and nodes ② and ④ are both child nodes of node ①. Furthermore, node ② is the parent node of node ③, and node ③ is a child node of node ②. Each node includes multiple data fields as shown in node ①. For simplicity, Figure 5A The data fields included in each node are not fully shown in the diagram.

[0071] For example, Figure 5A The view tree corresponding to nodes ①-④ shown can be as follows: Figure 5B As shown in the diagram, in this view tree, the root node is node ①, and nodes ② and ④ are child nodes of node ①. In the view tree, this can be represented as nodes ② and ④ being branches of node ①. Node ③ is a child node of node ②, and in the view tree, this can be represented as node ③ being a branch of node ②.

[0072] For example, when the application to be processed is such as Figure 1A The shopping application shown may include various data fields in the input file of the page to be processed, as shown in Table 1.

[0073] Table 1: Comparison of Data Field Meanings

[0074] Data fields meaning class Node attributes content-desc Content Description text text resource-id Resource Identifier index hierarchy bounds scope isImportant Important attributes isVisible Visual attributes package Package Name checkable Check attributes clickable Click Properties scrollable Scroll properties long-clickble Long press attributes focused Concentrated attributes password Password attributes selected Select attribute

[0075] Here, `class` is a tag name used to indicate node attributes. For example, `class="android.widget.FrameLayout"` indicates that the node's attribute is the frame layout of an application plugin (also known as a component) in the Android system. Another example is `class="android.widget.TextView"`, which indicates that the node's attribute is text, and the node mainly contains text content, such as detailed product information (e.g., place of origin, production date). `class="android.widget.ImageView"` indicates that the node's attribute is image, and the node mainly contains images, such as detailed product images.

[0076] `content-desc` is a short text description used to describe components in an application, such as buttons, images, and text fields. It's typically used in accessibility services to help users identify and interact with these components. For example, `content-desc="This is a button"` indicates that the component comprised of this node is a button. As another example, `content-desc="null"` indicates that the node has no text description.

[0077] The `text` parameter represents the text content contained in the node. For example, `text="music"` means that the text "music" will be displayed at the location of the node on the page to be processed; `text="null"` means that no text content will be displayed at the location of the node.

[0078] `resource-id` represents a resource identifier and can be used to locate nodes. Based on a node's `resource-id`, its location can be found in the input file of the page to be processed, thus allowing the retrieval of its data fields. For example, `resource-id="android:id / content"` means that the node can be found in the input file of the page to be processed by inputting "android:id / content". `resource-id="null"` indicates that the node's resource identifier is empty. Different nodes may have the same or different resource identifiers. If multiple nodes have the same resource identifier, assuming they are all "android:id / content", then searching for "android:id / content" in the input file will filter out all nodes with the resource identifier "android:id / content".

[0079] The `index` parameter represents the hierarchy of a node, indicating its distance from the root node in the view tree. Different nodes may have the same or different hierarchy levels. For example, two nodes with different hierarchy levels are not at the same level; two nodes with the same hierarchy are at the same level. Based on the `index` values ​​of multiple nodes, the structural relationships between them can be determined. For example, `index = "0"` indicates that this node is the largest view, such as... Figure 5A Node ① in the view tree is also called the root node of this view tree; index="1" indicates that this node is the next level node of the largest node, such as... Figure 5A The nodes ② and ④ in the diagram both have a level of 1, indicating that both nodes are children of node ① and are 1 unit away from the root node. Figure 5A The level of node ③ is 3, which means that node ③ is the next level node of the node with "index=1", such as node ③ being a child node of node ②.

[0080] `bounds` represents the scope, indicating the coordinates and size of the view represented by the node. For example, `bounds = "[0, 0][1122, 91]"` means that the coordinates of the top-left pixel (starting coordinates) of the node are [0, 0], and the coordinates of the bottom-right pixel (ending coordinates) are [1122, 91]. The rectangular area enclosed by the top-left and bottom-right pixels is the scope of the node. Based on this, the size (width × height) of the node can also be determined as 1122 pixels × 91 pixels. The `bounds` property of a specific node can be used to represent the screen size of an electronic device, such as... Figure 5A The bounds="[0,0][1122,2442]" value in the code indicates that the screen width of the electronic device is 1122 pixels and the height is 2442 pixels.

[0081] `isImportant` indicates whether a node is important. This attribute can be set in the application being processed. For example, `isImportant="false"` indicates that the node is marked as unimportant; `isImportant="true"` indicates that the node is marked as important. The "important" status of a node can be set and modified, and different processing methods can be applied to "important" and "unimportant" nodes. This application does not limit the specific processing methods.

[0082] `isVisible` is used to indicate whether a node is visible. For example, `isVisible="true"` indicates that the node is visible and can be displayed on electronic devices for direct viewing by the user. Figure 1AThe top navigation bar shown. `isVisible="false"` indicates that the node is invisible, like the hidden border of a view.

[0083] The other data fields in Table 1 and their corresponding meanings are as described in Table 1, and will not be repeated here.

[0084] In one possible implementation, the input file for the page to be processed can be obtained based on the accessibility service in the Android system. For example, based on the accessibility service, nodes can be extracted from the raw data of the page to be processed. Multiple nodes required by the electronic device can be extracted, and the extracted nodes can be filtered to retain multiple data fields as shown in Table 1, thereby determining the data fields included in each node. The nodes are arranged and stored according to the view tree structure, forming an XML file.

[0085] S402, Based on the input file, determine the node characteristics of each node in the page to be processed.

[0086] The node characteristics of each node in the page to be processed refer to feature information related to the page layout. Some node characteristics can be directly obtained from the node's data fields in the input file, while other node characteristics can be determined based on the data fields of one or more nodes. The node characteristics of each node can be divided into two categories: the first category can be size information, which can be called the first type of node characteristics; the second category can be view attribute information, which can be called the second type of node characteristics.

[0087] The size information, i.e., the first type of node features, can include information such as the node's width and height. Size information can be determined from the data field `bounds` as shown in Table 1 above. For example, if a node's data field `bounds = "[10, 20][500, 780]"`, then the node's width is determined to be 490 pixels and its height to be 760 pixels.

[0088] View attribute information, i.e., the second type of node features, can include view structural relationship information and view attributes. View attributes can include feature information represented by data fields such as node attributes (class) and hierarchy (index) in Table 1. Based on node attributes, it can be determined whether the content of a node is text or image; based on the index, the hierarchy of the node can be determined. The structural relationship of the view, also known as the structural feature of the node, is used to represent the structural relationship between nodes, such as the number of child nodes a node includes. By the hierarchy of a node and the hierarchy of multiple nodes adjacent to that node, the structural feature of the node can be determined, for example, it can be determined that... Figure 5ANode ① is the parent node of nodes ② and ④. If a node includes one or more child nodes, the feature information of that node must also include the width of the child nodes.

[0089] In summary, the node characteristics of a node may include node width, node height, node attributes, hierarchy and structural features, and the node width of its child nodes. Optionally, the node characteristics of a node may include some or all of the above-mentioned characteristics, and may also include other characteristics besides the node characteristics mentioned above, such as sliding attributes and concentration attributes shown in Table 1. This application does not limit these characteristics.

[0090] S403, based on the node characteristics of each node, determine the nodes in the page to be processed that meet the preset node characteristic conditions as functional nodes.

[0091] A functional node is a node on the page to be processed; it can also be called a functional module or display module. A functional node occupies a certain area on the page and is used to display the page content represented by the data fields of that functional node. A functional node can include one or more child nodes; in other words, a functional node can include one or more views.

[0092] Typically, a page to be processed may include multiple functional nodes. For example, such as Figure 1A The homepage of the shopping app shown can have functional nodes such as the search bar at the top and the navigation bar at the bottom.

[0093] Based on node characteristics (including first-type and second-type node characteristics), it can be determined whether a node is a functional node. The criteria used to determine whether a node is a functional node can be called preset node characteristic conditions. Electronic devices can be configured with preset node characteristic conditions; optionally, these preset node characteristic conditions can also be called preset node recognition rules. By traversing all nodes in the input file of the page to be processed, based on the preset node characteristic conditions, it can be sequentially determined whether the characteristic information of each node meets the preset node characteristic conditions, thereby determining whether the node is a functional node. If the node characteristics of a node meet the preset node characteristic conditions, then the node can be determined to be a functional node.

[0094] Among them, based on the first type of node features and the second type of node features, the preset node feature conditions can be divided into first feature conditions and second feature conditions. The first feature conditions may include height conditions and width conditions; the second feature conditions may include node attribute conditions or child node conditions. Please refer to the description below for the specific content of each condition in the preset node feature conditions.

[0095] For any node (referred to as node i) in the page to be processed, the method to determine whether the node is a functional node can be as follows: Figure 6 As shown. For example, as... Figure 6 As shown, taking node i in the page to be processed as an example, the method may include, but is not limited to, the following steps:

[0096] S601, Based on the data field of node i in the input file, determine the node characteristics of node i.

[0097] The first type of node features for node i may include features such as node width and node height. The second type of node features may include one or more features such as structural features, node attributes, and hierarchy. If node i includes one or more child nodes, the feature information of node i may also include the node features of all child nodes of node i, such as the node width, node height, and node attributes of the child nodes.

[0098] S602, determine whether the width of node i meets the width condition. If the width of node i meets the width condition, execute S603; otherwise, execute S608.

[0099] The node width can be determined based on the `bounds` data field of node `i`. For example, if `bonus = "[0, 0][216, 890]"`, the node width is determined to be 216 pixels. Furthermore, the width of the electronic device (referring to the screen width of the electronic device) can be determined from the `bounds` data field of the first node in the input file of the page to be processed; alternatively, the screen width of the electronic device can be directly obtained from the electronic device itself. For example, below we use `w1` to represent the node width and `w0` to represent the screen width, where both `w1` and `w0` are integers greater than 0, and the unit for both `w1` and `w0` is pixels.

[0100] The width condition is a preset width condition among the node characteristic conditions. If the width of node i satisfies any one of the following width conditions AC, then the node can be determined to satisfy the width condition:

[0101] A. The node width is equal to the screen width, which can be represented as: w1 = w0;

[0102] B. A node width equal to half the screen width can be represented as:

[0103] C. A node width greater than or equal to three-quarters of the screen width can be represented as:

[0104] In this context, if node i satisfies width condition A, it means that the node is equal to the screen width. Typically, a node satisfying width condition A can be the largest functional node in the onboarding page, such as a background node. If a node satisfies width condition B, it means that the node may be a module occupying a large display area on the screen, for example, it could be... Figure 1A The diagram shows multiple vertical product columns. If a node satisfies the width condition C, it can be represented as follows: Figure 1A The example shows wider functional nodes such as the search bar at the top or the navigation bar at the bottom.

[0105] S603, determine whether the node height of node i meets the height condition. If the node width of node i meets the height condition, then execute S604; otherwise, execute S608.

[0106] The height condition, which is a preset node feature condition, can be expressed as: the node height of node i is greater than a first threshold. The height of node i can be determined based on its data field `bounds`. Optionally, the size of the first threshold can be set based on the actual application scenario. For example, the first threshold can be set to 2 pixels.

[0107] For example, if the x-coordinates of the starting and ending coordinates of node i are equal, such as bounds = "[10, 210][110, 210]", then the node height can be determined to be 0, indicating that the node may be a horizontal dividing line. Alternatively, if the y-coordinates of the starting and ending coordinates of node i are equal, such as bounds = "[10, 210][10, 400]", then the node width can be determined to be 0, indicating that the node may be a vertical dividing line.

[0108] Specifically, if the width of node i satisfies the width condition and the height of node i satisfies the height condition, then the node characteristics of node i can be said to satisfy the first characteristic condition. Optionally, if the node characteristics of node i satisfy the first characteristic condition, the node can be determined as a candidate node. In other words, when node i satisfies the first characteristic condition, it can be ruled out that node i is a dividing line or a node whose corresponding display area is too small, which is beneficial to the accuracy of subsequently determining functional nodes from candidate nodes.

[0109] Furthermore, it is necessary to determine whether a candidate node is a functional node based on whether the second type of node features of the node satisfy the second feature condition.

[0110] For example, suppose a page to be processed is like Figure 7A As shown, the labels af represent 6 views, or in other words, 6 nodes. Node d can include one or more child nodes, and node f can also include one or more child nodes. Figure 7AIn the diagram, small squares represent child nodes of node d, and large squares represent child nodes of node f. For... Figure 7A For any node in the array, its status as a candidate node can be determined based on its first-type node characteristics. For example, node a has a width equal to the screen width, so node a is not a candidate node. Nodes b, c, d, and f all satisfy the condition that their node width is greater than or equal to three-quarters of the screen width, and their node height is greater than the first threshold (assumed to be 0 pixels). Therefore, nodes c, d, and f can also be determined as candidate nodes. The child nodes included in nodes d and f do not satisfy the width condition, so they are not candidate nodes. Node e has a node width greater than or equal to three-quarters of the screen width, but its node height is 0; therefore, node e is not a candidate node.

[0111] In one possible implementation, for node i, we can first determine whether the node height of node i meets the height condition; if the node height of node i meets the height condition, we then determine whether the node width of node i meets the width condition. Based on this, if the node height of node i meets the height condition and the node width meets the width condition, we can determine that node i meets the first feature condition and can identify node i as a candidate node.

[0112] S604, determine whether node i is a single node. If node i is a single node, execute S605; otherwise, execute S606.

[0113] Based on the structural characteristics of a node, it can be determined whether the node is a single node. If the structural characteristics of a node indicate that the node has no child nodes, then the node can be called a single node; conversely, if the structural characteristics of a node indicate that the node includes one or more child nodes, then the node is not a single node and can be called a non-single node. Based on the data field `index`, the hierarchy of node `i` can be determined. Based on the hierarchy of other nodes adjacent to node `i` in the input file of the page to be processed, it can be determined whether node `i` has a parent node or child nodes, thus determining the structural characteristics of node `i`. For example, based on... Figure 5A By examining the hierarchy of node ① and its adjacent nodes ②-④, we can determine that node ① is the parent node of nodes ② and ④. Therefore, node ① is not a single node; its structural characteristics indicate that it includes two child nodes. Figure 5A Nodes ③ and ④ in the diagram have no child nodes, therefore both nodes ③ and ④ are single nodes. The structural features of node ③ indicate that node ③ has no child nodes, and the structural features of node ④ indicate that node ④ has no child nodes.

[0114] Furthermore, if the width or height of all child nodes of node i is less than the first threshold, node i can also be considered a single node, and S605 can be executed. The width and height of the child nodes of node i can be determined based on the bounds of each child node.

[0115] S605, determine whether the node attribute of node i is a text attribute. If the node attribute of node i is a text attribute, then execute S607; otherwise, execute S608.

[0116] If node i is a single node and its node attribute is a text attribute, then node i can be said to satisfy the node attribute condition; if node i is a single node and its node attribute is not a text attribute, then node i can be said to not satisfy the node attribute condition.

[0117] Based on the value of the node attribute (class) of node i, it can be determined whether node i has a text attribute. For example, if class = "android.widget.TextView" in the feature information of node i, it means that the node attribute of node i is a text attribute. If class = "android.widget.ImageView", it means that the node attribute of node i is an image attribute.

[0118] In one possible implementation, the conditions that a node attribute must meet when the node is a single node can be adjusted based on different application scenarios. For example, if the application to be processed is an image-based application, the condition in S605 can be modified to: node i is a single node, and the node attribute of this node is an image attribute. This allows images in image-based applications to be treated as a separate module.

[0119] S606, determine whether the child nodes of node i meet the child node condition. If the child nodes of node i meet the child node condition, then execute S607; otherwise, execute S608.

[0120] Child node conditions are used to represent the conditions that child nodes of non-single nodes must satisfy. When node i is not a single node, if the child nodes of node i satisfy the following conditions, then it can be determined that the child nodes of node i satisfy the child node conditions: there exists a child node whose node width is different from the node width of node i, and the node width and node height of the child node are both greater than the second threshold.

[0121] Node i may include one or more child nodes, and the width of each child node can be determined based on the `bounds` data field of each child node. Among the one or more child nodes included in node i, one or more child nodes may have a width different from the width of node i. The second threshold can be set to any pixel value; typically, the second threshold can be equal to the first threshold, such as both being set to 2 pixels. Generally, if the width or height of a child node is less than the second threshold, that child node can be considered as a dividing line separating different nodes.

[0122] Normally, when the width or height of a child node is less than the second threshold, it indicates that the child node occupies a small display area on the screen. These child nodes are typically used as horizontal or vertical dividing lines to separate different nodes, and usually do not contain text, images, or other information. If the width or height of every child node in node i is less than the second threshold, it means that all child nodes in node i can be considered dividing lines and do not contain any information. Therefore, in this case, it can be determined that the child nodes of node i do not meet the child node condition, and thus S608 is executed.

[0123] S607, determine that node i is a functional node, and determine that the child nodes of node i are non-functional nodes.

[0124] In summary, when node i simultaneously satisfies both the width and height conditions, and node i is a single node with text attributes (meaning its node characteristics simultaneously satisfy the first feature condition (width and height conditions) and the second feature condition (node ​​attribute condition), node i can be determined as a functional node. Alternatively, when node i simultaneously satisfies both the width and height conditions, and node i is not a single node, and its child nodes satisfy the child node condition (meaning its node characteristics simultaneously satisfy the first feature condition (width and height conditions) and the second feature condition (child node condition), node i can be determined as a functional node. Furthermore, when node i is determined to be a functional node, its child nodes can be determined to be non-functional nodes; that is, when traversing the nodes in the page to be processed, the child nodes of nodes already determined to be functional nodes can be skipped.

[0125] For any given functional node, a pruning check is required. This check determines whether a node meets the pruning criteria if it is already identified as a functional node, and if it includes other functional nodes already identified as functional nodes, or if its width equals the screen width of the electronic device. In other words, the pruning check is a verification and correction process used to check for errors in the functional nodes identified based on the first and second categories of node characteristics, and to correct them promptly.

[0126] For example, such as Figure 7AThe view tree structure of the input file for the page to be processed shown can be as follows: Figure 7B As shown. Figure 7B As shown, the root node of the view tree is node a, and the child node of node a is node b; the child nodes of node b include nodes c through f. Node d includes multiple child nodes, and node f includes multiple child nodes. Figure 7B Not shown in the diagram. Based on preset node feature conditions and the idea of ​​a node-level clustering algorithm, the nodes are traversed sequentially according to their levels, as follows: Figure 7B The functional nodes are determined by examining each node in the view tree shown. Assume that nodes c-f are already identified as candidate nodes among nodes a-f. For example, since node a is determined not to be a candidate node, we can continue traversing from node b. Node b is a single node, but its node attribute is not text, but an image attribute (a background image), therefore node b is not a functional node. Node c is a single node with a text attribute, therefore node c is a functional node. Node d is not a single node, and the width of its child nodes does not meet the width requirement, therefore the child nodes of node d are not functional nodes. Furthermore, the width of the child nodes of node d is different from the width of node d itself, therefore node d is a functional node, and there is no need to continue traversing its child nodes. Node e is not a candidate node and can be skipped. Node f is similar to node d; the child nodes of node f are not functional nodes, and the width of the child nodes is different from that of node f, therefore node f is a functional node, and there is no need to continue traversing its child nodes. Based on this, nodes c, d, and f can be identified as functional nodes.

[0127] Optionally, the specific conditions in the preset node feature conditions can be modified, added, or deleted adaptively based on the application scenario, and this application does not limit this.

[0128] In one possible implementation, optionally, a node can first be determined as a candidate node based on the first type of node characteristics, as described in steps S602-S603 above. If it is a candidate node, it can then be further determined as a functional node based on the second type of node characteristics, as described in steps S604-S606 above. If a node is determined not to be a candidate node, it can be determined that the node is not a functional node. Optionally, for any given node, it can be preferentially determined whether the node is a candidate node based on the second type of node characteristics. If the node is a candidate node, then it can be determined whether the candidate node is a functional node based on the first type of node characteristics, i.e., whether the node is a functional node.

[0129] S608, determine that node i is a non-functional node.

[0130] If node i does not meet the width condition or the width condition, or if node i is a single node and does not meet the node attribute condition, or if node i is not a single node and does not meet the child node condition, it can be determined that node i does not meet the preset node characteristic condition, and thus node i can be determined as a non-functional node.

[0131] Based on such Figure 6 The method shown can traverse every node in the page to be processed, identify nodes that meet the preset node characteristic conditions, and determine these nodes as functional nodes.

[0132] S404, based on the function node, determine the function node file corresponding to the page to be processed.

[0133] The function node file contains some or all of the data fields for the function nodes in the page to be processed. The function node file can be used to adjust the page layout of the page to be processed. The function node file can be a JavaScript key-value pair data (JavaScript object notation, JSON) file; for example, the format of a function node file may be as follows: Figure 8 As shown.

[0134] like Figure 8 As shown, the functional node file may include data fields for each functional node in the page to be processed. This may include some or all of the data fields shown in Table 1, as well as data fields determined based on the data fields shown in Table 1. For example, the functional node file may include the package name, node identifier (node_indexId), node ID (node_id), level (index), node attribute (class), bounds, width, height, number of child nodes (son_nums), clickable attribute, scrollable attribute, and position attribute (loc) for each functional node in the page to be processed.

[0135] Here, the node identifier (node_indexId) is a unique identifier for each node in the page to be processed. Based on the natural order of the nodes in the input file of the page to be processed, each node can correspond to a node identifier in turn. For example, as... Figure 5A In the input file shown, in order, node ① is the first node in the page to be processed, and the node identity of node ① can be represented as node_indexId: "index: id / 1"; nodes ②-④ are the second, third, and fourth nodes in the page to be processed, respectively. For example, the node identity of node ③ can be represented as node_indexId: "index: id / 3".

[0136] `node_id` is the same data field as `resource-id` in Table 1 above. `index`, `class`, and `bounds` are also the same data fields as in Table 1.

[0137] Width and height are determined based on bounds. Width refers to the width of the node, and height refers to the height of the node.

[0138] The position attribute is a vector combining the coordinates of the top-left and bottom-right pixels of the functional node, which can be used to determine the position of the functional node. The position attribute can be determined based on the node's range, such as "loc": "[36, 542, 1086, 1102]" based on "bounds": "[36, 542, 1086, 1102]".

[0139] Figure 8 In the diagram, the functional node identified as 25 can be referred to as the main functional node in the page to be processed. The nine functional nodes identified as 26-34 are sub-functional nodes of the main functional node. Each sub-functional node also includes the aforementioned data fields. For the sake of brevity, Figure 8 The details of the data fields included in each sub-functional node are not shown.

[0140] S405: Based on the function node file and the preset display effect image of the page to be processed, determine the marker image corresponding to the page to be processed, and output the marker image on the image display interface.

[0141] The image display interface can refer to any interface in an electronic device that can be used to display images. The marked image corresponding to the page to be processed refers to an image obtained by marking the display areas corresponding to the identified functional nodes in the background image, based on the functional nodes. The marked image corresponding to the page to be processed can also be called the image corresponding to the functional node identification result of the page to be processed. The preset display effect image corresponding to the page to be processed refers to the image in which the page to be processed displays the complete preset page content on the screen. For example, for a page to be processed... Figure 1A By taking screenshots or performing other operations on the interface displayed on the electronic device, a preset display effect image corresponding to the page to be processed can be obtained.

[0142] Optionally, the electronic device can output the marker image to a gallery within the electronic device and display the marker image on the corresponding image display interface of the gallery. Optionally, the electronic device can output the marker image to another electronic device, which receives the marker image and displays it on the image display interface of that other electronic device.

[0143] Developers can verify the accuracy of functional node recognition based on the marked images, and can also adaptively modify and adjust the preset node feature conditions to ensure that each functional node in the page to be processed can be accurately identified based on the preset node feature conditions.

[0144] For example, the marker image displayed on the image display interface can be as follows: Figure 9 As shown. If the application to be processed is a shopping application, the page to be processed can identify, for example, [the following]. Figure 9 The functional nodes shown are 1 through 5. Functional node 1 can be called the top search bar, used to receive search keywords. Functional node 2 can be called the horizontal toolbox, which may include multiple sections such as "Recharge Center" and "Platform Subsidies". Functional node 3 can be called the horizontal product bar, which may include multiple sections such as "Limited-Time Offers" and "Live Streams". Functional node 4 can be called the product waterfall layout, which may include multiple product descriptions, prices, and other information. Functional node 5 can be called the bottom navigation bar, which may include multiple sections such as "Homepage" and "Messages".

[0145] In one possible implementation, based on, for example Figure 3 The AI ​​processing system shown includes a scene recognition module that can determine the functional scene corresponding to each functional node based on a functional node file, such as a search bar, vertical list, waterfall layout, and other functional scenes. For different functional scenes, the decision strategy module can adjust the layout strategy of the page to be processed based on the functional nodes and the functional scene, combined with preset layout parameters of the preset display area on the screen used to display the page to be processed, such as the width, height, and resolution of the preset display area. The preset display area can be the screen area in the unfolded state of a folded screen, the screen area in the folded state, or the outer screen area of ​​a folded screen, etc.; the layout strategy can include the size and proportion of each node in its corresponding display area on the screen, and the positional relationship between each node. Furthermore, the processor in the electronic device can render and draw the page to be processed based on the layout strategy determined by the AI ​​processing system, thereby obtaining a display page adapted to the electronic device.

[0146] As can be seen, based on the established preset node feature conditions and the node features of each node in the page to be processed, the functional nodes in the page to be processed can be accurately identified, improving the accuracy and efficiency of the electronic device in understanding and recognizing the page to be processed. This is beneficial for planning the layout strategy of the application to be processed on different display screens. In addition, the embodiments of this application can also display the recognition results of functional nodes in a visual way, that is, display the marked image corresponding to the page to be processed, which is beneficial for developers to adjust the preset node feature conditions to further improve the accuracy of functional node recognition.

[0147] The following describes an exemplary electronic device provided by an embodiment of this application. Please refer to... Figure 10 , Figure 10 A schematic diagram of the structure of the electronic device 100 is shown.

[0148] The following description uses electronic device 100 as an example to illustrate the embodiment. It should be understood that electronic device 100 may have more than Figure 10 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0149] Electronic device 100 may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0150] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0151] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0152] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0153] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0154] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0155] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0156] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0157] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), a flexible light-emitting diode (FLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0158] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0159] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0160] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, electronic device 100 may include other cameras.

[0161] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0162] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0163] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0164] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0165] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as facial recognition, fingerprint recognition, mobile payment, etc.). The data storage area may store data created during the use of electronic device 100 (such as facial information template data, fingerprint information templates, etc.). Furthermore, internal memory 121 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0166] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0167] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0168] Figure 11 This is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0169] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the system runtime library layer, the hardware abstraction layer (HAL), and the kernel layer.

[0170] The application layer can include a series of application packages.

[0171] like Figure 11As shown, the application package can include system user interface (systemUI), camera, calendar, WLAN, music, SMS, gallery, call, map, Bluetooth, video, and other applications (also known as apps), as well as... Figure 3 The AI ​​processing system shown is described. The system UI is used to display the interface of the electronic device 100, such as displaying the signal icon corresponding to the SIM card, displaying the call interface, etc. In this embodiment, the system UI can be used to display, for example... Figures 1A-1D The various application interfaces are shown in the diagram. The AI ​​processing system can receive an input file of a page to be processed from any application in the application layer, and determine the functional nodes in the page to be processed based on the input file.

[0172] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0173] like Figure 11 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0174] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0175] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0176] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0177] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0178] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0179] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog-style notifications on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0180] The system runtime library layer is divided into two parts: C / C++ libraries and the Android runtime library. The Android runtime includes the runtime environment, which typically comprises core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The C / C++ libraries mainly include the media framework, surface manager, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics engines (e.g., SGL).

[0181] The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The 3D graphics processing library implements 3D graphics drawing, image rendering, compositing, layer processing, etc. The 2D graphics engine is the drawing engine for 2D drawing. In this embodiment, the 3D graphics engine and the 2D graphics engine can be used to render the interfaces of various applications.

[0182] The Hardware Abstraction Layer (HAL) isolates the application framework layer from the kernel layer, preventing the Android system from overly relying on the kernel layer. This allows the application framework layer to be developed without considering drivers. The HAL can include multiple functional nodes, such as display HAL, camera HAL, audio HAL, and sensor HAL modules.

[0183] The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, a sensor driver, and a shared memory driver. In this embodiment, the display driver can be used to display the application's interface.

[0184] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0185] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0187] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0189] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0190] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for processing page data, characterized in that, The method includes: Upon receiving information indicating that the application has completed download or version update, the node characteristics of the nodes on the pending page of the application are determined based on the page data. These node characteristics include a first type of node characteristic and a second type of node characteristic. The first type of node characteristic includes the node's width and height. The second type of node characteristic includes the node's structural characteristics and node attributes. The structural characteristics of a node indicate the number of its child nodes. The node attributes include text attributes or image attributes. Based on the characteristics of the first type of node, candidate nodes are determined from the nodes of the page to be processed; Based on the second type of node features, functional nodes are determined from the candidate nodes; the functional node is a candidate node with no child nodes and a text attribute, or the functional node is a candidate node with at least one child node, and the node width of the at least one child node is different from the node width of the candidate node with at least one child node, and the node width and node height of the at least one child node are both greater than the second threshold. Based on the functional nodes, the functional node file corresponding to the page to be processed is determined; the functional node file includes some or all of the data fields of the functional nodes; Based on the function node file and preset layout parameters, the display size and position of the function node in the preset display area are adjusted to determine the page layout corresponding to the page to be processed; the preset display area is the area on the screen used to display the page to be processed; the preset layout parameters include the width and height of the preset display area.

2. The method according to claim 1, characterized in that, The step of determining candidate nodes from the nodes of the page to be processed based on the first type of node features includes: From the nodes of the page to be processed, candidate nodes are identified whose node height is greater than a first threshold and whose node width meets a first condition. Wherein, candidate node i is any one of the candidate nodes, and the node width of candidate node i satisfies one or more of the following conditions: The node width of candidate node i is equal to the screen width; The node width of candidate node i is equal to half the screen width; The node width of candidate node i is greater than or equal to three-quarters of the screen width.

3. The method according to claim 1, characterized in that, The non-functional nodes in the page to be processed include the child nodes of the candidate node p. The non-functional nodes are the nodes in the page to be processed that are not the functional nodes. The candidate node p is any one of the candidate nodes in the page to be processed.

4. The method according to any one of claims 1-3, characterized in that, In the preset display area, the display area corresponding to the functional node includes the functional node and N child nodes of the functional node; N is an integer greater than or equal to 0.

5. The method according to any one of claims 1-3, characterized in that, The method further includes: Using the preset display effect image of the page to be processed as the background image, the display area corresponding to the functional node is marked in the background image to obtain a marked image; the proportion of the marked image is the same as that of the background image; The marked image is output on the image display interface according to the scale of the marked image.

6. An electronic device, characterized in that, include: Memory, processor; where: The memory is used to store computer programs, the computer programs including program instructions; The processor is configured to invoke the program instructions to cause the electronic device to perform the method as described in any one of claims 1-5.

7. A chip system, characterized in that, The chip system includes at least one processor, a memory, and an interface circuit. The memory, the interface circuit, and the at least one processor are interconnected via circuits. The at least one memory stores program instructions. When the program instructions are executed by the processor, the chip system performs the method described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-5.

Images