Webpage node hit method and electronic device

Abstract

Provided in the present application are a webpage node hit method and an electronic device. The method comprises: an electronic device displaying a webpage; the electronic device detecting a consumption event for the webpage; the electronic device performing hit testing on nodes in at least one visual layer corresponding to the webpage, so as to obtain a first test result, wherein the nodes in each visual layer correspond to visual elements in the webpage on a one-to-one basis; the electronic device performing hit testing on nodes in at least one event layer corresponding to the webpage, so as to obtain a second test result, wherein the nodes in each event layer correspond on a one-to-one basis to visual elements in the webpage that support interaction; and when the first test result indicates that the consumption event hits a first node in a first visual layer and the second test result indicates that the consumption event hits a first node in a first event layer, the electronic device determining that the consumption event hits the first node. By means of the solution, an electronic device achieves the effect of correctly hitting nodes by means of combining hit test results of a visual layer and an event layer, thereby improving the node hit efficiency and reducing lag.

Description

A method for webpage node detection and an electronic device

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411534568.1, filed on October 30, 2024, entitled "A Method for Hitting Web Page Nodes and an Electronic Device", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of terminal technology, and in particular to a webpage node hitting method and an electronic device. Background Technology

[0004] With the development of mini-programs, more and more application functions can be implemented through mini-programs. A mini-program's web page consists of various elements, such as text, images, and controls. Among these, there is a special type of built-in component called a native component. Native components differ from the built-in components rendered by the WebView; native components are rendered by the native client. Because native components and the built-in components rendered by the WebView are at different levels, and the native components are at a higher level, they cannot overlap with the built-in components rendered by the WebView, which may cause display abnormalities or user interaction abnormalities. To solve the limitations brought by native components, same-level rendering technology was introduced. In same-level rendering technology, native components can be directly rendered onto the WebView level. Native components can be directly mounted onto WebView nodes, allowing native components to be used like non-native components, ensuring that the native components and the elements of the web page have a consistent feel.

[0005] The main functions of in-layer rendering technology include: drawing native components onto the same layer area within a web page, and forwarding events from the same layer area to the native components. When a user triggers an action on a web page, the electronic device needs to perform a hit test on the web page. This hit test includes traversing all nodes on the web page. When the main thread is heavily loaded, the hit test may cause page lag and affect performance. Summary of the Invention

[0006] This application provides a webpage node hitting method and an electronic device to improve the efficiency and accuracy of node hitting.

[0007] Firstly, this application provides a webpage node matching method, which can be applied to an electronic device. In this method, the electronic device displays a webpage, the webpage including at least one visual element; the electronic device detects a consumption event of the webpage, and the electronic device performs a match test on nodes in at least one visual layer corresponding to the webpage, obtaining a first test result; wherein each visual layer includes at least one node, and each node in each visual layer corresponds one-to-one with a visual element in the webpage. The electronic device performs a match test on nodes in at least one event layer corresponding to the webpage, obtaining a second test result; wherein each event layer includes one node, and each node in each event layer corresponds one-to-one with an interactive visual element in the webpage. When the first test result indicates that the consumption event matches a first node in the first visual layer and the second test result indicates that the consumption event matches a first node in the first event layer, the electronic device determines that the consumption event matches the first node.

[0008] In this method, electronic devices can achieve the effect of correctly hitting nodes by combining the hit test results of the visual layer and the event layer, thereby improving the efficiency of node hitting, improving event response performance, and reducing lag.

[0009] In some implementations, the webpage node matching method provided in this application further includes: creating at least one visual layer and at least one event layer corresponding to the webpage based on the Document Object Model (DOM) tree of the webpage. In this way, the electronic device can create the visual layer and event layer corresponding to the webpage based on the DOM tree, thereby classifying the nodes corresponding to each element in the webpage. This allows the electronic device to perform match tests based on the visual layer and event layer respectively, ensuring the accuracy of node matching.

[0010] In some implementations, a hit test is performed on at least one node in the visual layer corresponding to the webpage to obtain a first test result, including: comparing the location information corresponding to the consumption event with the location information of at least one node in the visual layer; when the location corresponding to the consumption event is within the area of ​​the first node in the first visual layer, the first test result indicates that the consumption event hits the first node in the first visual layer.

[0011] In some implementations, a hit test is performed on at least one node in the event layer corresponding to the webpage to obtain a second test result, including: comparing the location information corresponding to the consumption event with the location information of the node in at least one event layer; when the location corresponding to the consumption event is within the area of ​​the first node in the first event layer, the second test result indicates that the consumption event hits the first node in the first event layer.

[0012] In some implementations, the webpage node hitting method provided in this application further includes: when a first test result indicates that the consumption event hits a first node in a first visible layer and a second test result indicates that the consumption event hits any node in at least one event layer other than the first node, a hit test is performed according to the DOM tree hitting strategy to determine the node hit by the consumption event; wherein, the DOM tree hitting strategy is to perform a hit test on each node in the DOM tree of the webpage.

[0013] In some implementations, the webpage node hit method provided in this application further includes: when the first test result indicates that the consumption event hits the first node in the visible layer and the second test result indicates that the consumption event does not hit any node, a hit test is performed according to the DOM tree hit strategy to determine the node hit by the consumption event; wherein, the DOM tree hit strategy is to perform a hit test on each node in the DOM tree of the webpage.

[0014] In this way, when the first test result and the second test result are inconsistent, the electronic device needs to re-perform the hit test on each node in the DOM tree of the webpage to determine the node that the consumption event hits, so as to ensure the accuracy of the node hit.

[0015] In some implementations, the visual element corresponding to the first node is a native component rendered to the webpage using same-layer rendering technology. In this way, when a user triggers an operation on the native component, the electronic device can quickly determine that the node consuming the event is the node corresponding to the native component based on the node-hitting method provided in this application, improving the efficiency of node hitting. This allows the webpage to promptly forward the hit event to the native component, enabling the native component to respond to the consumed event and improving response speed.

[0016] In some implementations, the webpage node hit method is applied to the compositing thread of the electronic device. This reduces the load on the main thread, thereby lowering the probability of page lag.

[0017] In some implementations, the consumption events include click events or drag events of users on interactive elements on a webpage.

[0018] In some implementations, interactive visual elements on a webpage are those with listening capabilities or those that support event consumption.

[0019] Secondly, this application provides a webpage node hit method, which can be applied to electronic devices. In this method, the operating system listens for webpage consumption events of a web application process. The webpage includes at least one visual element. In response to the consumption event, a sub-thread of the web application process performs a hit test on nodes in at least one visual layer corresponding to the webpage, obtaining a first test result. Each visual layer includes at least one node, and each node in each visual layer corresponds one-to-one with a visual element in the webpage. The sub-thread performs a hit test on nodes in at least one event layer corresponding to the webpage, obtaining a second test result. Each event layer includes one node, and each node in each event layer corresponds one-to-one with an interactive visual element in the webpage. When the first test result indicates that the consumption event hits the first node in the first visual layer and the second test result indicates that the consumption event hits the first node in the first event layer, the sub-thread determines that the consumption event hits the first node.

[0020] In this method, the web application process of the electronic device may include a sub-thread that responds to consuming events. By combining the hit test results of the visual layer and the event layer, the correct node hit effect can be achieved, thereby improving the node hit efficiency and thus improving the event response performance. At the same time, the node hit test being performed by the sub-thread can reduce the load pressure on the main thread, thereby reducing the probability of page lag.

[0021] In some implementations, a sub-thread of the web application process performs a hit test on at least one node in the visual layer corresponding to the webpage to obtain a first test result, including: the sub-thread comparing the location information corresponding to the consumed event with the location information of at least one node in the visual layer; when the location corresponding to the consumed event is within the area of ​​the first node in the first visual layer, the first test result indicates that the consumed event hits the first node in the first visual layer.

[0022] In some implementations, the child thread performs a hit test on at least one node in the event layer corresponding to the webpage to obtain a second test result, including: the child thread compares the position information corresponding to the consumed event with the position information of the node in at least one event layer; when the position corresponding to the consumed event is within the area of ​​the first node in the first event layer, the second test result indicates that the consumed event hits the first node in the first event layer.

[0023] In some implementations, the method further includes: when the first test result indicates that the consumed event hits the first node in the first visible layer and the second test result indicates that the consumed event hits any node in at least one event layer other than the first node, the main thread of the web application process performs a hit test according to the DOM tree hit strategy to determine the node hit by the consumed event; wherein, the DOM tree hit strategy is to perform a hit test on each node in the DOM tree of the web page.

[0024] In some implementations, the method further includes: when the first test result indicates that the consumed event hits the first node in the visible layer and the second test result indicates that the consumed event does not hit any node, a hit test is performed according to the DOM tree hit strategy, and the main thread of the web application process determines the node hit by the consumed event; wherein, the DOM tree hit strategy is to perform a hit test on each node in the DOM tree of the webpage.

[0025] In this way, when the child thread cannot determine the node that the consumed event hits, the main thread can re-perform a hit test on each node in the DOM tree of the webpage to determine the node that the consumed event hits, thus ensuring the accuracy of node hits.

[0026] In some implementations, the main thread and the child thread are two separate threads within the web application process. This approach distributes different stages of the node hit test across different threads, thereby balancing the load and improving node hit efficiency.

[0027] In some implementations, before the operating system listens for web application process events, the method further includes: a child thread creating at least one visual layer and at least one event layer corresponding to the webpage based on the webpage's Document Object Model (DOM) tree. This allows the child thread to create the visual and event layers based on the DOM tree, enabling the classification of nodes corresponding to each element on the webpage. This allows for hit testing based on both the visual and event layers, ensuring high accuracy in node matching.

[0028] Thirdly, this application provides an electronic device comprising multiple functional modules; the multiple functional modules interact to implement the methods performed by the electronic device in any of the above aspects and their respective embodiments. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on specific implementations.

[0029] Fourthly, this application provides an electronic device including at least one processor and at least one memory, wherein the at least one memory stores computer program instructions, and when the electronic device is running, the at least one processor executes any of the above aspects and the methods executed by the electronic device in its various embodiments.

[0030] Fifthly, this application also provides a computer program product containing instructions that, when the computer program product is run on a computer, cause the computer to perform the method executed by the electronic device in any of the above aspects and embodiments.

[0031] Sixthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a computer, causes the computer to perform the method executed by the electronic device in any of the above aspects and embodiments.

[0032] In a seventh aspect, this application also provides a chip for reading a computer program stored in a memory and executing the method executed by the electronic device in any of the above aspects and embodiments.

[0033] Eighthly, this application also provides a chip system including a processor for supporting a computer device in implementing the methods executed by electronic devices in any of the above aspects and their embodiments. In one possible design, the chip system further includes a memory for storing programs and data necessary for the computer device. The chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0034] Figure 1 is a schematic diagram of a native component provided in an embodiment of this application;

[0035] Figure 2 is a schematic diagram of a same-layer rendering technology provided in an embodiment of this application;

[0036] Figure 3 is a schematic diagram of a DOM tree structure provided in an embodiment of this application;

[0037] Figure 4 is a schematic diagram of a method for performing hit testing on visible layer nodes;

[0038] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0039] Figure 6 is a software structure block diagram of an electronic device provided in an embodiment of this application;

[0040] Figure 7 is a schematic diagram of a webpage node hitting method provided in an embodiment of this application;

[0041] Figure 8 is a schematic diagram of a webpage node hitting method provided in an embodiment of this application;

[0042] Figure 9 is a schematic diagram of a webpage node hitting method and a hit testing method based on a DOM tree hitting strategy provided in an embodiment of this application;

[0043] Figure 10 is a flowchart of a webpage node hitting method provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0045] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0046] With the widespread application of mini-programs, the functions and services they provide are becoming increasingly rich. The pages displayed by mini-programs are web pages, which can include various elements such as text, images, and controls. One type of component in a web page is the native component. Native components differ from built-in components rendered by a WebView; native components are rendered by the operating system of the electronic device. For example, Figure 1 is a schematic diagram of a native component provided in an embodiment of this application. Referring to Figure 1(a), in a travel mini-program, the map control on the page is a native component; referring to Figure 1(b), in a mobile phone recharge mini-program, the control for entering a phone number on the page is a native component.

[0047] Because native components and built-in components rendered by WebView are at different levels, and the native components are at a higher level, they cannot overlap with the built-in components rendered by WebView, which may cause display errors or user interaction errors. To solve the limitations of native components, same-level rendering technology was introduced. In same-level rendering technology, native components can be rendered directly onto the WebView level, and native components can be directly mounted onto WebView nodes. This allows native components to be used like non-native components, ensuring that the feel of native components and elements on the web page is consistent.

[0048] The main functions of same-layer rendering technology include: drawing native components onto the same-layer area within a web page, and forwarding events from the same-layer area to the native components. For example, Figure 2 is a schematic diagram of a same-layer rendering technology provided in an embodiment of this application. Referring to Figure 2, taking a web page displaying a video as an example, the web page may include a page background, a video component, and a bullet screen component, where the video component is a native component. Based on same-layer rendering technology, when rendering a web page, an electronic device can render the page background, video component, and bullet screen component on the same layer, and different components can form a correct occlusion relationship. For example, the video component can occlude part of the page background, and the bullet screen component can occlude part of the video component. In the page rendered based on same-layer rendering technology, the area where the native components are located can be called the same-layer area. When the web page detects that the user has triggered an operation in the same-layer area, it can send the event corresponding to the user's triggered operation to the native component, so that the native component can respond to the user's operation.

[0049] When a web page rendered using same-layer rendering technology detects a user action, if the user action is applied to a native component, and the web page determines that the event has hit the same-layer area, the web page needs to forward the event to the native component. The following introduces two hit testing methods.

[0050] In some implementations, web pages can perform hit testing using a Document Object Model (DOM) tree hit strategy. First, the DOM tree involved in this application's embodiments is introduced. The DOM tree is an important data structure used to represent and render web pages. For example, the DOM tree is a tree structure generated after parsing the HTML document of a web page. The DOM tree represents the HTML document as a tree, and each node in the tree represents a part of the document, such as tags, attributes, or text. The DOM tree can include various types of DOM nodes, such as document nodes, element nodes, attribute nodes, text nodes, and comment nodes. For example, Figure 3 is a schematic diagram of a DOM tree structure provided in an embodiment of this application. Referring to Figure 3, the root node of the DOM tree is usually a document node, as shown in Figure 3. Element nodes are used to represent HTML tags, such as <html>, ... 、 Each element node may have child nodes, as shown in Figure 3, where the child nodes of the node are... <meta> Nodes and <title>node,< / title> The child nodes of the node are Nodes and node, The child nodes of the node are Nodes. Attribute nodes represent the attributes of element nodes, such as class and id. Attribute nodes are usually considered part of an element node and are not displayed as independent nodes in the DOM tree. Text nodes represent the text content within HTML tags. Text nodes are leaf nodes and have no child nodes. When performing hit testing based on a DOM tree hit strategy, electronic devices traverse all DOM nodes in the DOM tree to find the consumption event node corresponding to the user-triggered action. While this method can correctly hit nodes, it requires hitting all DOM nodes, which is time-consuming. Furthermore, the hit testing process runs on the main web thread. When the main thread is heavily loaded, hit testing may cause page lag and impact performance.

[0051] In other implementations, electronic devices can assemble visible nodes from the DOM tree into a visible layer and perform a hit test on all visible nodes in the visible layer, where visible nodes correspond to visible elements in the web page. In this implementation, the hit test on visible nodes in the visible layer can be performed in a thread other than the main web thread, reducing the probability of lag. However, this method can only perform hit tests on visible nodes, while a web page may include some consumer-enabled but invisible nodes. When an event hits such a node, the hit test method using visible nodes in the visible layer may result in incorrect node detection. For example, Figure 4 is a schematic diagram of a method for performing hit tests on visible layer nodes. Referring to Figure 4, taking a DOM tree including three nodes—div1, div2, and Embed—as an example, div1 and div2 are placeholder elements and not visible nodes. Therefore, the visible layer generated based on the DOM tree includes the Embed node. When the event corresponding to the user's action hits the div2 node, a hit test is performed on the visible nodes in the visual layer. The electronic device determines that the event corresponding to the user's action hits the Embed node, but the actual event corresponding to the user's action hits the div2 node, thus failing to hit the node correctly.

[0052] To address the aforementioned problems, this application provides a webpage node matching method to improve the efficiency and accuracy of node matching. This application can be executed by an electronic device. In this method, the electronic device creates at least one visual layer and at least one event layer based on the elements contained in the webpage. Each visual layer includes at least one visual node, and each node in the visual layer corresponds one-to-one with a visual element in the webpage. Each event layer includes one node, and each node in the event layer corresponds one-to-one with an interactive visual element in the webpage. When the electronic device displays the webpage, it detects a consumption event on the webpage. The electronic device performs a match test on the nodes in the at least one visual layer to obtain a first test result; the electronic device performs a match test on the nodes in the at least one event layer to obtain a second test result; when the first test result indicates that the consumption event matches a first node in the first visual layer and the second test result indicates that the consumption event matches a first node in the first event layer, it is determined that the consumption event matches the first node. In this way, electronic devices can create visual and event layers based on the elements of a webpage. By combining the hit test results of the visual and event layers, the correct element hit can be achieved, improving the efficiency of node hits, thereby enhancing event response performance and reducing lag.

[0053] The following describes an electronic device and embodiments for using such an electronic device. The electronic device in this application embodiment can be a tablet computer, mobile phone, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), wearable device, etc. This application embodiment does not impose any limitations on the specific type of electronic device.

[0054] In some embodiments of this application, the electronic device may also be a portable terminal device that includes other functions such as a personal digital assistant and / or a music player. Exemplary embodiments of the portable terminal device include, but are not limited to, devices equipped with... Or portable terminal devices with other operating systems.

[0055] Figure 5 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application. As shown in Figure 5, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0056] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of the electronic device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. Processor 110 may also include memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that processor 110 has recently used or is repeatedly used. If processor 110 needs to reuse an instruction or data, it can directly retrieve it from the memory. This avoids repeated access, reduces the waiting time of processor 110, and thus improves system efficiency.

[0057] The display screen 194 is used to display the display interface of an application, such as the display page of an application installed on the electronic device 100. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0058] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and software code for at least one application program. The data storage area may store data generated during the use of electronic device 100 (e.g., captured images, recorded videos, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0059] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, images, videos, and other files can be saved on the external memory card.

[0060] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0061] The sensor module 180 may include a pressure sensor 180A, an acceleration sensor 180B, a touch sensor 180C, etc.

[0062] Touch sensor 180C, also known as a "touch panel," can be located on display screen 194. The touch sensor 180C and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180C detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180C may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0063] It is understood that the components shown in Figure 5 do not constitute a specific limitation on the electronic device 100. The electronic device may include more or fewer components than shown, or combine some components, or separate some components, or have different component arrangements. Furthermore, the combination / connection relationships between the components in Figure 5 can also be adjusted and modified.

[0064] Figure 6 is a software structure block diagram of an electronic device provided in an embodiment of this application. As shown in Figure 6, the software structure of the electronic device can be a layered architecture. For example, the software can be divided into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into four layers, from top to bottom: the application layer, the application framework layer (framework, FWK), the runtime and system libraries, and the kernel layer.

[0065] The application layer can include a series of application packages. As shown in Figure 6, the application layer can include camera, settings, skin modules, user interface (UI), third-party applications, etc. Third-party applications can include gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, SMS, etc.

[0066] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer can include some predefined functions. As shown in Figure 6, the application framework layer can include a window manager, content provider, view system, phone manager, resource manager, and notification manager.

[0067] The window manager is used to manage windowed applications. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture screenshots. The content provider stores and retrieves data, making this data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0068] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images. In this embodiment, when a web page includes a visual control, the visual control can be considered a visual element constituting the web page. The visual control corresponds to a visual node in the DOM tree. After the electronic device creates at least one visual layer and at least one event layer based on the DOM tree, the node corresponding to the visual control belongs to one of the at least one visual layers. If the visual control supports user interaction, the node corresponding to the visual control also constitutes an event layer.

[0069] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0070] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0071] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0072] The runtime includes the core libraries and the virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0073] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), image processing libraries, etc.

[0074] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0075] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, and AMR.

[0076] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0077] A 2D graphics engine is a graphics engine for 2D drawing.

[0078] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0079] The hardware layer can include various types of sensors, such as accelerometers, gyroscopes, and touch sensors.

[0080] It should be noted that the structures shown in Figures 5 and 6 are merely examples of electronic devices provided in the embodiments of this application, and cannot be used to limit the electronic devices provided in the embodiments of this application. In specific implementations, electronic devices may have more or fewer devices or modules than those shown in Figures 5 or 6.

[0081] When an electronic device renders a webpage, if the webpage includes native components, the electronic device can render the native components and other elements on the webpage to the same level based on same-layer rendering technology. When a user triggers an operation on the webpage or other consumption events exist on the webpage, the webpage can forward the consumption event to the native component when it determines that the consumption event hits the same-layer area, so that the native component can respond to the consumption event. The node hit method provided in this application embodiment can be used to determine whether the consumption event hits the same-layer area. The webpage node hit method provided in this application embodiment will be further described below.

[0082] In this embodiment, the web page displayed by the electronic device includes at least one element, which can be a visual element. For example, the web page may include text, images, input boxes, buttons, etc. The at least one visual element in the web page may include interactive visual elements, such as input boxes. Each element in the web page corresponds one-to-one with a node in the DOM tree. The electronic device can parse at least one visual layer and at least one event layer from the DOM tree of the web page. Each visual layer may include at least one node, and each node in the visual layer corresponds one-to-one with a visual element in the web page. A visual element refers to an element visible to the user in the web page, such as text, images, etc. Optionally, for the same layer area where a native component is located, an Embed node corresponding to one layer area constitutes a visual layer. Each event layer obtained from the electronic device's parsing includes one node. Each node in the event layer corresponds one-to-one with an interactive visual element on the web page. Interactive visual elements are elements on the web page that have listening capabilities or support event consumption, such as input boxes and buttons. Event consumption can include click or drag events from the user on interactive elements on the web page. In some examples, when an element on the web page is visible and also supports user interaction, the DOM node in the corresponding DOM tree belongs to both a visual layer and an event layer. For instance, when a native component on the web page is visible to the user, the DOM node corresponding to that native component belongs to a visual layer; and if the native component supports user interaction, then the DOM node corresponding to that native component also constitutes an event layer.

[0083] In some embodiments, the electronic device detects a consumption event occurring on the web interface. This consumption event could be a click or drag event by a user on an interactive element within the web interface. The electronic device needs to determine the node to which the consumption event occurred. The electronic device can perform a hit test on nodes in at least one visible layer to obtain a first test result. As described above, each visible layer includes at least one node. The electronic device can compare the location information corresponding to the consumption event with the location information of nodes in at least one visible layer. When the location corresponding to the consumption event is within the area of ​​the first node, the first test result indicates that the consumption event has hit the first node in the visible layer.

[0084] Optionally, after obtaining the first test result, the electronic device can perform a hit test on at least one event layer to obtain a second test result. As described above, each interactive element in a web page corresponds to an event layer, so the electronic device can perform a hit test on nodes in at least one event layer. For example, the electronic device can compare the location information corresponding to a consumption event with the location information of nodes in at least one event layer. When the location corresponding to the consumption event is within the area of ​​the first node in the first event layer, the second test result indicates that the consumption event hits the first node in the first event layer.

[0085] In this embodiment of the application, after obtaining the first test result and the second test result, when the first test result indicates that the consumption event hits the first node in the visual layer and the second test result indicates that the consumption event hits the first node in the first event layer, the electronic device can determine that the consumption event hits the first node.

[0086] For example, Figure 7 is a schematic diagram of a webpage node hitting method provided in an embodiment of this application. Referring to Figure 7, taking a webpage DOM tree including three nodes: div1, div2, and Embed nodes at the same level as an example, the electronic device parses the DOM tree to obtain a visible layer, event layer 1, and event layer 2. The visible layer includes Embed nodes, event layer 1 includes div2 nodes, and event layer 2 includes Embed nodes. When the electronic device performs a hit test on a consumption event, it first compares the position information of the consumption event with the position information of the Embed nodes in the visible layer. If the position information of the consumption event matches the position information of the Embed nodes in the visible layer, the electronic device obtains a first test result indicating that the consumption event hits an Embed node in the visible layer. The electronic device then compares the position information of the consumption event with the position information of the div2 nodes in event layer 1, and with the position information of the Embed nodes in event layer 2. If the position information of the consumption event matches the position information of the Embed nodes in event layer 2, the electronic device obtains a second test result indicating that the consumption event hits an Embed node in event layer 2. The electronic device can determine that the consumption event hits the Embed node based on the results of the first and second tests.

[0087] Optionally, the first node can be a native component rendered to the Web page by an electronic device using same-layer rendering technology. The area where this native component is located in the Web page can be called the same-layer area. Therefore, a consumption event hitting the native component can also be called a consumption event hitting the same-layer area. The Web component can forward the consumption event to the native component, and the native component can respond to the consumption event. For example, when the Web page is the travel mini-program page shown in Figure 1(a), the native component is the map component in the mini-program page. The area where this map component is located can be called the same-layer area. Assuming the consumption event is a user-triggered swipe operation, the Web page determines that the consumption event hits the same-layer area based on the node-hitting method provided in the above embodiments of this application. The Web page can forward the consumption event to the native component, and the native component can adjust the displayed map according to the user's swipe operation.

[0088] In some embodiments, if the electronic device determines that the first test result is that the consumption event hits the first node, and the second test result is that the consumption event hits other nodes besides the first node, or the consumption event does not hit any node, then the electronic device needs to re-perform the hit test according to the DOM tree hit strategy. That is, the electronic device traverses each node in the DOM tree of the Web page to perform the hit test in order to determine the node hit by the consumption event.

[0089] For example, Figure 8 is a schematic diagram of a webpage node hitting method provided in an embodiment of this application. Referring to Figure 8, taking a webpage DOM tree including three nodes: div1, div2, and Embed nodes at the same level as an example, the electronic device parses the DOM tree to obtain a visible layer, event layer 1, and event layer 2. The visible layer includes Embed nodes, event layer 1 includes div2 nodes, and event layer 2 includes Embed nodes. When the electronic device performs a hit test on a consumption event, it first compares the position information of the consumption event with the position information of the Embed nodes in the visible layer. If the position information of the consumption event matches the position information of the Embed nodes in the visible layer, the electronic device obtains a first test result indicating that the consumption event hits an Embed node in the visible layer. The electronic device then compares the location information of the consumption event with the location information of the div2 node in event layer 1, and with the location information of the Embed node in event layer 2. If the location information of the consumption event matches the location information of the div2 node in event layer 1, the electronic device obtains a second test result indicating that the consumption event hits the div2 node in event layer 1. Since the first test result and the second test result are inconsistent, the electronic device needs to re-perform the hit test on each node in the DOM tree of the web page to determine the node hit by the consumption event.

[0090] Optionally, in the node hit method provided in this application embodiment, the web application process may include a main thread layer and sub-threads. The sub-threads may be composition threads. The hit test performed by the electronic device on the visual layer and at least one event layer can be executed in the composition thread, thereby reducing the load pressure on the main thread and thus reducing the probability of page lag. For example, the composition thread may be the Chromium Compositor thread.

[0091] For example, Figure 9 is a schematic diagram of a webpage node hit method and a hit test method based on the DOM tree hit strategy provided in an embodiment of this application. Referring to (a) in Figure 9, when performing node hit test based on the DOM tree hit strategy, the Web main thread creates a DOM tree based on the elements in the Web page. When the synthesis thread detects a consumption event, it forwards the consumption event to the main thread. The main thread traverses all nodes in the DOM tree based on the DOM tree hit strategy to perform a hit test. After obtaining the event hit result, the main thread forwards the event hit result to the synthesis thread. The synthesis thread then processes the event hit result. For example, when the event hits the same layer area, the synthesis thread forwards the touch event to the native component. Referring to Figure 9(b), in the webpage node hit method provided in this embodiment, the main web thread creates a DOM tree based on the elements in the web page, and the synthesis thread creates a visual layer and an event layer based on the DOM tree. When the synthesis thread detects a consumed event, it performs hit tests in both the visual layer and the event layer. After obtaining the event hit result, the synthesis thread processes it based on the result. For example, if the event hits the same area, the synthesis thread forwards the consumed event to the native component. If the synthesis thread cannot find a hit result, it forwards the consumed event to the main thread, which then performs node hit tests based on the DOM tree hit strategy. The node hit method provided in this embodiment allows the synthesis thread to traverse the nodes in the visual layer beforehand and then traverse the nodes in the event layer to obtain the event hit result. This reduces the number of hit tests performed by the main thread based on traversing all nodes in the DOM tree, ensuring hit accuracy while improving node hit efficiency and reducing response lag after user gestures.

[0092] Based on the same concept, this application also provides a webpage node hitting method, which can be executed by an electronic device. Figure 10 is a flowchart of a webpage node hitting method provided by an embodiment of this application. Referring to Figure 10, the method includes the following steps:

[0093] S1001: Electronic device displays webpage.

[0094] A webpage includes at least one visual element, and this visual element may include interactive elements. For example, the visual elements in a webpage can be text, images, input boxes, buttons, etc., and the interactive elements can be input boxes, buttons, etc.

[0095] S1002: The electronic device detected a consumption event of a webpage.

[0096] For example, consumption events include click events or drag events of users on interactive elements on a webpage.

[0097] S1003: The electronic device performs a hit test on at least one node in the visual layer corresponding to the webpage and obtains a first test result.

[0098] Each of the at least one visual layer includes at least one node, and each node in the visual layer corresponds one-to-one with a visible element in the webpage.

[0099] S1004: The electronic device performs a hit test on at least one node in the event layer corresponding to the webpage and obtains a second test result.

[0100] Each event layer includes at least one node, and each node in the event layer corresponds one-to-one with an interactive element in the webpage.

[0101] S1005: When the first test result indicates that the consumption event hits the first node in the first visible layer and the second test result indicates that the consumption event hits the first node in the first event layer, the electronic device determines that the consumption event hits the first node.

[0102] It should be noted that the webpage node hitting method shown in Figure 10 of this application can be referred to the above embodiments of this application in specific implementation, and repeated parts will not be described again.

[0103] Based on the above embodiments, this application also provides an electronic device, which includes multiple functional modules; the multiple functional modules interact to realize the functions performed by the electronic device in the various methods described in the embodiments of this application. The steps performed by the electronic device in the embodiment shown in FIG10 are executed as follows. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on specific implementations.

[0104] Based on the above embodiments, this application also provides an electronic device, which includes at least one processor and at least one memory, wherein the at least one memory stores computer program instructions. When the electronic device is running, the at least one processor performs the functions performed by the electronic device in the various methods described in the embodiments of this application. The steps performed by the electronic device in the embodiment shown in FIG10 are executed as follows.

[0105] Based on the above embodiments, this application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the methods described in the embodiments of this application.

[0106] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the methods described in the embodiments of this application.

[0107] Based on the above embodiments, this application also provides a chip for reading computer programs stored in a memory to implement the methods described in the embodiments of this application.

[0108] Based on the above embodiments, this application provides a chip system including a processor for supporting a computer device in implementing the methods described in the embodiments of this application. In one possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. This chip system may be composed of chips or may include chips and other discrete devices.

[0109] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0111] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0112] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0113] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for webpage node detection, characterized in that, Applied to electronic devices, the method includes: Display a webpage, the webpage including at least one visual element; A consumption event on the webpage was detected; A hit test is performed on the nodes in at least one visual layer corresponding to the webpage to obtain a first test result; wherein, each visual layer includes at least one node, and the nodes in each visual layer correspond one-to-one with the visual elements in the webpage; A hit test is performed on the nodes in at least one event layer corresponding to the webpage to obtain a second test result; wherein, each event layer in the at least one event layer includes a node, and the node in each event layer corresponds one-to-one with the interactive visual element in the webpage; When the first test result indicates that the consumption event hits the first node in the first visual layer and the second test result indicates that the consumption event hits the first node in the first event layer, it is determined that the consumption event hits the first node.

2. The method as described in claim 1, characterized in that, The method further includes: Create at least one visual layer and at least one event layer corresponding to the webpage based on the document object model (DOM) tree of the webpage.

3. The method as described in claim 1 or 2, characterized in that, The step of performing a hit test on at least one node in the visual layer corresponding to the webpage to obtain a first test result includes: Compare the location information corresponding to the consumption event with the location information of the nodes in at least one visual layer; When the location corresponding to the consumption event is within the area of ​​the first node in the first visual layer, the first test result indicates that the consumption event hits the first node in the first visual layer.

4. The method according to any one of claims 1-3, characterized in that, The step of performing a hit test on at least one node in the event layer corresponding to the webpage to obtain a second test result includes: Compare the location information corresponding to the consumption event with the location information of the nodes in the at least one event layer; When the location corresponding to the consumption event is within the area of ​​the first node in the first event layer, the second test result indicates that the consumption event hits the first node in the first event layer.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: When the first test result indicates that the consumption event hits the first node in the first visible layer and the second test result indicates that the consumption event hits any node in the at least one event layer other than the first node, a hit test is performed according to the DOM tree hit strategy to determine the node hit by the consumption event; wherein, the DOM tree hit strategy is to perform a hit test on each node in the DOM tree of the webpage.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: When the first test result indicates that the consumption event hits the first node in the visual layer and the second test result indicates that the consumption event does not hit any node, a hit test is performed according to the DOM tree hit strategy to determine the node hit by the consumption event; wherein, the DOM tree hit strategy is to perform a hit test on each node in the DOM tree of the webpage.

7. The method according to any one of claims 1-6, characterized in that, The visual element corresponding to the first node is a native component rendered to the webpage based on same-layer rendering technology.

8. The method according to any one of claims 1-4, characterized in that, The webpage node hit method is applied to the synthesis thread of the electronic device.

9. The method according to any one of claims 1-8, characterized in that, The consumption events include click events or drag events of users on interactive elements in the webpage.

10. The method according to any one of claims 1-9, characterized in that, The interactive visual elements in the webpage are those that have listening functions or support event consumption.

11. A method for webpage node detection, characterized in that, Applied to electronic devices, the method includes: The operating system listens for web page consumption events of the web application process, the web page including at least one visual element; In response to the consumption event, a sub-thread of the web application process performs a hit test on nodes in at least one visual layer corresponding to the webpage and obtains a first test result; wherein, each visual layer includes at least one node, and each node in the visual layer corresponds one-to-one with a visual element in the webpage; The sub-thread performs a hit test on the nodes in at least one event layer corresponding to the webpage to obtain a second test result; wherein, each event layer in the at least one event layer includes a node, and the node in each event layer corresponds one-to-one with the interactive visual elements in the webpage; When the first test result indicates that the consumption event hits the first node in the first visual layer and the second test result indicates that the consumption event hits the first node in the first event layer, the sub-thread determines that the consumption event hits the first node.

12. The method as described in claim 11, characterized in that, The child thread of the web application process performs a hit test on at least one node in the visual layer corresponding to the webpage to obtain a first test result, including: The sub-thread compares the location information corresponding to the consumed event with the location information of the nodes in at least one visual layer; When the location corresponding to the consumption event is within the area of ​​the first node in the first visual layer, the first test result indicates that the consumption event hits the first node in the first visual layer.

13. The method as described in claim 11 or 12, characterized in that, The sub-thread performs a hit test on at least one node in the event layer corresponding to the webpage to obtain a second test result, including: The sub-thread compares the location information corresponding to the consumed event with the location information of the nodes in at least one event layer; When the location corresponding to the consumption event is within the area of ​​the first node in the first event layer, the second test result indicates that the consumption event hits the first node in the first event layer.

14. The method according to any one of claims 11-13, characterized in that, The method further includes: When the first test result indicates that the consumption event hits the first node in the first visual layer and the second test result indicates that the consumption event hits any node in the at least one event layer other than the first node, the main thread of the web application process performs a hit test according to the DOM tree hit strategy to determine the node hit by the consumption event; wherein, the DOM tree hit strategy is to perform a hit test on each node in the DOM tree of the webpage.

15. The method according to any one of claims 11-13, characterized in that, The method further includes: When the first test result indicates that the consumption event hits the first node in the visual layer and the second test result indicates that the consumption event does not hit any node, a hit test is performed according to the DOM tree hit strategy, and the main thread of the web application process determines the node hit by the consumption event; wherein, the DOM tree hit strategy is to perform a hit test on each node in the DOM tree of the webpage.

16. The method as described in claim 14 or 15, characterized in that, The main thread and the child thread are two threads of the web application process.

17. The method according to any one of claims 11-16, characterized in that, Before the operating system detects the web application process's webpage consumption event, the method further includes: The sub-thread creates at least one visual layer and at least one event layer corresponding to the webpage based on the webpage's Document Object Model (DOM) tree.

18. An electronic device, characterized in that, The method includes at least one processor coupled to at least one memory, the at least one processor being configured to read a computer program stored in the at least one memory to perform the method as described in any one of claims 1-10, or to perform the method as described in any one of claims 11-17.

19. A computer program product containing instructions, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-10, or the method as described in any one of claims 11-17.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-10, or the method as described in any one of claims 11-17.

Images