Page element processing method and device, storage medium and electronic equipment
By using breadth-first search in the virtual DOM tree and combining the depth residual model based on attention mechanism to classify front-end component types, the problem of low detection efficiency of dynamic elements under the complex structure of the virtual DOM tree is solved, and the update efficiency of page elements is significantly improved .
Patent Information
- Application Number
- CN202410989352.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
AI Technical Summary
When the virtual DOM tree structure is complex, the data search comparison between the old and new virtual DOM trees takes a long time, resulting in low efficiency in detecting dynamic elements, affecting the update efficiency of page elements.
By obtaining the changing position of the page, cropping the target picture, classifying using the depth residual model based on the attention mechanism, obtaining the front-end component type, and performing breadth-first searching according to the preset search depth, detecting the dynamic elements in the virtual DOM tree DOM node.
Improve the efficiency of detecting dynamic elements corresponding to DOM nodes in the virtual DOM tree, and improve the efficiency of updating page elements.
Smart Images

Figure CN118939900A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method, device, storage medium and electronic device for processing page elements. Background Art
[0002] During the user's interaction with a World Wide Web (WEB) page (hereinafter referred to as a page), there may be elements in the page that do not exist at the time of initialization, but will only appear after the user performs some kind of interaction. Therefore, it is necessary to determine the DOM node corresponding to the dynamically changing page element (hereinafter referred to as a dynamic element) in the virtual Document Object Model (DOM) tree, so as to facilitate the corresponding page element configuration update.
[0003] Currently, when the virtual DOM tree changes, related technologies can compare the differences between the new and old virtual DOM trees through a diff algorithm, and then detect the DOM nodes corresponding to the dynamic elements in the virtual DOM tree.
[0004] However, if the structure of the virtual DOM tree is relatively complex, the relevant technology needs to perform more data search and comparison on the complex new and old virtual DOM trees, which will take a long time and will affect the efficiency of detecting the corresponding DOM nodes of dynamic elements in the virtual DOM tree, thereby affecting the update efficiency of page elements. Summary of the invention
[0005] In view of this, the present application provides a method, device, storage medium and electronic device for processing page elements, the main purpose of which is to improve the current technical problem that it takes a long time to search and compare data between the new and old virtual DOM trees, resulting in low efficiency in detecting the corresponding DOM nodes of dynamic elements in the virtual DOM tree.
[0006] In a first aspect, the present application provides a method for processing page elements, comprising: obtaining a position where a change occurs in a page; cropping an image corresponding to the page according to the position to obtain a target image corresponding to the position; classifying the target image and obtaining a front-end component type corresponding to the classification of the target image; performing a breadth-first search in a virtual DOM tree of the page according to a preset search depth corresponding to the front-end component type to detect DOM nodes corresponding to dynamic elements of the page in the virtual DOM tree, wherein the search depth is the number of edges passed from the root node of the virtual DOM tree to the target node.
[0007] Optionally, obtaining the location where the change occurs in the page includes: obtaining pixel frames before and after the page changes; and analyzing the pixel frames using a canvas to obtain the location where the change occurs in the page.
[0008] Optionally, a canvas is used to analyze the pixel frame to obtain a location where changes occur in the page, including: drawing pixel information of the page onto a canvas according to the pixel frame; scanning the pixel information drawn before and after the page changes on the canvas to obtain coordinate information of a target area, the target area being an area in the canvas where the pixel information changes; and determining the location where the changes occur in the page based on the coordinate information.
[0009] Optionally, classifying the target image includes: identifying the target image through an image classification model to obtain a classification of the target image, wherein the image classification model is a deep residual model based on an attention mechanism, the deep residual model includes a Resnet50 network, and the Resnet50 network includes multiple attention modules.
[0010] Optionally, a breadth-first search is performed in the virtual DOM tree of the page according to a preset search depth corresponding to the front-end component type, including: searching and comparing the same branches in the virtual DOM tree before and after the page changes according to the recommended search depth to obtain difference nodes in the virtual DOM tree before and after the page changes; and determining, based on the difference nodes, a DOM node corresponding to the dynamic element in the virtual DOM tree.
[0011] Optionally, element configuration information corresponding to the DOM node is loaded, and updated and mapped to the real DOM tree of the page.
[0012] In a second aspect, the present application provides a device for processing page elements, including:
[0013] An acquisition module, configured to acquire a location where a change occurs in a page;
[0014] A cropping module is configured to crop the image corresponding to the page according to the position to obtain a target image corresponding to the position;
[0015] A classification module, configured to classify the target image and obtain a front-end component type corresponding to the classification of the target image;
[0016] The search module is configured to perform a breadth-first search in the virtual DOM tree of the page according to a preset search depth corresponding to the front-end component type to detect DOM nodes corresponding to dynamic elements of the page in the virtual DOM tree, wherein the search depth is the number of edges passed from the root node of the virtual DOM tree to the target node.
[0017] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0018] In a fourth aspect, the present application provides an electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.
[0019] In a fifth aspect, the present application provides a computer program product having a computer program stored thereon, wherein the computer program implements the method described in the first aspect when executed by a processor.
[0020] By means of the above technical scheme, the processing method, device, storage medium and electronic device of the page element provided by the present application first obtain the position where the change occurs in the page; crop the picture corresponding to the page according to the position to obtain the target picture corresponding to the position; then classify the target picture and obtain the front-end component type corresponding to the classification of the target picture; then perform a breadth-first search in the virtual DOM tree of the page according to the preset search depth corresponding to the front-end component type to detect the DOM node corresponding to the dynamic element of the page in the virtual DOM tree, and the search depth is the number of edges passed from the root node of the virtual DOM tree to the target node. Compared with the current prior art, the present application improves the efficiency of detecting the DOM node corresponding to the dynamic element in the virtual DOM tree by finding the front-end component type corresponding to the dynamic element in the page, and adopts the preset search depth corresponding to the front-end component type to perform a breadth-first search in the virtual DOM tree, thereby improving the updating efficiency of the page elements.
[0021] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0024] Figure 1A schematic diagram showing a flow chart of a method for processing page elements provided in an embodiment of the present application;
[0025] Figure 2 A schematic diagram showing a flow chart of a method for processing page elements provided in an embodiment of the present application;
[0026] Figure 3 A schematic diagram of an example process provided by an embodiment of the present application is shown;
[0027] Figure 4 A schematic diagram of an example process provided by an embodiment of the present application is shown;
[0028] Figure 5 A schematic diagram of an example process provided by an embodiment of the present application is shown;
[0029] Figure 6 A schematic diagram of an example process provided by an embodiment of the present application is shown;
[0030] Figure 7 A schematic diagram of the structure of a device for processing page elements provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0031] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.
[0032] In order to improve the current technical problem that the time required for data search and comparison between the new and old virtual DOM trees is long, resulting in low efficiency in detecting the corresponding DOM nodes of dynamic elements in the virtual DOM tree. This embodiment provides a method for processing page elements, such as Figure 1 As shown, the method includes:
[0033] Step 101: Get the position where the change occurs in the page.
[0034] The page in this embodiment may be a WEB page, such as a user interacting with the WEB page through a browser. Exemplarily, the location where the change occurs may be determined by comparing the difference between pages at different times.
[0035] Step 102: crop the image corresponding to the page according to the position where the change occurs in the page to obtain a target image corresponding to the position of the change.
[0036] For example, a circular image with a radius of r can be cropped from the image corresponding to the page with the position where the change occurs as the center point by using a cropping tool as the target image, or a square image of a fixed size (such as 280px*280px) can be cropped as the target image. This embodiment does not specifically limit the shape of the target image.
[0037] Step 103: classify the target image and obtain the front-end component type corresponding to the classification of the target image.
[0038] Exemplarily, the front-end component is the basic unit of the user interface, which encapsulates specific UI elements, behavior logic, and possible data management, allowing developers to build and reuse interfaces in a modular manner. For example, the front-end component types can be divided into layout components, data display components, form components, feedback components, navigation components, media components, interaction components, etc.
[0039] Step 104: perform a breadth-first search in the virtual DOM tree of the page according to the preset search depth corresponding to the front-end component type to detect the DOM nodes corresponding to the dynamic elements of the page in the virtual DOM tree.
[0040] The search depth is the number of edges from the root node of the virtual DOM tree to the target node. In this embodiment, the corresponding search depths can be pre-set (referred to as preset) for different front-end component types. For example, according to the depth of the DOM node corresponding to the dynamic element of the front-end component of type a in the virtual DOM tree, the search depth corresponding to the front-end component type a is set.
[0041] For example, the search depth of the root node in the virtual DOM tree is 0, the search depth of the child node of the root node is 1, the search depth of the child node of the child node (i.e., the grandchild node of the root node) is 2, and so on. The branch in the virtual DOM tree may have multiple nodes according to the actual situation. If the front-end component type A (the front-end component type obtained through steps 101 to 103) corresponds to a preset search depth of 2, then a breadth-first search can be performed in the root node, the child node, and the grandchild node in the virtual DOM tree of the page, because the dynamic elements of the front-end components of type A usually have a depth of 2 corresponding to the DOM node in the virtual DOM tree. This embodiment preferentially performs this shallow depth search and quickly searches for the DOM node corresponding to the dynamic element in the virtual DOM tree. There is no need to search all the way from one branch to the end until the next branch is backtracked and searched recursively until the boundary. The efficiency of detecting the corresponding DOM node of the dynamic element in the virtual DOM tree is improved.
[0042] In this embodiment, breadth-first search is an algorithm for traversing or searching a tree structure (such as a tree or a graph). It follows a "breadth-first" strategy, that is, it first visits nodes that are close to the starting point, and then gradually visits nodes that are far from the starting point. The main feature of breadth-first search is that it can ensure that among all the nodes visited, nodes that are the same distance from the starting point are always visited before nodes that are farther away.
[0043] The state changes of page elements are usually controlled by front-end engineers in the code using Cascading Style Sheets (CSS) and / or Javascript (JS). When the product is released to the online environment, the styles of some elements in the page need to be adjusted due to some unexpected reasons, or the same page needs to be connected to different projects through different style adjustments. Usually, the non-intrusive configuration task of the page change elements is completed in the form of configuration files or external interfaces.
[0044] In some examples, the virtual DOM tree can act as an intermediary layer to help developers update the user interface more efficiently. The real DOM tree is the document object model actually used by the browser. It is a tree-structured representation of a Hypertext Markup Language (HTML) document, which contains all elements, attributes, and content in the page. The virtual DOM is a lightweight memory data structure of the real DOM structure simulated by JavaScript. When a JavaScript framework (such as React, Vue.js, etc.) modifies the user interface, it does not directly operate the real DOM, but first creates and updates the virtual DOM tree in memory. When the virtual DOM tree changes, the JavaScript framework compares the differences between the new and old virtual DOM trees, finds a strategy to minimize operations, and then applies the necessary changes to the real DOM tree. This method significantly reduces unnecessary operations and improves the update efficiency and performance of the page.
[0045] Compared with the current existing technology, a more efficient page element processing method is provided by applying the technical solution of this embodiment. First, the position where the change occurs in the page is obtained; the picture corresponding to the page is cropped according to the position to obtain the target picture corresponding to the position; then the target picture is classified, and the front-end component type corresponding to the classification of the target picture is obtained; then according to the recommended search depth corresponding to the front-end component type, a breadth-first search is performed in the virtual DOM tree of the page to detect the DOM nodes corresponding to the dynamic elements of the page in the virtual DOM tree. By finding the front-end component type corresponding to the dynamic elements in the page, and using the recommended search depth corresponding to the front-end component type, a breadth-first search is performed in the virtual DOM tree, which improves the efficiency of detecting the DOM nodes corresponding to the dynamic elements in the virtual DOM tree, thereby improving the updating efficiency of the page elements.
[0046] In order to further illustrate the specific implementation process of the method of this embodiment, this embodiment provides the following Figure 2 The specific method shown includes:
[0047] Step 201: Obtain pixel frames before and after the page changes.
[0048] For example, in the process of user interaction with the page, elements that did not initially exist appear in the page, such as dialog boxes, message prompt boxes, etc. Obtaining the position of the dynamically appearing elements in the browser DOM tree is the key to configuring the dynamic elements of the page. This embodiment can determine the position of the dynamic elements through pixel frames at different times before and after the page changes.
[0049] For example, the user's screen, window, or application can be directly captured as a video stream, and by processing this real-time video stream, the pixel frame at the current time point can be periodically extracted.
[0050] Step 202: Use canvas to analyze the pixel frame to obtain the position where the change occurs in the page.
[0051] In this embodiment, the pixel frames at different times before and after the page changes can be sent to the page image processing module, and then the canvas is used to analyze the differences before and after the page changes. The two important application programming interfaces (APIs) mainly used in this process are ctx.getImageData and ctx.putImageData, where ctx.getImageData is responsible for obtaining pixel information and ctx.putImageData is responsible for drawing pixel information on the canvas.
[0052] Optionally, step 202 may specifically include: drawing the pixel information of the page onto a canvas according to a pixel frame; scanning the pixel information drawn before and after the page changes on the canvas to obtain coordinate information of a target area, where the target area is an area where the pixel information in the canvas changes; and determining a location where the change occurs in the page based on the coordinate information.
[0053] For example, you can first obtain all the pixel information of the canvas, then set a fixed scanning area, and scan from left to right and from top to bottom. Every time you pass through an area, calculate the number of different pixel values in the current area and store the coordinates of the current area in an array. Then traverse the array to find out the area where the pixel information in the image changes. The radius of the scanned rectangle generally needs to be slightly adjusted according to the area of different points (generally needs to be larger than the average radius of different points). In this way, the specific coordinate information of the location where the change occurs in the page can be confirmed.
[0054] Step 203: crop the image corresponding to the page according to the position where the change occurs in the page to obtain a target image corresponding to the position of the change.
[0055] In some examples, after obtaining the specific coordinate information of the position where the change occurs in the page, the image can be cropped according to the coordinates to obtain one or more target images.
[0056] Step 204: classify the target image and obtain the front-end component type corresponding to the classification of the target image.
[0057] For example, according to business needs, a mapping table or function can be set in advance to associate the image classification results (category labels) with the front-end component types. Then, according to the category labels output by the classification model, the corresponding front-end component types can be obtained by looking up the table or calling the mapping function.
[0058] Optionally, step 204 may specifically include: identifying the target image through an image classification model to obtain a classification of the target image, wherein the image classification model is a deep residual model based on an attention mechanism, the deep residual model includes a Resnet50 network, and the Resnet50 network includes multiple attention modules.
[0059] Exemplarily, the target image can be sent as input information to the image classification model to determine the component types in different areas. Among them, the image classification model uses a deep residual model based on the attention mechanism, selects the Resnet50 network that combines the deep network structure and the residual learning idea as the basic classification model, and introduces the attention mechanism to further optimize the deep residual model. Adding multiple attention modules to the Resnet50 network can further improve the classification performance of the model. The attention mechanism is a key technology widely used in natural language processing, statistical learning, image detection and other computer fields. The main idea is to use high weights to focus on important information and low weights to ignore relatively unimportant information. When facing different situations, it is also necessary to adjust the weights according to different information to adapt to complex information characteristics. Figure 3 As shown in the figure, the attention module is mainly divided into a trunk branch and a mask branch. The trunk branch is used for feature extraction. Specifically, the mask branch starts from the input and performs multiple maximum poolings so that the receptive field can be increased after a small number of residual units. When the lowest resolution is reached, the same linear interpolation as the maximum pooling is used after the residual unit to enlarge the features, thereby ensuring that the input and output sizes are consistent. This is followed by two consecutive 1×1 convolutional layers (conv), and finally the output is normalized by a sigmoid layer. In addition, since multiple maximum pooling and interpolation processes may occur, thus affecting the performance of the network, bypass connections can be added in the middle to solve this problem.
[0060] In this embodiment, considering that the addition of the attention module can improve the feature extraction capability of the network, and the improvement of the residual learning method ensures that adding multiple attention modules will not affect the overall performance of the network, multiple attention modules can be added to the Resnet50 network to enhance the feature extraction capability at different stages. For an attention module, since there is a residual module before the input information is decomposed into the trunk branch and the mask branch, the trunk branch is composed of two residual modules, and the mask branch is downsampled (down sample) and upsampled (up sample) and two consecutive 1×1 convolutional layers and then normalized by the sigmoid layer output. After the two branches are fused, the activated feature map is element-wise multiplied (element-wise product) with the input feature map and then element-wise added (element-wisesum) with the input feature map, and the result is processed by a residual module, so the depth of each attention module is the same as that of the four residual modules, that is, 12 layers.
[0061] For example, the improved residual attention network model structure is as follows Figure 4 As shown in the figure, after three consecutive 3×3 convolutional layers, the image performs a maximum pooling layer to reduce the image resolution, then extracts key information from the image through residual attention modules, and further reduces the resolution through an average pooling layer. The linear combination of features is performed in the fully connected layer. Finally, a softmax layer is used to calculate the probability distribution, obtain the probability of each image category and predict the processing result. After the image processing module, the position of the added / deleted elements and the component type when the page changes can be located, providing the precondition basis for the front-end component feature optimization search algorithm for the subsequent dynamic element detection and processing module. The improved deep residual attention image classification model is used to identify the type of dynamically changing front-end components, optimize the range of search depth in the depth-first search algorithm, and improve the efficiency of dynamic detection.
[0062] Step 205: perform a breadth-first search in the virtual DOM tree of the page according to the preset search depth corresponding to the front-end component type to detect the DOM nodes corresponding to the dynamic elements of the page in the virtual DOM tree.
[0063] Optionally, step 205 may specifically include: searching and comparing the same branches in the virtual DOM tree before and after the page changes according to the recommended search depth to obtain difference nodes in the virtual DOM tree before and after the page changes; and determining the DOM node corresponding to the dynamic element in the virtual DOM tree based on the difference nodes.
[0064] In some examples, searching and comparing according to the recommended search depth is mainly for detecting the difference nodes in the DOM tree. The difference nodes in the DOM tree can be put into an asynchronous queue and matched with the element configuration information on the page using a depth-first search method. If the target page element is found, the page element state is directly changed. Conventional depth-first search selects a branch each time and searches all the way to the end, that is, it recurses to the boundary before backtracking. However, in the process of more frequent dynamic element changes, if the search tree has a large number of branches, and the changed page elements are at a relatively deep level in the DOM tree and are relatively shallow on the branch, the conventional depth-first search method will waste a lot of search time, resulting in a relatively low update efficiency of the page.
[0065] In this embodiment, for different components, a conventional recommended depth can be selected for priority search. For example, the recommended depth of the dialog component (Modal) is [1, 3]; the recommended depth of the message prompt component (Message) is [1, 2]; the recommended depth of the drop-down menu component (Dropdown) is [1, 2], etc. If the target is not found after the breadth-first search at the recommended depth is completed, the original depth-first search can be continued. Since this method only needs to store the branch being visited, the space complexity is O(d). Since there is a high probability that the search result can be obtained in the breadth-first search at the recommended depth, its time complexity can be close to the time complexity of the breadth-first search.
[0066] Optionally, step 205 may further specifically include: loading element configuration information corresponding to the DOM node, and updating the real DOM tree mapped to the page.
[0067] In some examples, such as Figure 5 As shown, an application architecture proposed in this embodiment mainly includes an application layer, a plug-in layer, a service layer, and a data storage layer. Among them, the application layer is divided into two parts: the product front-end engineering that uses the capabilities of this proposal, and the permission configuration platform for setting the status of page elements; the service layer may include configuration management services, user management services, and other configuration item services. The product side accesses the plug-in amp-probe, and completes the plug-in access work after the corresponding configuration. In the permission configuration platform, the user configures the status rules of the corresponding product, corresponding page, and corresponding element (including whether to display, whether to disable, etc.), which can be used to manage the permissions of configuration item services and applications running in the Kubernetes (K8s) cluster in this embodiment. The data source of the configuration platform and the plug-in is the same, and the user needs to deploy the front-end and back-end engineering of the permission configuration platform accordingly. By configuring the plug-in, the code intrusion of the page element configuration on the original project can be minimized. Among them, the plug-in can be publicly released in the form of an npm package for users to download and use freely, which greatly reduces the code intrusion of the page element status configuration on the original project.
[0068] For example, Figure 6As shown, when the user switches pages, the plug-in updates the rule information and runs the regular page element processing module at the same time. During the page loading process, the page element configuration in the permission configuration platform is first initialized and rendered, that is, static element configuration is performed, and the static element is sent to the regular page element processing module. After obtaining the page DOM that needs to change the state, the element attributes are directly updated. At the same time, the appearance of dynamic elements is detected during page interaction, and the pictures before and after the page changes are sent to the page picture processing module to determine the position of the changed page elements. The improved deep residual attention network model is used to classify the components and the results are input into the dynamic element detection and processing module. Then, the front-end component feature optimization search algorithm is used to obtain the DOM node corresponding to the dynamic element. Combined with the page configuration information, the update is mapped to the real DOM tree of the page, and finally all the page change configuration tasks are completed.
[0069] Compared with the current existing technology, by applying the technical solution of this embodiment, a more efficient page element processing method is provided. The position of the dynamic elements in the page is located by using canvas technology, and then the front-end component type corresponding to the dynamic elements in the page is found through a deep residual model based on the attention mechanism, and the recommended search depth corresponding to the front-end component type is used to perform a breadth-first search in the virtual DOM tree, thereby improving the efficiency of detecting the corresponding DOM nodes of the dynamic elements in the virtual DOM tree, thereby improving the updating efficiency of the page elements.
[0070] Further, as Figure 1 and Figure 2 The specific implementation of the method shown in the embodiment provides a device for processing page elements, such as Figure 7 As shown, the device includes: an acquisition module 31, a cropping module 32, a classification module 33, and a search module 34.
[0071] An acquisition module 31 is configured to acquire a location where a change occurs in a page;
[0072] A cropping module 32 is configured to crop the image corresponding to the page according to the position to obtain a target image corresponding to the position;
[0073] A classification module 33 is configured to classify the target image and obtain a front-end component type corresponding to the classification of the target image;
[0074] The search module 34 is configured to perform a breadth-first search in the virtual DOM tree of the page according to a preset search depth corresponding to the front-end component type to detect DOM nodes corresponding to the dynamic elements of the page in the virtual DOM tree, wherein the search depth is the number of edges passed from the root node of the virtual DOM tree to the target node.
[0075] In some examples of this embodiment, the acquisition module 31 is specifically configured to acquire pixel frames before and after the page changes; and analyze the pixel frames using a canvas to obtain a location where the change occurs in the page.
[0076] In some examples of this embodiment, the acquisition module 31 is further specifically configured to draw the pixel information of the page onto a canvas according to the pixel frame; scan the pixel information drawn before and after the page changes on the canvas to obtain coordinate information of a target area, where the target area is an area in which the pixel information in the canvas changes; and determine a location where the change occurs in the page based on the coordinate information.
[0077] In some examples of this embodiment, the classification module 33 is specifically configured to identify the target image through an image classification model to obtain a classification of the target image, wherein the image classification model is a deep residual model based on an attention mechanism, and the deep residual model includes a Resnet50 network, and the Resnet50 network includes multiple attention modules.
[0078] In some examples of this embodiment, the search module 34 is specifically configured to search and compare the same branches in the virtual DOM tree before and after the page changes according to the recommended search depth to obtain the difference nodes in the virtual DOM tree before and after the page changes; and determine the DOM node corresponding to the dynamic element in the virtual DOM tree based on the difference nodes.
[0079] In some examples of this embodiment, the search module 34 is further configured to load the element configuration information corresponding to the DOM node, and update the real DOM tree mapped to the page.
[0080] It should be noted that the present embodiment provides a device for processing page elements, and other corresponding descriptions of the functional units involved can be referred to in Figure 1 and Figure 2 The corresponding description in will not be repeated here.
[0081] Based on the above Figure 1 and Figure 2 The method shown in the embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 2 The method shown.
[0082] Based on the above Figure 1 and Figure 2The method shown in the embodiment also provides a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 2 The method shown.
[0083] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.
[0084] Based on the above Figure 1 and Figure 2 The method shown, and Figure 7 In order to achieve the above-mentioned purpose, the embodiment of the present application also provides an electronic device, such as a personal computer, a server, a laptop computer, an intelligent robot or other intelligent terminal, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 2 The method shown.
[0085] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.
[0086] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.
[0087] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.
[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or by hardware. By applying the solution of this embodiment, compared with the current prior art, this embodiment improves the efficiency of detecting the DOM nodes corresponding to the dynamic elements in the virtual DOM tree by finding the front-end component type corresponding to the dynamic elements in the page and adopting the recommended search depth corresponding to the front-end component type, and performing a breadth-first search in the virtual DOM tree, thereby improving the efficiency of updating the page elements.
[0089] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0090] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.
Claims
1. A method for processing page elements, It is characterized in that include: Get the position where the change occurs in the page; Cropping the image corresponding to the page according to the position to obtain a target image corresponding to the position; Classify the target image, and obtain the front-end component type corresponding to the classification of the target image; According to a preset search depth corresponding to the front-end component type, a breadth-first search is performed in the virtual DOM tree of the page to detect DOM nodes corresponding to the dynamic elements of the page in the virtual DOM tree, wherein the search depth is the number of edges passed from the root node of the virtual DOM tree to the target node.
2. The method according to claim 1, It is characterized in that The location where the change occurs in the acquisition page includes: Obtain pixel frames before and after the page changes; The pixel frame is analyzed using a canvas to obtain a position where a change occurs in the page.
3. The method according to claim 2, It is characterized in that Analyzing the pixel frame using a canvas to obtain a location where a change occurs in the page includes: Drawing the pixel information of the page onto a canvas according to the pixel frame; Scanning the pixel information drawn respectively before and after the page changes on the canvas to obtain coordinate information of a target area, where the target area is an area in the canvas where the pixel information changes; The position where the change occurs in the page is determined according to the coordinate information.
4. The method according to claim 1, It is characterized in that The classifying the target image includes: The target image is identified by an image classification model to obtain a classification of the target image, wherein the image classification model is a deep residual model based on an attention mechanism, the deep residual model includes a Resnet50 network, and the Resnet50 network includes multiple attention modules.
5. The method according to claim 1, It is characterized in that According to the preset search depth corresponding to the front-end component type, a breadth-first search is performed in the virtual DOM tree of the page, including: Search and compare the same branches in the virtual DOM tree before and after the page changes according to the search depth to obtain difference nodes in the virtual DOM tree before and after the page changes; According to the difference node, a DOM node corresponding to the dynamic element in the virtual DOM tree is determined.
6. The method according to claim 1, It is characterized in that The method further comprises: The element configuration information corresponding to the DOM node is loaded, and updated and mapped to the real DOM tree of the page.
7. A device for processing page elements, It is characterized in that include: An acquisition module, configured to acquire a location where a change occurs in a page; A cropping module is configured to crop the image corresponding to the page according to the position to obtain a target image corresponding to the position; A classification module, configured to classify the target image and obtain a front-end component type corresponding to the classification of the target image; The search module is configured to perform a breadth-first search in the virtual DOM tree of the page according to a preset search depth corresponding to the front-end component type to detect DOM nodes corresponding to dynamic elements of the page in the virtual DOM tree, wherein the search depth is the number of edges passed from the root node of the virtual DOM tree to the target node.
8. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
9. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.
10. A computer program product having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
