Web page testing method, device, electronic device and medium

Abstract

The present disclosure provides a method for establishing a sample library for testing web pages, a web page testing method, device, electronic device and medium, and relates to the field of computer technology, and in particular to the field of web page monitoring technology. The implementation scheme is: obtaining web page data corresponding to a plurality of reference web pages respectively; extracting the framework features and search result features of the reference web pages; dividing the reference web pages with the same framework features into the same category to obtain at least one category; taking the same framework features corresponding to each category as sample framework features; determining sample search result features based on the search result features of each reference web page in the category; obtaining sample features corresponding to the category based on the sample framework features and the sample search result features; determining a sample library based on the sample features corresponding to each category in at least one category.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of web page monitoring technology, and specifically to a method for establishing a sample library for testing web pages, a web page testing method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of communications and the Internet, web page content is becoming increasingly rich, from the original text to pictures, videos, etc. The increasingly rich web page content has led to more and more display styles of search engine result pages. In the process of continuous iteration of search engine effects, how to effectively monitor the display effect of web pages is particularly important.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides a method for establishing a sample library for testing a web page, a web page testing method, a device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of the present disclosure, a method for establishing a sample library for testing web pages is provided, comprising: obtaining web page data corresponding to a plurality of reference web pages respectively; for each reference web page in the plurality of reference web pages, identifying search result data in the web page data corresponding to the reference web page, and using data in the web page data other than the search result data as framework data; extracting framework features of the reference web page based on the framework data corresponding to the reference web page; extracting search result features of the reference web page based on the search result data corresponding to the reference web page; dividing one or more reference web pages in the plurality of reference web pages having the same framework features into the same category to obtain at least one category; for each category in the at least one category, using the same framework features corresponding to the category as sample framework features corresponding to the category; determining sample search result features corresponding to the category based on the search result features of each reference web page in the category; obtaining sample features corresponding to the category based on the sample framework features and the sample search result features; and determining a sample library based on the sample features corresponding to each category in the at least one category.

[0006] According to another aspect of the present disclosure, a method for testing a web page is provided, comprising: obtaining a sample library and web page data to be tested corresponding to the web page to be tested; identifying search result data in the web page data to be tested, and using data other than the search result data in the web page data to be tested as framework data; comparing the framework data corresponding to the web page to be tested with sample framework features in each sample feature in the sample library; in response to the framework data corresponding to the web page to be tested conforming to a sample framework feature in a sample feature in the sample library, comparing the search result data corresponding to the web page to be tested with sample search result features in the sample feature; and in response to the search result data corresponding to the web page to be tested conforming to any sub-result feature in the sample search result features, determining that the web page to be tested passes the test, wherein the sample library is obtained according to a method for establishing a sample library for testing web pages.

[0007] According to another aspect of the present disclosure, there is provided an apparatus for establishing a sample library for testing web pages, comprising: a first acquisition module, configured to acquire web page data corresponding to a plurality of reference web pages respectively; a first identification module, configured to identify, for each reference web page among the plurality of reference web pages, search result data in the web page data corresponding to the reference web page, and use data in the web page data other than the search result data as framework data; a first extraction module, configured to extract framework features of the reference web page based on the framework data corresponding to the reference web page; a second extraction module, configured to extract search result features of the reference web page based on the search result data corresponding to the reference web page; a division module, configured to divide the search result data into the search result data corresponding to the reference web page; One or more reference web pages with the same framework features among multiple reference web pages are divided into the same category to obtain at least one category; a first determination module is configured to, for each category in the at least one category, use the same framework features corresponding to the category as the sample framework features corresponding to the category; a second determination module is configured to determine the sample search result features corresponding to the category based on the search result features of each reference web page in the category; a third determination module is configured to obtain the sample features corresponding to the category based on the sample framework features and the sample search result features; and a fourth determination module is configured to determine a sample library based on the sample features corresponding to each category in the at least one category.

[0008] According to another aspect of the present disclosure, a testing device for a web page is provided, comprising: a second acquisition module, configured to acquire a sample library and web page data to be tested corresponding to the web page to be tested; a second identification module, configured to identify search result data in the web page data to be tested, and use data other than the search result data in the web page data to be tested as framework data; a first comparison module, configured to compare the framework data corresponding to the web page to be tested with a sample framework feature in each sample feature in the sample library; a second comparison module, configured to, in response to the framework data corresponding to the web page to be tested conforming to a sample framework feature in a sample feature in the sample library, compare the search result data corresponding to the web page to be tested with a sample search result feature in the sample feature; and an eighth determination module, configured to determine that the web page to be tested passes the test in response to the search result data corresponding to the web page to be tested conforming to any sub-result feature in the sample search result feature, wherein the sample library is obtained according to a method for establishing a sample library for testing web pages.

[0009] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the aforementioned methods.

[0010] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the aforementioned methods.

[0011] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements any of the aforementioned methods when executed by a processor.

[0012] According to one or more embodiments of the present disclosure, a method for establishing a sample library for testing web pages is provided, by dividing a reference web page into a frame part and a search result part, classifying the reference web page based on the frame part and performing feature extraction on the two parts respectively, so as to construct a general sample feature with a certain style for a category for testing the web page.

[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0015] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0016] Figure 2 A flowchart of a method for establishing a sample library for testing web pages according to an embodiment of the present disclosure is shown;

[0017] Figure 3 A flow chart of a method for testing a web page according to an embodiment of the present disclosure is shown;

[0018] Figure 4 A structural block diagram of an apparatus for establishing a sample library for testing web pages according to an embodiment of the present disclosure is shown;

[0019] Figure 5 A structural block diagram showing a web page testing device according to an embodiment of the present disclosure; and

[0020] Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0021] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0022] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0023] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0024] In the related art, monitoring the display effect of a web page often requires testers to manually add monitoring items on the monitoring platform, and it takes a long time to determine the search terms that hit the new product or strategy, determine the features that need to be monitored, and the appropriate monitoring rules to monitor, debug, and observe the stability of the monitoring web page. This process takes a long time, and the quality of the manually added monitoring scheme is closely related to factors such as the ability of the tester, and there may be uneven monitoring quality. In some related technologies, web page changes can also be detected by comparing the overall structure of the document object model (DOM) tree or the image of the DOM tree structure. In this method, the web pages need to be compared item by item, so it is only applicable to simple web pages. For web pages with complex web page results such as search engine result pages, if the entire page is simply used as the baseline version, it will cause the baseline to swell, resulting in too many false positives.

[0025] To solve the above problems, the present disclosure provides a method for establishing a sample library for testing web pages, by dividing a reference web page into a framework part and a search result part, classifying the reference web page based on the framework part and extracting features from the two parts respectively, so as to construct a universal sample feature with a certain style for a category for testing web pages to realize automatic monitoring of web pages, which greatly reduces the input of manual labor. This feature extraction method and the construction of corresponding sample features for different categories of web pages make the extracted features more flexible, effectively reduce the number of features tested, and improve the test efficiency.

[0026] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0028] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of any of the aforementioned methods.

[0029] In some embodiments, server 120 may also provide other services or software applications that may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0030] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0031] The user may use client devices 101, 102, 103, 104, 105 and / or 106 to perform any of the aforementioned methods. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0032] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0033] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0034] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0035] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0036] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0037] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0038] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the databases used by the server 120 may be, for example, relational databases. One or more of these databases may store, update, and retrieve data to and from the databases in response to commands.

[0039] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0040] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0041] Figure 2 FIG. 1 is a flow chart showing a method for establishing a sample library for testing web pages according to an embodiment of the present disclosure. Figure 2 As shown, the method 200 for establishing a sample library for testing web pages includes:

[0042] Step S201, obtaining web page data corresponding to a plurality of reference web pages. Exemplarily, the web page data corresponding to the plurality of reference web pages obtained are Hyper Text Markup Language (HTML) data corresponding to the web pages, and subsequent processing operations are all performed based on the HTML data corresponding to the web pages.

[0043] Step S202: for each reference web page among the plurality of reference web pages, identify the search result data in the web page data corresponding to the reference web page, and use the data in the web page data other than the search result data as the framework data. It is understandable that for a web page such as a search engine result page, the web page data includes framework data for displaying the web page framework and search result data for displaying the search results. These two types of data have different characteristics in the display process, and identifying and distinguishing the two types of data helps to improve the efficiency of web page testing.

[0044] Step S203: extracting the frame features of the reference web page based on the frame data corresponding to the reference web page.

[0045] Step S204: extracting search result features of the reference web page based on the search result data corresponding to the reference web page.

[0046] The framework data and the search result data have different characteristics in the web page display process. Feature extraction is performed on the framework data and the search result data respectively to obtain corresponding framework features and search result features.

[0047] Step S205: classify one or more reference web pages with the same frame feature among the multiple reference web pages into the same category to obtain at least one category.

[0048] Step S206: For each category in the at least one category, the same framework feature corresponding to the category is used as the sample framework feature corresponding to the category.

[0049] It is understandable that web pages with the same frame features are web pages of the same category. For example, in the search engine result page, multiple result pages corresponding to the same search engine have the same search box, web page logo, operation bar at the bottom of the web page, etc. Therefore, compared with using the frame features of each reference web page as sample frame features for comparison, classifying web pages based on frame features and using the same frame features corresponding to the category as sample frame features corresponding to the category can effectively reduce the number of frame features to be compared and prevent feature expansion.

[0050] Step S207: Determine sample search result features corresponding to the category based on the search result features of each reference web page in the category.

[0051] Step S208: Based on the sample framework features and the sample search result features, obtain the sample features corresponding to the category. Thus, based on the sample search result features and the sample search result features, the common style or features that web pages of the same category should have can be obtained. This process can reduce the number of extracted features, make feature extraction more flexible, and improve the efficiency of subsequent feature comparison of the web pages to be tested based on the sample library.

[0052] Step S209: Determine a sample library based on the sample features corresponding to each category in the at least one category, thereby obtaining a sample library for testing web pages of different categories.

[0053] Thus, a sample library for web page testing is established, and the web page to be tested can be tested subsequently by comparing it with the established sample library. In the process of establishing the sample library, the reference web page is divided into a framework part and a search result part, and the reference web page is classified based on the framework part and features are extracted for the two parts respectively, so as to construct a universal sample feature with a certain style for a category for comparison with the web page to be tested to realize automatic monitoring of the web page, which greatly reduces the manual input. This feature extraction method and the construction of corresponding sample features for different categories of web pages make the extracted features more flexible, effectively reduce the number of features tested, and improve the test efficiency.

[0054] According to some embodiments, step S203 includes: for each reference web page among the multiple reference web pages, extracting leaf node data in the framework data corresponding to the reference web page as the framework feature of the reference web page, wherein the leaf node data includes attribute data representing the attributes of the leaf node and path data representing the position of the leaf node in the DOM tree structure corresponding to the reference web page; and wherein step S204 includes: extracting leaf node data in the search result data corresponding to the reference web page as the search result feature of the reference web page.

[0055] For example, the DOM tree of a web page such as a search engine result page is often very complex. If all elements in the DOM tree are extracted as features and compared one by one to test the web page, the testing process will be very slow and prone to errors. The inventor has observed the result pages of various search engines through statistics and found that the elements displayed on the web page are basically on the leaf nodes of the DOM tree. Therefore, all leaf nodes of the DOM tree and the corresponding leaf node paths can be traversed to obtain them, and these two types of information can be used to describe the content displayed on the result page. This feature extraction process greatly reduces the amount of data corresponding to the DOM tree structure to be tested.

[0056] According to some embodiments, the attribute data corresponding to a leaf node whose type is text includes the type, label and value of the leaf node, and the attribute data corresponding to a leaf node whose type is non-text includes the type and label of the leaf node.

[0057] It is understandable that when testing a web page to be tested, the text content displayed on the web page needs to be verified. Therefore, when constructing a sample library, the value of the leaf node of type text needs to be used as part of the search result feature. In some examples, the style of the description text, such as whether it is bold, whether it has special effects, etc., can be included in the attribute data corresponding to the leaf node for testing the text content in the web page to be tested according to the test needs. For non-text nodes, only the label and type of the node need to be included in its attribute data. During the test of the web page to be tested, there is no need to verify the specific value of the non-text node.

[0058] According to some embodiments, the method 200 for establishing a sample library for testing web pages also includes: for each reference web page among the multiple reference web pages, determining at least one parent node directly connected to the search result data corresponding to the reference web page in the DOM tree structure; for each parent node among the at least one parent node, determining at least one search result directly connected to the parent node; and determining a sub-result feature in the search result feature of the reference web page corresponding to each search result of the at least one search result.

[0059] In the search engine result page, multiple results with similar structures are usually displayed. For example, in the display of image results, it can be considered that the display of one or more images belongs to a similar structure; in the web page displaying blog posts, each blog post is usually displayed in the form of a user name, blog post text content and / or blog post image content and operation tags (such as forwarding, like or comment icons), etc., and each blog post can be considered to have a similar structure. Correspondingly, in the HTML data corresponding to the web page, the result data corresponding to multiple results will be connected to the same parent node. When it is recognized that there are multiple nodes with the same attributes and similar substructures under the same parent node, it means that there are multiple results under the parent node, and the sub-result features corresponding to each result can be determined in the search result features of the web page for the aggregation of search result features in the subsequent process, so that when the web page to be tested is tested, as long as the web page meets one of the multiple sub-result features corresponding to the multiple results under the same parent node, the web page can be considered to have passed the test. In one example, when the template of the web page is not standardized, an intervention identification path can be reserved and the structure of multiple results can be manually intervened and identified.

[0060] According to some embodiments, step S207 includes: for each parent node of at least one parent node corresponding to a reference web page of each category, aggregating sub-result features corresponding to at least one search result directly connected to the parent node in each reference web page in the category to obtain the search result features corresponding to the parent node; and determining the sample search result features corresponding to the category based on the search result features corresponding to each parent node of the at least one parent node.

[0061] It can be understood that the above-mentioned aggregation of sub-result features means merging multiple sub-result features while deleting duplicate sub-result features, so that when testing a web page to be tested, as long as the web page meets one of the multiple sub-result features corresponding to multiple results under the same parent node, the web page can be considered to have passed the test without the need to perform feature permutations and combinations in the process of constructing sample features. In the same framework, aggregating multiple results of the same parent node can effectively reduce the number of features and prevent feature expansion compared to the permutations and combinations of multiple features.

[0062] According to some embodiments, the method 200 for establishing a sample library for testing web pages further includes: for each reference web page among the multiple reference web pages, in response to the search result data corresponding to the reference web page including text data consisting of multiple leaf nodes of type text that are directly connected to the same parent node and the text data is split into the multiple leaf nodes due to a preset style, splicing the multiple leaf nodes and annotating a leaf node obtained by splicing with the preset style.

[0063] Due to the existence of expressions, special effects and other styles, the same cluster of text will be divided into multiple leaf nodes. If the features are extracted according to the leaf nodes, there will be a large number of leaf nodes and the order will be uncertain. For the display test of the web page, this part is all text, so the text can be aggregated and spliced ​​to obtain a leaf node containing the text segment, and the style of the text can be marked (whether there is an expression, bold text, etc.), which helps to reduce the number of features.

[0064] According to another aspect of the present disclosure, a method for testing a web page is provided. Figure 3 As shown, the web page testing method 300 includes: step S301, obtaining the sample library and the web page data to be tested corresponding to the web page to be tested; step S302, identifying the search result data in the web page data to be tested, and using the data in the web page data to be tested except the search result data as the framework data; step S303, comparing the framework data corresponding to the web page to be tested with the sample framework features in each sample feature in the sample library; step S304, in response to the framework data corresponding to the web page to be tested conforming to the sample framework features in a sample feature in the sample library, comparing the search result data corresponding to the web page to be tested with the sample search result features in the sample feature; and step S305, in response to the search result data corresponding to the web page to be tested conforming to any sub-result feature in the sample search result features, determining that the web page to be tested passes the test, wherein the sample library is obtained according to the aforementioned method for establishing a sample library for testing web pages.

[0065] When testing a web page to be tested, the framework of the web page to be tested is first verified based on the sample library. After finding the matching sample framework features in the sample library, the search result part is compared with the corresponding sample search result features. In the process of verifying the search results, as long as the web page meets one of the multiple sub-result features corresponding to multiple results under the same parent node, the web page can be considered to have passed the test.

[0066] According to some embodiments, the web page testing method 300 also includes: in response to the web page to be tested failing the test, extracting the frame features and search result features of the web page to be tested; updating the sample library based on the frame features and search result features of the web page to be tested; and testing the web page to be tested based on the updated sample library.

[0067] It is understandable that web pages such as search engine result pages have very complex and rich styles, so a large number of samples are needed to characterize all the features of the web pages as much as possible, and too many samples will lead to excessive storage burden in the process of storing the sample library. For this problem, the present disclosure uses a step-by-step iterative method to first use a part of the reference web pages to build a sample library for web page testing. During the test, the web pages that fail the test are checked and the normal web pages that fail the test are subjected to feature extraction and other steps described above to update the sample library, thereby expanding the samples while reducing the possibility of redundant samples. For the web pages to be tested that fail the test, the web pages are tested before and after the sample library is updated to verify the expansion effect of the sample library.

[0068] According to another aspect of the present disclosure, a device for establishing a sample library for testing web pages is provided. Figure 4 As shown, the device 400 for establishing a sample library for testing web pages includes: a first acquisition module 401, configured to acquire web page data corresponding to a plurality of reference web pages respectively; a first identification module 402, configured to identify, for each reference web page among the plurality of reference web pages, search result data in the web page data corresponding to the reference web page, and use the data in the web page data other than the search result data as framework data; a first extraction module 403, configured to extract framework features of the reference web page based on the framework data corresponding to the reference web page; a second extraction module 404, configured to extract search result features of the reference web page based on the search result data corresponding to the reference web page; a division module 405, configured to divide the plurality of reference web pages into One or more reference web pages in a web page having the same framework feature are divided into the same category to obtain at least one category; a first determination module 406 is configured to, for each category in the at least one category, use the same framework feature corresponding to the category as the sample framework feature corresponding to the category; a second determination module 407 is configured to determine the sample search result feature corresponding to the category based on the search result feature of each reference web page in the category; a third determination module 408 is configured to obtain the sample feature corresponding to the category based on the sample framework feature and the sample search result feature; and a fourth determination module 409 is configured to determine a sample library based on the sample feature corresponding to each category in the at least one category.

[0069] Exemplarily, the web page data corresponding to the multiple reference web pages acquired by the first acquisition module 401 are Hypertext Markup Language (HTML) data corresponding to the web pages, and the subsequent processing operations of the device 400 are all based on the HTML data corresponding to the web pages. Since the content information in the web page usually has different attributes, for example, in the search engine's result page, the search box, web page logo, and the operation bar at the bottom of the web page are fixed, while the search results are usually changing. Therefore, this feature of the web page can be used to identify different contents in the web page data and classify the web pages, and feature extraction is performed for different areas to extract the common styles or features that web pages of the same category should have, so as to reduce the number of extracted features, make the extracted features more flexible, and improve the efficiency of feature comparison.

[0070] Thus, the web page to be tested can be tested by comparing with the sample library established by the device 400. In the process of establishing the sample library by the device 400, the reference web page is divided into a framework part and a search result part, and the reference web page is classified based on the framework part and features are extracted for the two parts respectively, so as to construct a universal sample feature with a certain style for a category for comparison with the web page to be tested to realize automatic monitoring of the web page, which greatly reduces the input of manual labor. This feature extraction method and the construction of corresponding sample features for different categories of web pages make the extracted features more flexible, effectively reduce the number of features tested, and improve the test efficiency.

[0071] According to some embodiments, the first extraction module 403 is further configured to: for each reference web page among the multiple reference web pages, extract leaf node data in the framework data corresponding to the reference web page as the framework feature of the reference web page, wherein the leaf node data includes attribute data representing the attributes of the leaf node and path data representing the position of the leaf node in the DOM tree structure corresponding to the reference web page; and wherein the second extraction module 404 is further configured to: extract leaf node data in the search result data corresponding to the reference web page as the search result feature of the reference web page.

[0072] For example, the DOM tree of a web page such as a search engine result page is often very complex. If all elements in the DOM tree are extracted as features and compared one by one to test the web page, the testing process will be very slow and prone to errors. The inventor has observed the result pages of various search engines through statistics and found that the elements displayed on the web page are basically on the leaf nodes of the DOM tree. Therefore, the first extraction module 403 and the second extraction module 404 can extract all the leaf nodes of the DOM tree and the corresponding leaf node paths by traversing, and use these two types of information to describe the content displayed on the result page. This feature extraction process greatly reduces the amount of data corresponding to the DOM tree structure to be tested.

[0073] According to some embodiments, the attribute data corresponding to a leaf node whose type is text includes the type, label and value of the leaf node, and the attribute data corresponding to a leaf node whose type is non-text includes the type and label of the leaf node.

[0074] It is understandable that when testing a web page to be tested based on a sample library, the text content displayed on the web page needs to be verified. Therefore, when the device 400 establishes a sample library, the value of the leaf node of type text needs to be used as part of the search result feature. In some examples, the second extraction module 404 can include the style of the description text, such as whether it is bold, whether it has special effects, etc., in the attribute data corresponding to the leaf node for testing the text content in the web page to be tested according to the test needs. For non-text nodes, the first extraction module 403 only needs to include the label and type of the node in its attribute data. During the test of the web page to be tested, there is no need to verify the specific value of the non-text node.

[0075] According to some embodiments, the device 400 for establishing a sample library for testing web pages also includes: a fifth determination module, configured to determine, for each reference web page among the multiple reference web pages, at least one parent node directly connected to the search result data corresponding to the reference web page in the DOM tree structure; a sixth determination module, configured to determine, for each parent node among the at least one parent node, at least one search result directly connected to the parent node; and a seventh determination module, configured to determine a sub-result feature in the search result feature of the reference web page corresponding to each search result of the at least one search result.

[0076] In the search engine's result page, multiple results with similar structures are usually displayed. For example, in the display of image results, it can be considered that the display of one or more images belongs to a similar structure; in the web page displaying blog posts, each blog post is usually displayed in the form of including user name, blog post text content and / or blog post image content and operation tags (such as forwarding, like or comment icons), etc., and each blog post can be considered to have a similar structure. Correspondingly, in the HTML data corresponding to the web page, the result data corresponding to multiple results will be connected to the same parent node. When the sixth determination module recognizes that there are multiple nodes with the same attributes and similar substructures under the same parent node, it means that there are multiple results under the parent node, and the sub-result feature corresponding to each result can be determined in the search result feature of the web page, so as to be used for the aggregation of search result features in the subsequent process, so that when the web page to be tested is tested, as long as the web page meets one of the multiple sub-result features corresponding to the multiple results under the same parent node, the web page can be considered to have passed the test. In one example, when the template of the web page is not standardized, the apparatus 400 may reserve an intervention identification path and manually perform intervention identification on the structure of multiple results.

[0077] According to some embodiments, the second determination module 407 includes: an aggregation unit, configured to aggregate, for each parent node of at least one parent node corresponding to a reference web page of each category, sub-result features corresponding to at least one search result directly connected to the parent node in each reference web page in the category, to obtain the search result features corresponding to the parent node; and a determination unit, configured to determine the sample search result features corresponding to the category based on the search result features corresponding to each parent node of the at least one parent node.

[0078] It can be understood that the aggregation unit aggregates the sub-result features, which means merging multiple sub-result features while deleting duplicate sub-result features, so that when testing a web page to be tested, as long as the web page meets one of the multiple sub-result features corresponding to multiple results under the same parent node, the web page can be considered to have passed the test without the need to perform feature permutations and combinations in the process of constructing sample features. Under the same framework, the aggregation unit aggregates multiple results of the same parent node, which can effectively reduce the number of features and prevent feature expansion compared to the permutations and combinations of multiple features.

[0079] According to some embodiments, the device 400 for establishing a sample library for testing web pages also includes: a splicing module, which is configured to, for each reference web page among the multiple reference web pages, in response to the search result data corresponding to the reference web page including text data consisting of multiple leaf nodes of type text that are directly connected to the same parent node and the text data is split into the multiple leaf nodes due to a preset style, splice the multiple leaf nodes and mark a leaf node obtained by splicing with the preset style.

[0080] Due to the existence of expressions, special effects and other styles, the same cluster of text will be divided into multiple leaf nodes. If the features are extracted according to the leaf nodes, there will be a large number of leaf nodes and the order will be uncertain. For the display test of the web page, this part is all text, so the splicing module can aggregate and splice the text to obtain a leaf node containing the text segment, and mark the style of the text (whether there is an expression, bold text, etc.), which helps to reduce the number of features.

[0081] According to another aspect of the present disclosure, a web page testing device is provided. Figure 5 As shown, the web page testing device 500 includes: a second acquisition module 501, configured to acquire the sample library and the web page data to be tested corresponding to the web page to be tested; a second identification module 502, configured to identify the search result data in the web page data to be tested, and use the data in the web page data to be tested except the search result data as the framework data; a first comparison module 503, configured to compare the framework data corresponding to the web page to be tested with the sample framework features in each sample feature in the sample library; a second comparison module 504, configured to compare the search result data corresponding to the web page to be tested with the sample search result features in the sample feature in response to the framework data corresponding to the web page to be tested conforming to the sample framework features in a sample feature in the sample library; and an eighth determination module 505, configured to determine that the web page to be tested passes the test in response to the search result data corresponding to the web page to be tested conforming to any sub-result feature in the sample search result features, wherein the sample library is obtained according to the aforementioned method for establishing a sample library for testing web pages.

[0082] When the device 500 tests the web page to be tested, the framework part of the web page to be tested is first verified, and after finding the matching sample framework feature in the sample library, the search result part is compared with the corresponding sample search result feature. In the process of verifying the search result, as long as the web page meets one of the multiple sub-result features corresponding to the multiple results under the same parent node, the web page can be considered to have passed the test.

[0083] According to some embodiments, the web page testing device 500 also includes: an extraction module, configured to extract the framework features and search result features of the web page to be tested in response to the web page to be tested failing the test; an update module, configured to update the sample library based on the framework features and search result features of the web page to be tested; and a testing module, configured to test the web page to be tested based on the updated sample library.

[0084] It is understandable that web pages such as search engine result pages have very complex and rich styles, so a large number of samples are needed to characterize all the features of the web pages as much as possible, and too many samples will lead to excessive storage burden in the process of storing the sample library. For this problem, the web page testing device first uses a part of the reference web pages to build a sample library for web page testing in a step-by-step iterative manner. During the testing process of the web page testing device 500, the extraction module checks the web pages that have not passed the test and extracts features from the normal web pages that have not passed the test, and other steps described above are used to update the sample library, thereby expanding the samples while reducing the possibility of redundant samples. For the web pages to be tested that have not passed the test, the web pages are tested before and after the sample library is updated to verify the expansion effect of the sample library.

[0085] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0086] Multiple components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, an output unit 607, a storage unit 608 and a communication unit 609. The input unit 606 can be any type of device that can input information to the electronic device 600. The input unit 606 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone and / or a remote control. The output unit 607 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator and / or a printer. The storage unit 608 can include but is not limited to a disk, an optical disk. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as Bluetooth TM devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0087] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a method for establishing a sample library for testing a web page or a testing method for a web page. For example, in some embodiments, a method for establishing a sample library for testing a web page or a testing method for a web page may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method for establishing a sample library for testing a web page or a testing method for a web page described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured in any other appropriate manner (eg, by means of firmware) to execute the method for establishing a sample library for testing a web page or the method for testing a web page.

[0088] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0089] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0090] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0092] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0093] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0094] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0095] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A method for establishing a sample library for testing web pages, comprising: Obtaining web page data corresponding to each of the plurality of reference web pages; For each reference web page in the plurality of reference web pages, Identifying search result data in the web page data corresponding to the reference web page, and using data in the web page data other than the search result data as framework data; Extracting the frame features of the reference webpage based on the frame data corresponding to the reference webpage includes: For each reference web page among the plurality of reference web pages, extracting leaf node data in the frame data corresponding to the reference web page as the frame feature of the reference web page, wherein the leaf node data includes attribute data representing the attributes of the leaf node and path data representing the position of the leaf node in the DOM tree structure corresponding to the reference web page; Extracting search result features of the reference webpage based on the search result data corresponding to the reference webpage includes: Extracting leaf node data from the search result data corresponding to the reference web page as search result features of the reference web page; Determine at least one parent node in the DOM tree structure that is directly connected to the search result data corresponding to the reference web page; For each parent node of the at least one parent node, determining at least one search result directly connected to the parent node; and Determine a sub-result feature corresponding to each of the at least one search result in the search result feature of the reference webpage; Classifying one or more reference web pages having the same frame feature among the plurality of reference web pages into the same category to obtain at least one category; For each of the at least one category, The same framework features corresponding to the category are used as the sample framework features corresponding to the category; Based on the search result features of each reference web page in the category, determining the sample search result features corresponding to the category includes: For each parent node of at least one parent node corresponding to each category of reference web pages, aggregating sub-result features corresponding to at least one search result directly connected to the parent node in each reference web page in the category to obtain a search result feature corresponding to the parent node; and Determine, based on the search result feature corresponding to each parent node of the at least one parent node, a sample search result feature corresponding to the category; Based on the sample frame feature and the sample search result feature, obtain the sample feature corresponding to the category; and A sample library is determined based on sample features corresponding to each category in the at least one category.

2. The method according to claim 1, further comprising: For each reference web page among the multiple reference web pages, in response to the search result data corresponding to the reference web page including text data consisting of multiple leaf nodes of type text that are directly connected to the same parent node and the text data is split into the multiple leaf nodes due to a preset style, the multiple leaf nodes are spliced ​​and a leaf node obtained by splicing is labeled with the preset style.

3. The method according to claim 1 or 2, wherein: The attribute data corresponding to a leaf node whose type is text includes the type, label and value of the leaf node, and the attribute data corresponding to a leaf node whose type is non-text includes the type and label of the leaf node.

4. A web page testing method, comprising: Obtain the sample library and the web page data to be tested corresponding to the web page to be tested; Identifying search result data in the web page data to be tested, and using data in the web page data to be tested other than the search result data as framework data; Comparing the framework data corresponding to the web page to be tested with the sample framework features in each sample feature in the sample library; In response to the framework data corresponding to the web page to be tested conforming to a sample framework feature in a sample feature in the sample library, comparing the search result data corresponding to the web page to be tested with a sample search result feature in the sample feature; as well as In response to the search result data corresponding to the web page to be tested meeting any sub-result feature of the sample search result feature, determining that the web page to be tested passes the test, Wherein, the sample library is obtained according to any one of the methods of claims 1-3.

5. The method according to claim 4, further comprising: In response to the webpage to be tested failing the test, extracting a frame feature and a search result feature of the webpage to be tested; Based on the framework characteristics of the web page to be tested and the search result characteristics, updating the sample library; as well as The web page to be tested is tested based on the updated sample library.

6. A device for establishing a sample library for testing web pages, comprising: A first acquisition module is configured to acquire web page data corresponding to a plurality of reference web pages respectively; A first identification module is configured to identify, for each reference web page among the plurality of reference web pages, search result data in the web page data corresponding to the reference web page, and use data in the web page data other than the search result data as framework data; The first extraction module is configured to extract the frame features of the reference web page based on the frame data corresponding to the reference web page, wherein the first extraction module is further configured to: For each reference web page among the plurality of reference web pages, extracting leaf node data in the frame data corresponding to the reference web page as the frame feature of the reference web page, wherein the leaf node data includes attribute data representing the attributes of the leaf node and path data representing the position of the leaf node in the DOM tree structure corresponding to the reference web page; The second extraction module is configured to extract the search result features of the reference webpage based on the search result data corresponding to the reference webpage, wherein the second extraction module is further configured to: Extracting leaf node data from the search result data corresponding to the reference web page as search result features of the reference web page; A fifth determination module is configured to determine, for each reference web page among the plurality of reference web pages, at least one parent node directly connected to the search result data corresponding to the reference web page in the DOM tree structure; a sixth determination module configured to determine, for each parent node of the at least one parent node, at least one search result directly connected to the parent node; and A seventh determination module is configured to determine a sub-result feature corresponding to each of the at least one search result in the search result feature of the reference webpage; a classification module configured to classify one or more reference web pages having the same framework feature among the plurality of reference web pages into the same category to obtain at least one category; The first determining module is configured to, for each of the at least one category, The same framework features corresponding to the category are used as the sample framework features corresponding to the category; The second determination module is configured to determine the sample search result features corresponding to the category based on the search result features of each reference web page in the category, wherein the second determination module includes: an aggregation unit configured to aggregate, for each parent node of at least one parent node corresponding to each category of reference web pages, sub-result features corresponding to at least one search result directly connected to the parent node in each reference web page in the category, to obtain a search result feature corresponding to the parent node; and a determining unit configured to determine a sample search result feature corresponding to the category based on a search result feature corresponding to each parent node of the at least one parent node; A third determination module is configured to obtain sample features corresponding to the category based on the sample framework features and the sample search result features; and The fourth determination module is configured to determine a sample library based on sample features corresponding to each category in the at least one category.

7. The apparatus according to claim 6, further comprising: The splicing module is configured to, for each reference web page among the multiple reference web pages, in response to the search result data corresponding to the reference web page including text data consisting of multiple leaf nodes of type text that are directly connected to the same parent node and the text data is split into the multiple leaf nodes due to a preset style, splice the multiple leaf nodes and mark a leaf node obtained by splicing with the preset style.

8. The device according to any one of claims 6 or 7, wherein: The attribute data corresponding to a leaf node whose type is text includes the type, label and value of the leaf node, and the attribute data corresponding to a leaf node whose type is non-text includes the type and label of the leaf node.

9. A web page testing device, comprising: A second acquisition module is configured to acquire the sample library and the web page data to be tested corresponding to the web page to be tested; A second identification module is configured to identify search result data in the web page data to be tested, and use data in the web page data to be tested other than the search result data as framework data; A first comparison module is configured to compare the framework data corresponding to the web page to be tested with the sample framework feature in each sample feature in the sample library; A second comparison module is configured to compare the search result data corresponding to the web page to be tested with a sample search result feature in a sample feature in the sample library in response to the framework data corresponding to the web page to be tested conforming to a sample framework feature in the sample library; as well as The eighth determination module is configured to determine that the web page to be tested passes the test in response to the search result data corresponding to the web page to be tested meeting any sub-result feature of the sample search result feature, Wherein, the sample library is obtained according to any one of the methods of claims 1-3.

10. The apparatus according to claim 9, further comprising: an extraction module, configured to extract the framework features and search result features of the web page to be tested in response to the web page to be tested failing the test; An updating module, configured to update the sample library based on the frame features of the web page to be tested and the search result features; as well as The testing module is configured to test the web page to be tested based on the updated sample library.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

13. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images