Automatic testing method, device and equipment for web page and storage medium

Abstract

The embodiment of the invention provides an automatic testing method and device for a webpage, equipment and a storage medium. The method can be used in artificial intelligence field. The automatic testing method comprises the steps that a first page image corresponding to a to-be-tested web page is obtained, the first page image is input into an information extraction model obtained through pre-training, and attribute information of page elements in the web page is obtained, the information extraction model is used for extracting features of the first page image to obtain a heat map feature map of the page image, obtaining attribute information based on the heat map feature map, automatically generating a test case for the webpage according to the attribute information, and automatically testing the webpage based on the test case. By means of the method and device, the accuracy and efficiency of attribute information extraction can be improved, the development and maintenance cost of the test case is reduced, the efficiency of test case generation is improved, and then the efficiency of automatic testing is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an automated testing method, device, equipment and storage medium for web pages. Background Art

[0002] Automated testing is a very important part of modern software development. It can improve testing efficiency, reduce manual errors, and ensure the quality and stability of software. How to conduct efficient automated testing is particularly important.

[0003] In the related art, a person manually clicks on a web page to obtain element locations, and the element location operations are connected in series to obtain a test case for the web page, and an automated test of the web page is performed according to the test case.

[0004] Using the above technology, there are problems such as high test case development and maintenance costs and low testing efficiency. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, device and storage medium for automated testing of web pages, which are used to achieve high testing efficiency and low test case development and maintenance costs.

[0006] In a first aspect, an embodiment of the present application provides an automated testing method for a web page, comprising:

[0007] Obtaining a first page image corresponding to the web page to be tested;

[0008] Inputting the first page image into a pre-trained information extraction model to obtain attribute information of page elements in the web page, wherein the information extraction model is used to extract features of the first page image to obtain a heat map feature map of the page image, and obtain attribute information based on the heat map feature map;

[0009] Generate test cases for web pages based on attribute information;

[0010] Perform automated testing on web pages based on test cases.

[0011] In a possible implementation, the attribute information includes location coordinates, image category, and text content, and the information extraction model is specifically used to:

[0012] Extracting features of the first page image to obtain a heat map feature map of the page image;

[0013] According to the peak coordinates in the heat map feature map, pixel positioning is performed on the page element in the first page image to obtain the position coordinates of the page element;

[0014] Perform image classification on the heat map feature map to obtain the image category corresponding to the page element;

[0015] Perform text recognition on the heat map feature map to obtain the text content corresponding to the page elements.

[0016] In a possible implementation, the attribute information further includes page element type, function description, and result assertion, and the information extraction model is further used to:

[0017] Perform semantic analysis on the word vectors of the text content;

[0018] Classify the semantic analysis results to obtain the page element types corresponding to the page elements;

[0019] Apply similarity algorithm to fuse text content and image categories to obtain functional descriptions of page elements.

[0020] Get the result assertion of the page element based on the element type and function description.

[0021] In a possible implementation, before semantic analysis is performed on the word vector of the text content, the process further includes: using a one-hot encoding method to word vectorize the text content to obtain the word vector of the text content.

[0022] In a possible implementation, generating a test case for a web page according to the attribute information includes:

[0023] For each page element in the web page, build a test case corresponding to the page element according to the attribute information of the page element;

[0024] Combine multiple test cases to obtain test cases for web pages.

[0025] In a possible implementation, constructing a test case corresponding to a page element according to the attribute information of the page element includes:

[0026] According to the page element type of the page element, construct a first test case corresponding to the triggering action of the page element and a second test case corresponding to the page element type;

[0027] According to the position coordinates of the page elements, a third test case corresponding to the element positioning is constructed;

[0028] According to the result assertion of the page element, construct a fourth test case corresponding to the result assertion;

[0029] According to the first test case, the second test case, the third test case and the fourth test case, test cases corresponding to the page elements are obtained.

[0030] In a possible implementation, the information extraction model is obtained in the following manner:

[0031] Acquire a training sample, where the training sample includes a second page image corresponding to a sample web page and annotation attribute information of page elements in the sample web page, where the annotation attribute information is obtained based on a human-computer interaction method and / or an automatic annotation method;

[0032] Inputting the second page image into the information extraction model to extract information, and obtaining actual attribute information output by the information extraction model;

[0033] Based on the loss function, determine the loss value of the actual attribute information relative to the labeled attribute information, where the loss function is determined based on the cosine distance and the intersection-and-union ratio, the cosine distance is the cosine distance of the word vector of the text content, and the intersection-and-union ratio is the intersection-and-union ratio of the position coordinates;

[0034] Based on the loss value, adjust the model parameters of the information extraction model.

[0035] In a second aspect, the present application provides an automated testing device for a web page, comprising:

[0036] An acquisition module, used to acquire a first page image corresponding to a web page to be tested;

[0037] A processing module, used to input the first page image into a pre-trained information extraction model to obtain attribute information of page elements in the web page, wherein the information extraction model is used to extract features of the first page image to obtain a heat map feature map of the page image, and obtain attribute information based on the heat map feature map;

[0038] A generation module, used to generate test cases for web pages according to attribute information;

[0039] The test module is used to perform automated testing on web pages based on test cases.

[0040] In a possible implementation, the attribute information includes location coordinates, image categories, and text content, and the information extraction model is specifically used to: extract features of the first page image to obtain a heat map feature map of the page image; perform pixel positioning of page elements in the first page image based on peak coordinates in the heat map feature map to obtain location coordinates of the page elements; perform image classification on the heat map feature map to obtain image categories corresponding to the page elements; perform text recognition on the heat map feature map to obtain text content corresponding to the page elements.

[0041] In a possible implementation, the attribute information also includes a page element type, a functional description, and a result assertion, and the information extraction model is also used to: perform semantic analysis on the word vectors of the text content; classify the semantic analysis results to obtain the page element type corresponding to the page element; apply a similarity algorithm to perform feature fusion on the text content and the image category to obtain the functional description corresponding to the page element; and obtain the result assertion of the page element based on the element type and the functional description.

[0042] In a possible implementation, the processing module is further used to: before performing semantic analysis on the word vector of the text content, use a one-hot encoding method to word vectorize the text content to obtain the word vector of the text content.

[0043] In a possible implementation, the generation module is specifically used to: for each page element in the web page, construct a test case corresponding to the page element according to the attribute information of the page element; and combine multiple test cases to obtain a test case for the web page.

[0044] In a possible implementation, the generation module is also used to: construct a first test case corresponding to the trigger action of the page element and a second test case corresponding to the page element type according to the page element type of the page element; construct a third test case corresponding to the element positioning according to the position coordinates of the page element; construct a fourth test case corresponding to the result assertion according to the result assertion of the page element; and obtain a test case corresponding to the page element according to the first test case, the second test case, the third test case and the fourth test case.

[0045] In one possible implementation, the information extraction model is obtained in the following manner: obtaining a training sample, the training sample including a second page image corresponding to a sample web page and annotated attribute information of page elements in the sample web page, wherein the annotated attribute information is obtained based on a human-computer interaction method and / or an automatic annotation method; inputting the second page image into the information extraction model for information extraction to obtain actual attribute information output by the information extraction model; determining a loss value of the actual attribute information relative to the annotated attribute information based on a loss function, wherein the loss function is determined based on a cosine distance and an intersection-and-union ratio, wherein the cosine distance is the cosine distance of the word vector of the text content, and the intersection-and-union ratio is the intersection-and-union ratio of the position coordinates; and adjusting the model parameters of the information extraction model based on the loss value.

[0046] In a third aspect, an embodiment of the present application provides an automated testing device for a web page, including: a memory, a processor;

[0047] Memory stores computer-executable instructions;

[0048] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0050] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0051] The automated testing method, apparatus, device and storage medium for web pages provided in the embodiments of the present application obtain attribute information of page elements in the web page by inputting the acquired first page image corresponding to the web page to be tested into a pre-trained information extraction model, wherein the information extraction model is used to extract features of the first page image to obtain a heat map feature map of the page image, and obtain attribute information based on the heat map feature map, and automatically extract attribute information of page elements using the information extraction model to improve the accuracy and efficiency of attribute information extraction; automatically generate test cases for web pages based on the attribute information, and perform automated testing on web pages based on the test cases, thereby reducing the cost of test case development and maintenance, improving the efficiency of test case generation, and thereby improving the efficiency of automated testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0053] Figure 1 A schematic diagram of a scenario of an automated testing method for a web page provided by an exemplary embodiment of the present application;

[0054] Figure 2 A schematic diagram of a flow chart of an automated testing method for a web page provided by an exemplary embodiment of the present application;

[0055] Figure 3 A schematic diagram of the structure of an automated testing device for web pages provided by an exemplary embodiment of the present application;

[0056] Figure 4 A schematic diagram of the structure of an automated testing device for web pages provided in an exemplary embodiment of the present application.

[0057] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0060] It should be noted that the automated testing methods, devices, equipment and storage media for web pages of the present application can be used in the field of artificial intelligence technology, and can also be used in any field outside the field of artificial intelligence technology. The application field of the automated testing methods, devices, equipment and storage media for web pages of the present application is not limited.

[0061] First, the terms involved in this application are explained:

[0062] Automated testing of web pages: Test the web pages in an automated way.

[0063] Usually, when browsing the web, there are often some repetitive or publicly recognized pages. For example, for a search box, enter the content in the search box, and then click the search button, and then you will get the results related to the search content; for a navigation bar, click the navigation bar title, and you will jump to the page related to the navigation bar. Therefore, in the process of web page testing, there are often these repetitive or publicly recognized parts. This part of the content does not require too much business requirements description, and the page can be tested only based on common sense. In the related art, if there is an automated test case for a web page, it is necessary to manually click to obtain the element location, and the element location operation is connected in series to obtain an automated test case, which often requires more manual development costs and maintenance costs, and the test case generation efficiency is low, resulting in low automated testing.

[0064] In response to the above technical problems, the present application provides a method, device, equipment and storage medium for automated testing of web pages. The method inputs a first page image corresponding to the web page to be tested into a pre-trained information extraction model to obtain attribute information of page elements in the web page. According to the attribute information, test cases for the web page are automatically generated. Based on the test cases, the web page is automatically tested, which reduces the cost of test case development and maintenance, improves the efficiency of test case generation, and thereby improves the efficiency of automated testing.

[0065] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0066] Figure 1 A scenario diagram of an automated testing method for web pages provided for an exemplary embodiment of the present application. As shown in the figure, a first page image corresponding to a web page to be tested is obtained by an acquisition device 11, and the first page image is input into an image processing device 12 deployed with an information extraction model. The image processing device 12 extracts features of the first page image through the information extraction model to obtain a heat map feature map of the page image, and obtains attribute information of page elements based on the heat map feature map. According to the attribute information, a test case for the web page is automatically generated. The test device 13 obtains the test case from the image processing device 12, and the test device 13 performs automated testing on the web page based on the test case.

[0067] It should be noted that Figure 1 The application scenarios shown are only examples. Among them, any two or three of the acquisition device 11, the image processing device 12 and the test device 13 can be the same device, and there is no limit on the number of various devices. For example, for the same first page image, multiple image processing devices can be used to extract page image features, and heat map feature maps of the corresponding page images and page element attribute information can be output respectively; or, for the same test case, multiple test devices can be used to jointly perform automated testing of the web page to be tested, etc., and specific settings need to be made according to actual needs.

[0068] in addition, Figure 1 The acquisition device 11, image processing device 12 and test device 13 are also illustrated by taking a computer as an example, and can also be a wearable device, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a server cluster, etc.

[0069] Figure 2 The following is a flow chart of an automated testing method for web pages provided by an exemplary embodiment of the present application. Figure 2 As shown, the automated testing method for web pages provided by the exemplary embodiment of the present application includes:

[0070] S201: Obtain a first page image corresponding to a web page to be tested.

[0071] The first page image may also be referred to as a snapshot or a screenshot. For example, based on automated tools and scripts, a screenshot of the web page to be tested is obtained. For example, Selenium is a tool for automating browser operations, and a screenshot of the web page to be tested can be obtained programmatically; or, an API interface corresponding to a web page screenshot service provided by a third party is used to obtain a screenshot of the web page through the API interface; or, a crawler tool is used to obtain a snapshot of the web page to be tested; and so on.

[0072] In an embodiment of the present application, the first page image can be generated online or in advance. For example, it can be pre-generated and stored in a storage device. When the web page to be tested is subjected to automated testing, the execution body of the automated testing method for the web page provided in the embodiment of the present application executes the first page image from the storage device.

[0073] The embodiment of the present application does not limit the method of generating and acquiring the first page image.

[0074] S202. Input the first page image into a pre-trained information extraction model to obtain attribute information of page elements in the web page, wherein the information extraction model is used to extract features of the first page image to obtain a heat map feature map of the page image, and obtain attribute information based on the heat map feature map.

[0075] The information extraction model can be understood as a deep learning model that can effectively extract image features. For example, the features of the first page image are extracted through the convolutional layer of the convolutional neural network algorithm to generate a feature map. The feature map contains the spatial hierarchical information of the first page image and is the basis for generating a heat map. Grad-CAM (Gradient-weighted Class Activation Mapping) is further used to calculate the gradient information of the convolutional layer to generate a heat map feature map. The heat map feature map can be used to identify and visualize important areas in the image. Based on the heat map feature map, the attribute information of the page elements can be obtained in combination with relevant algorithms.

[0076] S203: Generate a test case for the web page according to the attribute information.

[0077] It can be understood that in automated testing, test cases are expressed in the form of scripts, and the writing of test scripts requires obtaining the attribute information of page elements as parameters. For example, in order to improve the efficiency of test script generation, a test script framework can be preset in the image processing device 12. Based on the test script framework, combined with the attribute information, test cases for web pages can be quickly generated.

[0078] S204: Based on the test cases, perform automated testing on the web pages.

[0079] The embodiments of the present application utilize an information extraction model obtained through pre-training to automatically extract attribute information of page elements, thereby improving the accuracy and efficiency of attribute information extraction, and automatically generating test cases for web pages based on the attribute information. Based on the test cases, the web pages are automatically tested, thereby reducing the cost of test case development and maintenance, improving the efficiency of test case generation, and thereby improving the efficiency of automated testing.

[0080] In some embodiments, the attribute information includes position coordinates, image categories and text content, and the information extraction model is specifically used to: extract features of the first page image to obtain a heat map feature map of the page image; perform pixel positioning of page elements in the first page image based on peak coordinates in the heat map feature map to obtain position coordinates of the page elements; perform image classification on the heat map feature map to obtain image categories corresponding to the page elements; perform text recognition on the heat map feature map to obtain text content corresponding to the page elements.

[0081] Extracting attribute information of page elements from a web page involves the combination of computer vision and natural language processing technology. By analyzing the peak coordinates in the heat map feature map, the position of the page element in the first page image is determined. Furthermore, the bounding box of the page element can be accurately located by using techniques such as non-maximum suppression, thereby accurately obtaining the position coordinates of the page element. The convolutional neural network algorithm can classify the page elements into image categories, which can help identify different types of images such as icons, advertisements, and product images on the first page image. The text area in the heat map feature map is identified using a text detection algorithm, such as CTPN (Connectionist Text Proposal Network) and EAST (Efficient and Accurate Scene Text Detector) based on deep learning, which can generate a bounding box of the text area, identify the part of the first page image that may contain text, and apply techniques such as optical character recognition to the detected text area to obtain the text content. Optionally, the extracted text content can also be further analyzed and processed using natural language processing technology.

[0082] In the embodiment of the present application, the position coordinates, image categories and text content of page elements on a web page are extracted through a pre-trained information extraction model, thereby improving the accuracy and efficiency of the extraction of position coordinates, image categories and text content.

[0083] In a possible implementation, the attribute information also includes a page element type, a functional description, and a result assertion, and the information extraction model is also used to: perform semantic analysis on the word vectors of the text content; classify the semantic analysis results to obtain the page element type corresponding to the page element; apply a similarity algorithm to perform feature fusion on the text content and the image category to obtain the functional description corresponding to the page element; and obtain the result assertion of the page element based on the element type and the functional description.

[0084] The text content of the page element can be represented as a continuous word vector. By performing semantic analysis on the meaning of the words expressed in the text content, the key topics and intentions in the text can be identified. The key topics and intentions are further analyzed and classified to determine the page element type, such as buttons, links, input boxes, etc. For example, if the page element type is a button, the corresponding text content can be button or a similar description.

[0085] For example, a similarity algorithm (such as cosine similarity, Euclidean distance, etc.) is used to organically integrate the text information type (including the text content of the page element type) and the image feature information (image category) of the page element. By combining the text information and the image information, the functional role of the page element is analyzed, that is, a functional description of the page element is generated.

[0086] Furthermore, according to the element type and the functional description, a result assertion of the page element is generated. For example, the assertion may be “the element should be a clickable button” or “the element should display specific text”.

[0087] In the embodiment of the present application, a pre-trained information extraction model is used to extract page element types, function descriptions, and result assertions of page elements on a web page, thereby improving the accuracy and efficiency of the extraction of page element types, function descriptions, and result assertions.

[0088] In some embodiments, before semantic analysis is performed on the word vectors of the text content, the process further includes: using a one-hot encoding method to word vectorize the text content to obtain the word vector of the text content.

[0089] Among them, One-Hot Representation is a common method to convert categorical data into numerical data. In natural language processing, One-Hot Representation can be used to convert words in a text into word vectors. For example, the text content "apple banana" will be converted into `[[1,0,0],[0,1,0]]` through One-Hot Representation.

[0090] In an embodiment of the present application, the text content is word-vectorized through one-hot encoding to obtain word vectors for the text content, thereby ensuring that all words are completely retained, so that the information extraction model obtained through pre-training can clearly identify and distinguish different words, thereby improving the accuracy of the information extraction model in recognizing the attribute information of page elements.

[0091] In some embodiments, word embedding technology (such as Word2Vec, GloVe, BERT, etc.) can also be used to vectorize the text content to obtain the word vector of the text content.

[0092] In some embodiments, test cases for web pages are generated based on attribute information, including: for each page element in the web page, based on the attribute information of the page element, constructing a test case corresponding to the page element; combining multiple test cases to obtain a test case for the web page.

[0093] Among them, the generation of test cases (scripts) is generated in the form of a combination of multiple operation steps, and each operation step consists of four parts: trigger action, page element type, input value and result assertion.

[0094] By way of example, in one implementation, constructing test cases corresponding to page elements based on attribute information of page elements may include: constructing a first test case corresponding to a trigger action of the page element and a second test case corresponding to the page element type based on the page element type of the page element; constructing a third test case corresponding to element positioning based on the position coordinates of the page element; constructing a fourth test case corresponding to a result assertion based on a result assertion of the page element; and obtaining a test case corresponding to the page element based on the first test case, the second test case, the third test case and the fourth test case.

[0095] In this implementation, the attribute information can be substituted as a parameter into a pre-set test script template. For example, according to different page element types, the test script template corresponding to the page element type is matched, and the position coordinates, result assertion, trigger action and page element type of the page element are substituted into the script template, and different input values ​​can be preset in the script template. For example, the trigger action and page element type of the page element are substituted into the script template, and the first test case corresponding to the trigger action of the page element and the second test case corresponding to the page element type are constructed. The position coordinates are substituted into the script template, and the third test case corresponding to the element positioning is constructed. The result assertion is substituted into the script template, and the fourth test case corresponding to the result assertion is constructed. When the position coordinates, result assertion, trigger action and page element type of the page element are substituted into the script template, the test case corresponding to the page element can be obtained.

[0096] A web page contains several page elements. Based on the test cases of the page elements obtained above, the test cases of multiple page elements are combined to obtain the test cases for the web page.

[0097] In some embodiments, the information extraction model is obtained by:

[0098] Step 1.1, obtaining training samples, wherein the training samples include a second page image corresponding to a sample web page and annotation attribute information of page elements in the sample web page, wherein the annotation attribute information is obtained based on a human-computer interaction method and / or an automatic annotation method.

[0099] The method of obtaining the second page image corresponding to the sample web page is the same as the method of obtaining the first page image corresponding to the web page to be tested, which has been described in S201 and will not be repeated here.

[0100] For example, the second page image corresponding to the massive sample web page is obtained based on big data, and then the attribute information of the page elements in the second page image is annotated. In one implementation, the text content, element type, image category, location coordinates, function description, trigger action and result assertion of the page elements in the second page image are manually annotated based on annotation tools (such as LabelImg, RectLabel, etc.) or based on the development of a customized annotation platform; in another implementation, the attribute information data of the page elements in the second page image is automatically annotated based on a machine learning model, or predefined rules or templates; in another implementation, the large-scale second page image is preliminarily processed using an automatic annotation method, and then the annotation quality is improved through manual review and correction.

[0101] Step 1.2: Input the second page image into the information extraction model to extract information, and obtain the actual attribute information output by the information extraction model.

[0102] Step 1.3: Based on the loss function, determine the loss value of the actual attribute information relative to the labeled attribute information, where the loss function is determined based on the cosine distance and the intersection-and-union ratio. The cosine distance is the cosine distance of the word vector of the text content, and the intersection-and-union ratio is the intersection-and-union ratio of the position coordinates.

[0103] Among them, the loss function is used to quantify the difference between the model prediction and the actual value. The intersection of the union of the position coordinates is regarded as the intersection of the union of the page element position, which is an indicator used to evaluate the overlap degree of two bounding boxes (such as the predicted box and the real box). In the detection task, the intersection of the union is used to measure the matching degree between the predicted box and the real box. A higher intersection of the union value indicates a better match. The cosine distance is used to measure the similarity between two word vectors. In multimodal tasks, the cosine distance of word vectors can be used to measure the similarity between text descriptions and image content.

[0104] The intersection-and-union ratio of the element position and the cosine distance of the word vector are used as parameters to calculate the loss function.

[0105] Step 1.4: Adjust the model parameters of the information extraction model according to the loss value.

[0106] Based on the calculated loss value, the model parameters of the information extraction model are updated through optimization algorithms (such as gradient descent, Adam, etc.) to minimize the total loss. The information extraction model is iteratively trained on the training samples until the loss converges.

[0107] By properly selecting and calculating the loss function, the information extraction model can be continuously optimized during the training process, gradually improving the accuracy and reliability of the prediction.

[0108] In the embodiment of the present application, the intersection-and-union ratio of element positions and the cosine distance of word vectors are combined as parameters of the loss function, and the model parameters of the information extraction model are continuously optimized through an optimization algorithm, thereby improving the information extraction model's ability to understand and recognize page element attribute information, improving recognition accuracy and efficiency, and providing important support for subsequent improvement of the efficiency, accuracy of test case generation and automated testing efficiency.

[0109] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0110] Figure 3 This is a schematic diagram of the structure of the automated testing device for web pages provided in this application. Figure 3As shown, the automatic testing device 30 for web pages includes an acquisition module 31, a processing module 32, a generation module 33 and a testing module 34. Among them:

[0111] An acquisition module 31 is used to acquire a first page image corresponding to a web page to be tested;

[0112] A processing module 32 is used to input the first page image into a pre-trained information extraction model to obtain attribute information of page elements in the web page, wherein the information extraction model is used to extract features of the first page image to obtain a heat map feature map of the page image, and obtain attribute information based on the heat map feature map;

[0113] A generating module 33, used to generate a test case for a web page according to the attribute information;

[0114] The testing module 34 is used to perform automated testing on web pages based on test cases.

[0115] In a possible implementation, the attribute information includes location coordinates, image categories, and text content, and the information extraction model is specifically used to: extract features of the first page image to obtain a heat map feature map of the page image; perform pixel positioning of page elements in the first page image based on peak coordinates in the heat map feature map to obtain location coordinates of the page elements; perform image classification on the heat map feature map to obtain image categories corresponding to the page elements; perform text recognition on the heat map feature map to obtain text content corresponding to the page elements.

[0116] In a possible implementation, the attribute information also includes a page element type, a functional description, and a result assertion, and the information extraction model is also used to: perform semantic analysis on the word vectors of the text content; classify the semantic analysis results to obtain the page element type corresponding to the page element; apply a similarity algorithm to perform feature fusion on the text content and the image category to obtain the functional description corresponding to the page element; and obtain the result assertion of the page element based on the element type and the functional description.

[0117] In a possible implementation, the processing module 32 is further used to: before performing semantic analysis on the word vector of the text content, use a one-hot encoding method to word vectorize the text content to obtain the word vector of the text content.

[0118] In a possible implementation, the generation module 33 is specifically used to: for each page element in the web page, construct a test case corresponding to the page element according to the attribute information of the page element; and combine multiple test cases to obtain a test case for the web page.

[0119] In a possible implementation, the generation module 33 is also used to: construct a first test case corresponding to the trigger action of the page element and a second test case corresponding to the page element type according to the page element type of the page element; construct a third test case corresponding to the element positioning according to the position coordinates of the page element; construct a fourth test case corresponding to the result assertion according to the result assertion of the page element; and obtain a test case corresponding to the page element according to the first test case, the second test case, the third test case and the fourth test case.

[0120] In one possible implementation, the information extraction model is obtained in the following manner: obtaining a training sample, the training sample including a second page image corresponding to a sample web page and annotated attribute information of page elements in the sample web page, wherein the annotated attribute information is obtained based on a human-computer interaction method and / or an automatic annotation method; inputting the second page image into the information extraction model for information extraction to obtain actual attribute information output by the information extraction model; determining a loss value of the actual attribute information relative to the annotated attribute information based on a loss function, wherein the loss function is determined based on a cosine distance and an intersection-and-union ratio, wherein the cosine distance is the cosine distance of the word vector of the text content, and the intersection-and-union ratio is the intersection-and-union ratio of the position coordinates; and adjusting the model parameters of the information extraction model based on the loss value.

[0121] The automated testing device for web pages provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0122] Figure 4 A schematic diagram of the structure of the automated testing device for web pages provided in the present application. As shown in 4, the automated testing device 40 for web pages includes: at least one processor 41 and a memory 42. Among them, the memory 42 is used to store instructions, and the processor 41 is used to call the instructions in the memory to execute the method steps provided in the above embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0123] Optionally, the memory 42 may be independent or integrated with the processor 41 .

[0124] The memory 42 may include a random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0125] The processor 41 can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0126] Optionally, the automated testing device 40 for the web page may further include a communication interface 43. In a specific implementation, if the communication interface 43, the memory 42 and the processor 41 are implemented independently, the communication interface 43, the memory 42 and the processor 41 may be interconnected through a bus and communicate with each other. The system bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0127] Optionally, in a specific implementation, if the communication interface 43, the memory 42 and the processor 41 are integrated on a chip, the communication interface 43, the memory 42 and the processor 41 can communicate through an internal interface.

[0128] The implementation principle and technical effects of the automatic testing device for web pages provided in this embodiment can be found in the aforementioned embodiments and will not be described in detail here.

[0129] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0130] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0131] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0132] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0133] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0134] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0136] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0137] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0138] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for automated testing of web pages, characterized in that: include: Obtaining a first page image corresponding to the web page to be tested; Inputting the first page image into a pre-trained information extraction model to obtain attribute information of page elements in the webpage, wherein the information extraction model is used to extract features of the first page image to obtain a heat map feature graph of the page image, and obtaining the attribute information based on the heat map feature graph; Generating a test case for the web page according to the attribute information; Based on the test case, the web page is automatically tested.

2. The automated testing method according to claim 1, characterized in that: The attribute information includes location coordinates, image category and text content, and the information extraction model is specifically used for: Extracting features of the first page image to obtain a heat map feature map of the page image; Perform pixel positioning on a page element in the first page image according to the peak coordinates in the heat map feature map to obtain position coordinates of the page element; Performing image classification on the heat map feature map to obtain image categories corresponding to the page elements; Text recognition is performed on the heat map feature map to obtain text content corresponding to the page element.

3. The automated testing method according to claim 2, characterized in that: The attribute information also includes page element type, function description and result assertion, and the information extraction model is also used to: Performing semantic analysis on the word vectors of the text content; Classifying the semantic analysis results to obtain page element types corresponding to the page elements; Applying a similarity algorithm to perform feature fusion on the text content and the image category to obtain a functional description corresponding to the page element; According to the element type and the functional description, a result assertion of the page element is obtained.

4. The automated testing method according to claim 3, characterized in that: Before the semantic analysis of the word vector of the text content is performed, the method further includes: The text content is word-vectorized using a one-hot encoding method to obtain a word vector for the text content.

5. The automated testing method according to any one of claims 1 to 4, characterized in that: Generating a test case for the web page according to the attribute information includes: For each page element in the web page, construct a test case corresponding to the page element according to the attribute information of the page element; Combine a plurality of the test cases to obtain a test case for the web page.

6. The automated testing method according to claim 5, characterized in that: The step of constructing a test case corresponding to the page element according to the attribute information of the page element includes: According to the page element type of the page element, construct a first test case corresponding to the triggering action of the page element and a second test case corresponding to the page element type; According to the position coordinates of the page elements, a third test case corresponding to the element positioning is constructed; According to the result assertion of the page element, construct a fourth test case corresponding to the result assertion; According to the first test case, the second test case, the third test case and the fourth test case, a test case corresponding to the page element is obtained.

7. The automated testing method according to any one of claims 1 to 4, characterized in that: The information extraction model is obtained in the following way: Acquire a training sample, wherein the training sample includes a second page image corresponding to a sample web page and annotation attribute information of page elements in the sample web page, wherein the annotation attribute information is obtained based on a human-computer interaction method and / or an automatic annotation method; Inputting the second page image into the information extraction model to extract information, and obtaining actual attribute information output by the information extraction model; Based on the loss function, determine the loss value of the actual attribute information relative to the labeled attribute information, wherein the loss function is determined according to the cosine distance and the intersection-and-union ratio, the cosine distance is the cosine distance of the word vectors of the text content, and the intersection-and-union ratio is the intersection-and-union ratio of the position coordinates; According to the loss value, the model parameters of the information extraction model are adjusted.

8. An automated testing device for web pages, characterized in that: include: An acquisition module, used to acquire a first page image corresponding to a web page to be tested; a processing module, configured to input the first page image into a pre-trained information extraction model to obtain attribute information of page elements in the web page, wherein the information extraction model is configured to extract features of the first page image to obtain a heat map feature graph of the page image, and obtain the attribute information based on the heat map feature graph; A generating module, used for generating a test case for the web page according to the attribute information; The testing module is used to perform automated testing on the web page based on the test case.

9. An automated testing device for web pages, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed.

Images