Webpage element acquisition method and assembly, computer equipment and storage medium

Abstract

The invention relates to the technical field of front-end automatic testing, in particular to a webpage element obtaining method and assembly, computer equipment and a storage medium. The method comprises the following steps: acquiring a webpage image and a template element image; matching the template element image with the webpage image to obtain a matching area, and calculating a matching value of the matching area; if the matching value is smaller than a preset matching threshold value, returning to the step of matching the template element image with the webpage image; if the matching value is greater than or equal to a preset matching threshold value, taking the matching area as a target area, and calculating a center point coordinate of the target area; and obtaining a target element located in the target area according to the center point coordinate. By adopting the method, the obtaining efficiency of the webpage elements can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of front-end automated testing, and in particular to a web page element acquisition method, component, computer device and storage medium. Background Art

[0002] Web page elements refer to all the components that make up a web page. They are visual objects defined by Hyper Text Markup Language (HTML) code and rendered by the browser. For example, web page elements can be text, images, buttons, links, form input fields, tables, videos, and audio. Each web page element has its own specific function and style, and they work together to create a complete and feature-rich web page.

[0003] Front-end automated testing is an important means to ensure the functional correctness of web applications. The testing tool verifies whether the web page functions normally by obtaining these elements and simulating user operations. In front-end automated testing, obtaining elements within the web page is crucial, which is related to the accuracy and efficiency of web page functional testing.

[0004] The related web page element acquisition method mainly relies on manual work, and each element in the web page needs to be checked one by one, resulting in low efficiency in element acquisition and high labor costs, which in turn causes low efficiency in element testing. Summary of the invention

[0005] Based on this, it is necessary to provide a web page element acquisition method, component, computer device and storage medium to improve the efficiency of acquiring web page elements in response to the above technical problems.

[0006] In a first aspect, a method for obtaining a web page element is provided, the method comprising:

[0007] Get web page images and template element images;

[0008] Performing template matching between the template element image and the web page image to obtain a matching area, and calculating a matching value of the matching area;

[0009] If the matching value is less than a preset matching threshold, returning to the step of performing template matching between the template element image and the web page image;

[0010] If the matching value is greater than or equal to a preset matching threshold, the matching area is taken as a target area, and the coordinates of the center point of the target area are calculated;

[0011] A target element located in the target area is acquired according to the center point coordinates.

[0012] In combination with the first aspect, in a first possible implementation of the first aspect, performing template matching on the template element image and the web page image to obtain a matching area includes:

[0013] Extracting the outline of the template element in the template element image to obtain a template outline map;

[0014] Extracting the outline of each web page element in the web page image to obtain a web page outline image including the outlines of each web page element;

[0015] Filtering out web page element contours similar to the template contour map in the web page contour map as target contours, and taking the area occupied by the target contour in the web page contour map as the to-be-matched area;

[0016] The template contour image is subjected to template matching in the area to be matched to obtain a matching area.

[0017] In combination with the first aspect, in a second possible implementation of the first aspect, performing template matching on the template element image and the web page image to obtain a matching area includes:

[0018] Extracting features of template elements in the template element image to obtain a template feature map;

[0019] Extracting features of each web page element in the web page image to obtain a web page feature map including features of each web page element;

[0020] Screening out web page element features similar to the template feature map in the web page feature map as target features, and taking the area occupied by the target features in the web page feature map as the to-be-matched area;

[0021] The template feature map is subjected to template matching in the area to be matched to obtain a matching area.

[0022] In combination with the first aspect, in a third possible implementation manner of the first aspect, performing template matching on the template element image and the web page image to obtain a matching area includes:

[0023] Get the preset sliding step size;

[0024] Obtain a sliding window having the same size as the template element image;

[0025] The sliding window is slid on the web page image according to the sliding step length to obtain a matching area.

[0026] In combination with the first aspect, in a fourth possible implementation manner of the first aspect, calculating the coordinates of the center point of the target area includes:

[0027] Establishing a reference pixel coordinate system according to the plane where the web page image is located;

[0028] In the reference pixel coordinate system, obtaining the coordinates of any corner of the target area;

[0029] Obtaining the pixel width and pixel height of the target area;

[0030] Coordinate calculation is performed according to the coordinates of the arbitrary angle, the pixel width and the pixel height to obtain the coordinates of the center point of the target area.

[0031] In combination with the first aspect, in a fifth possible implementation of the first aspect, after acquiring the target element located in the target area according to the center point coordinates, the method further includes:

[0032] Obtaining a label of a template element corresponding to the template element image, wherein the label is used to define the type of the element;

[0033] Get the label of the target element;

[0034] If the label of the template element is consistent with the label of the target element, the target element is valid;

[0035] If the label of the template element is inconsistent with the label of the target element, the target element is not valid.

[0036] In combination with the first aspect, in a sixth possible implementation of the first aspect, after acquiring the target element located in the target area according to the center point coordinates, the method further includes:

[0037] Draw a border and a center point for the target element;

[0038] The hypertext markup language content and attribute information of the target element are obtained, and the hypertext markup language content and the attribute information are stored in a preset file.

[0039] In a second aspect, a web page element acquisition component is provided, the component comprising:

[0040] An image acquisition unit, used to acquire web page images and template element images;

[0041] An element matching unit, used to perform template matching between the template element image and the web page image to obtain a matching area, and calculate a matching value of the matching area;

[0042] The element matching unit is further configured to return to the step of performing template matching between the template element image and the web page image if the matching value is less than a preset matching threshold;

[0043] The element matching unit is further configured to, if the matching value is greater than or equal to a preset matching threshold, use the matching area as a target area and calculate the coordinates of a center point of the target area;

[0044] An element acquisition unit is used to acquire a target element located in the target area according to the center point coordinates.

[0045] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the web page element acquisition method as described in any one of the first aspect or the first to sixth possible implementations of the first aspect are implemented.

[0046] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the web page element acquisition method as described in any one of the first aspect or the first to sixth embodiments of the first aspect are implemented.

[0047] The web page element acquisition method, component, computer device and storage medium described above acquire a web page image and a template element image; perform template matching on the template element image in the web page image to obtain a matching area, and calculate the matching value of the matching area; if the matching value is lower than a preset matching threshold, return to the step of performing template matching on the template element image in the web page image; if the matching value is greater than or equal to the preset matching threshold, use the matching area as the target area, and calculate the coordinates of the center point of the target area; and acquire the target element located in the target area according to the coordinates of the center point. It can be seen that the present application acquires the target element that meets the requirements of the template element image in the web page by means of template matching and coordinate calculation. On the one hand, it can realize automatic acquisition of elements without manually checking each element in the web page one by one, thereby improving the efficiency of element acquisition, reducing labor costs, and thus improving the efficiency of element testing; on the other hand, it can accurately locate the target element on the web page image, and improve the phenomenon of element acquisition errors caused by changes in element position or adjustments to the web page layout. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A diagram of an application environment of a web page element acquisition method in an embodiment;

[0049] Figure 2 A schematic diagram of a flow chart of a method for obtaining web page elements in one embodiment;

[0050] Figure 3 It is a structural block diagram of a web page element acquisition component in one embodiment;

[0051] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] It should be noted that the illustrations provided in this embodiment only illustrate the basic concept of the present application in a schematic manner, and therefore the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0054] The structures, proportions, sizes, etc. illustrated in the drawings in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which this application can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by this application, should still fall within the scope of the technical contents disclosed in this application.

[0055] The directions or positional relationships indicated by the terms "upper", "lower", "left", "right", "middle", "longitudinal", "lateral", "horizontal", "inner", "outer", "radial", "circumferential" and the like in this specification are based on the directions or positional relationships shown in the drawings and are only for the convenience of simplifying the description. They do not indicate or imply that the devices or elements referred to must have a specific direction, be constructed and operate in a specific direction, and therefore cannot be understood as limiting the present application. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0056] In front-end automated testing, the testing tool verifies whether the web page functions normally by acquiring elements in the web page and simulating user operations. Therefore, the acquisition of elements in the web page plays a vital role, which is related to the accuracy and efficiency of web page function testing. However, the relevant technology relies on manpower, resulting in low efficiency in element acquisition and increased labor costs; moreover, commonly used selectors, such as CSS selectors or Extensible Markup Language Path Language (Xml PathLanguage, XPath), are prone to failure or errors when the web page structure changes, resulting in the inability of the test script to run normally.

[0057] To this end, the present application provides a web page element acquisition method, which can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through the network, and the terminal 102 is installed with a front-end automated testing tool, such as Selenium. The terminal 102 matches the acquired template element image in the acquired web page image through the testing tool to obtain a matching area and calculate the matching value of the matching area; if the matching value is less than the preset matching threshold, the step of matching the template element image in the web page image is returned to obtain a matching area again, and the matching value of the re-obtained matching area is calculated; if the matching value is greater than or equal to the matching threshold, the matching area is used as the target area, and the center point coordinates of the target area are calculated; the target element located in the target area is obtained according to the center point coordinates; since the template element image is the standard image of the element to be tested specified in the test case, the target element is the element in the web page image that matches the template element image, and the target element is the expected element to be tested. In this way, accurate positioning and automatic acquisition of web page elements are achieved, the efficiency of element acquisition is improved, the labor cost is reduced, and the testing efficiency of web page elements is improved. The terminal 102 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0058] In one embodiment, Figure 2 As shown, a method for obtaining web page elements is provided, and the method is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0059] Step 201, obtaining a web page image and a template element image.

[0060] In a specific implementation, the web page to be tested is accessed, and after the web page to be tested is fully loaded, a screenshot of the web page to be tested is taken to generate a web page image. Exemplarily, Python and Selenium are used to implement web page screenshot and save: the specified URL address is accessed through the "driver.get(url)" instruction; when the web page is fully loaded, the web page at the current moment is screenshotted through the "driver.save_screenshot()" instruction to obtain the web page image, and save it as a file in a picture format, for example, a PNG format file; the file name of the web page image uses a characteristic string corresponding to a Uniform Resource Locator (URL) address to facilitate subsequent management and identification of each web page; to ensure the clarity and order of the file structure, the web page image is saved in a specified directory.

[0061] Among them, the Uniform Resource Locator is a unified address format used on the Internet to specify any resource on the network (such as web pages, pictures, videos and audio, etc.), which consists of an access protocol (Access Protocol), a host name (Host Name) and a file path (File Path). Internet users access the resources specified by the URL address by entering the URL address.

[0062] Elements are all the components that make up a web page. They are any of a variety of types such as text, images, buttons, links, form input fields, tables, videos, and audios. The template element image is a standard image of the element to be tested specified in the test case. It is generated and saved during the development and design process so that it can be obtained during subsequent tests. The element to be tested can be any element that constitutes the web page to be tested, which depends on the specific content of the test case. For example, the test case will give information such as the element name of the element to be tested. The template element image is used to perform template matching on the web page image to filter out the target element that is consistent with the template element image from the numerous web page elements on the web page image. The target element is the element to be tested indicated by the test case.

[0063] As an example, when developing a web page, the front-end development engineer cuts out the corresponding image of the element according to the known element position and size based on the page layout and structure of the web page and saves it in the storage space. The file name of the image can use the element name; when executing the test case later, the image with the file name consistent with the element name of the element to be tested is retrieved from the storage space and used as the template element image. As another example, after designing the web page interface, the front-end interface designer saves all the elements in the web page through images, so that the corresponding image can be called as the template element image when testing a certain element to be tested later. The element image obtained by the second example is consistent with the design specification, which can improve the accuracy of the element image, especially in complex front-end interface design.

[0064] Step 202: Perform template matching between the template element image and the web page image to obtain a matching area, and calculate a matching value of the matching area.

[0065] The "matching" described in this application refers to the process of finding the best matching result, that is, the process of finding the target element in each web page element of the web page image. Since each web page element is included in the web page image, and the template element image is the standard image of the element to be tested, the template element image is template matched in the web page image, and the standard image of the element to be tested can be template matched with each web page element image in the web page image; after matching, a matching area will be obtained. In order to determine whether the element image in the matching area meets the standard image requirements of the element to be tested, and then determine whether the web page element in the matching area is consistent with the element to be tested corresponding to the test case, it is necessary to calculate the matching degree or similarity of the matching area, so as to subsequently screen out the matching area with a high matching degree according to the matching degree or similarity, and regard the matching area with a high matching degree as an area where potential target elements exist, and the target element is an element that meets the standard image requirements of the given element to be tested.

[0066] As an example, template matching is performed between the template element image and the web page image by window sliding, feature extraction or contour extraction, wherein the template is a known small image, and template matching is to search for a target in a large image, and the target and the template have the same size, direction and image elements. For example, a window of the same size as the template element image is slid on the web page image, and when the window stops sliding, the matching area at that position is obtained; the matching degree of the matching area is evaluated by feature point matching method, histogram matching method or structural similarity index.

[0067] Step 203: If the matching value is less than a preset matching threshold, return to the step of performing template matching between the template element image and the web page image.

[0068] By setting a matching threshold, for example, 0.8, valid matching areas can be screened out, and therefore, the matching degree of the matching area is compared with the matching threshold to ensure that only areas with high matching degrees are regarded as areas where potential target elements are located. When the matching degree does not reach or exceed the matching threshold, it is considered that there is no potential target element in the matching area and re-matching is required. Therefore, at this time, the process returns to step 202 to perform template matching between the template element image and the web page image and calculate the matching value of the matching area until the matching value of the re-matched matching area reaches or exceeds the matching threshold, and the matching is stopped.

[0069] Step 204: If the matching value is greater than or equal to a preset matching threshold, the matching area is used as a target area, and the coordinates of the center point of the target area are calculated.

[0070] When the matching degree reaches or exceeds the matching threshold, it is considered that there is a potential target element in the matching area, so the matching area is used as the target area; in order to accurately locate the target element in the target area, it is necessary to calculate the coordinates of the center point of the target area so that the target element can be located and obtained through the coordinates of the center point. Since the target element is in the target area, the coordinates of the center point of the target area can be calculated through the coordinates of any corner of the target area and the size of the target area.

[0071] Step 205: Acquire the target element located in the target area according to the center point coordinates.

[0072] Get the element at the center point coordinates and use the element as the target element, which is used to execute subsequent test cases. As an example, the target element is obtained through the browser's Document Object Model (DOM) interface using JavaScript's "document.elementFromPoint (center point coordinates)" instruction. If there is an element at the position corresponding to the center point coordinates, a reference to the element is returned. If there is no element at the center point coordinates, "null" is returned, indicating invalidity.

[0073] In the related technology, after designing the web page interface, the coordinates of each element are saved, and the corresponding elements are obtained through the coordinates during testing. However, when the position of the elements on the web page changes and / or the layout of the web page is adjusted, errors may occur in obtaining the corresponding elements through the pre-saved fixed coordinates. The present application adopts the center point coordinates of the target area for real-time matching. The center point coordinates are the coordinates of the target element relative to the web page at the current moment. The corresponding target element is obtained through the center point coordinates, which can improve the phenomenon of element acquisition errors caused by changes in the element coordinate position and / or the web page layout, and improve the accuracy of element acquisition.

[0074] In the above-mentioned web page element acquisition method, template matching is performed on the template element image and the web page image to obtain a matching area, and the matching value of the matching area is calculated; through matching, the matching area with a matching value greater than or equal to the matching threshold is used as the target area; and the corresponding target element is obtained according to the coordinates of the center point of the target area. The present application obtains the target element in the web page by means of template matching and coordinate calculation, which not only realizes the automatic acquisition of web page elements without relying on manpower, improves the efficiency of element acquisition, reduces labor costs, and thus improves the efficiency of element testing; and compared with the related method of acquiring elements according to fixed element coordinates, the present application uses the center point coordinates of the target area after real-time matching as the coordinates of the target element relative to the web page at the current moment, improves the phenomenon of element acquisition errors caused by changes in element position or adjustments to the web page layout, and improves the accuracy of web page element positioning.

[0075] For the step of performing template matching between the template element image and the web page image to obtain the matching area in the above step 202, in a specific implementation, a preset sliding step length is obtained, for example, the sliding step length is a pixel value; a sliding window with the same size as the template element image is obtained; and the sliding window is slid on the web page image according to the sliding step length to obtain the matching area, which is a partial area detected by the sliding window in the web page image. To ensure orderly matching, a certain top corner of the web page image is used as the starting point during sliding, and the pixel width direction or the pixel height direction of the web page image is used as the sliding direction.

[0076] In order to improve the speed of template matching and thus improve the efficiency of element acquisition, contour extraction or feature extraction is performed on the elements in the template element image and the web page image; through contour comparison or feature comparison, areas with similar contours or similar features are obtained; template matching is performed in areas with similar contours or similar features, and there is no need to perform template matching on the entire web page image. In this way, the template matching area is reduced and the speed of template matching is improved.

[0077] Therefore, in a specific implementation, the contours of the template elements in the template element image are extracted to obtain a template contour map; the contours of each web page element in the web page image are extracted to obtain a web page contour map including the contours of each web page element; since the template contour map is the contour map of the template element, the web page contour map includes the contour maps of each web page element, and therefore, the contours of the web page elements similar to the template contour map are screened out in the web page contour map as the target contour, and the area occupied by the target contour in the web page contour map is used as the area to be matched; the area of ​​the area to be matched is not less than the area covered by the target contour, and to facilitate subsequent matching, the area to be matched adopts a rectangular area or a square area; template matching is performed on the template contour map in the area to be matched to obtain a matching area.

[0078] As an example, a pre-generated sliding window is slid in the area to be matched according to a preset sliding step size to obtain a matching area. The preset sliding step size can be a pixel value. The pre-generated sliding window is a window of the same size as the template contour map. The matching area is a partial area detected by the pre-generated sliding window in the area to be matched. To ensure orderly matching, a vertex of the area to be matched is used as the starting point when sliding, and the pixel width direction or pixel height direction of the area to be matched is used as the sliding direction. When the size of the area to be matched is the same as that of the template contour map, the area to be matched is the matching area, and no sliding matching is required.

[0079] In another alternative embodiment, feature extraction is performed on the template elements in the template element image to obtain a template feature map; feature extraction is performed on each web page element in the web page image to obtain a web page feature map including the features of each web page element; since the template feature map is a feature map of the template element, the web page feature map includes the feature maps of each web page element, and therefore, the web page element features similar to the template feature map are screened out in the web page feature map as target features, and the area occupied by the target features in the web page feature map is used as the area to be matched; the area of ​​the area to be matched is not less than the area covered by the target features, and in order to facilitate subsequent matching, the area to be matched adopts a rectangular area or a square area; template matching is performed on the template feature map in the area to be matched to obtain a matching area. In this way, the template matching area can be reduced, the speed of template matching can be accelerated, and the efficiency of element acquisition can be improved.

[0080] Similar to the above example, the pre-generated sliding window is slid in the area to be matched according to the preset sliding step size to obtain the matching area. The preset sliding step size can be a pixel value. The pre-generated sliding window is a window of the same size as the template feature map. The matching area is the partial area detected by the pre-generated sliding window in the area to be matched. To ensure orderly matching, one of the top corners of the area to be matched is used as the starting point when sliding, and the pixel width direction or pixel height direction of the area to be matched is used as the sliding direction. When the size of the area to be matched is the same as that of the template feature map, the area to be matched is the matching area, and no sliding matching is required.

[0081] Regarding the step of calculating the center position of the target area in the above step 204, in a specific implementation, a reference pixel coordinate system is established based on the plane where the web page image is located; in the reference pixel coordinate system, the coordinates of any corner of the target area are obtained; the pixel width and pixel height of the target area are obtained; coordinate calculation is performed based on the coordinates of the any corner, the pixel width and the pixel height to obtain the coordinates of the center point of the target area.

[0082] When establishing a reference pixel coordinate system, any vertex or center point of the web page image can be used as the coordinate origin, the pixel width direction of the web page image can be used as the horizontal coordinate direction or the vertical coordinate direction, and the pixel height direction of the web page image can be used as the vertical coordinate direction or the horizontal coordinate direction. This application does not impose any limitation on this.

[0083] When calculating the coordinates of the center point of the target area, in the constructed reference pixel coordinate system, take the coordinates of any vertex of the target area as the reference, and according to the upper and lower positional relationship between the target area and the vertex, divide the pixel height and pixel width of the target area by 2 and add or subtract the horizontal and vertical coordinates of the vertex to obtain the coordinates of the center point.

[0084] As an example, a reference pixel coordinate system is established with the lower left corner of the web page image as the coordinate origin, the pixel height direction of the web page image as the vertical coordinate direction, and the pixel width direction of the web page image as the horizontal coordinate direction; in the reference pixel coordinate system, the coordinates of the lower left corner of the target area are obtained; the pixel width of the target area is divided by 2 and then superimposed with the horizontal coordinate of the lower left corner, and the pixel height of the target area is divided by 2 and then superimposed with the vertical coordinate of the lower left corner to obtain the coordinates of the center point of the target area.

[0085] The example of establishing a coordinate system based on other corners of the web page image and calculating the center point coordinates based on other corners of the target area is similar to this example and will not be repeated here. In some cases, if the center point coordinates are not calculated, a no match result is returned to reduce the possibility of subsequent steps causing operation errors or exceptions due to lack of center point coordinates.

[0086] In order to ensure that the acquired target element is the element to be tested corresponding to the test case, and then ensure the correct operation of the subsequent test case, it is necessary to determine whether the target element is valid after the target element is acquired. Therefore, in a specific implementation, by acquiring the label of the template element corresponding to the template element image, the label is used to define the type of the element; acquiring the label of the target element; since the label of the template element is saved during the front-end development and design, if the label of the template element is consistent with the label of the target element, the target element is valid, and the target element can be used as a test object to execute the test case in the subsequent execution of the test case; if the label of the template element is inconsistent with the label of the target element, the target element is not valid, and the target element cannot be used as a test object to execute the test case in the subsequent execution of the test case. At this time, the acquired target element is deleted, and the process returns to step 202, re-matching the target area, acquiring the target element of the re-matched target area, and determining whether the re-acquired target element is valid, until the re-acquired target element is valid, the acquisition step of the target element corresponding to the current test case is stopped, and the corresponding target element can be acquired according to the template element image corresponding to other test cases in the subsequent web page element acquisition method of the present application.

[0087] In some specific implementations, after acquiring the target element located in the target area according to the center point coordinates, OpenCV is used to draw the border and center point of the target element to achieve visualization of the target element, so that development engineers or test engineers can visually verify the acquired target element, intuitively confirm the accuracy of the target element, and improve the efficiency of verification and debugging.

[0088] In some specific implementations, after obtaining the target element located in the target area according to the center point coordinates, the hypertext markup language content and attribute information of the target element are obtained, and the hypertext markup language content and attribute information are stored in a preset file, and the storage format adopts a structured format, such as JSON or CSV, etc., for subsequent reading and processing. This not only improves the record of the test results, but also can be used for subsequent analysis, testing and verification, for example, providing a reference for subsequent web page troubleshooting or function verification.

[0089] Among them, Hyper Text Markup Language (HTML) content refers to text, nested elements, comments, inline styles and scripts, etc. These contents together constitute the structure and display content of the web page and are the basis of web page development and design. Attribute information refers to various attributes attached to web page elements. These attributes provide additional information about the elements to help browsers or other tools better understand and process these elements. Attribute information usually includes: ID, class name, style attributes, data attributes, event handling attributes, form attributes, link attributes, image attributes and assistive technology attributes.

[0090] In order to improve the effect of template matching, in some specific implementations, before matching the template element image with the web page image, the template element image and the web page image are respectively converted from color images to grayscale images. Since the grayscale image reduces the complexity of the image, the amount of calculation can be reduced during the processing. Therefore, the speed and accuracy of template matching can be improved by grayscale conversion. As an example, the template element image and the web page image are converted into grayscale images through the "cv2.cvtColor()" instruction of OpenCV.

[0091] Since some methods of image contour extraction and image feature extraction are accompanied by image grayscale processing, if the web page image and template element image have been grayscaled before template matching, there is no need to perform grayscale processing again when performing contour extraction or feature extraction on the web page image and template element image.

[0092] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0093] In one embodiment, Figure 3 As shown, a web page element acquisition component is provided, including: an image acquisition unit, an element matching unit and an element acquisition unit, wherein:

[0094] An image acquisition unit, used to acquire web page images and template element images;

[0095] An element matching unit, used to perform template matching between the template element image and the web page image to obtain a matching area, and calculate a matching value of the matching area;

[0096] The element matching unit is further configured to return to the step of performing template matching between the template element image and the web page image if the matching value is less than a preset matching threshold;

[0097] The element matching unit is further configured to, if the matching value is greater than or equal to a preset matching threshold, use the matching area as a target area and calculate the coordinates of a center point of the target area;

[0098] An element acquisition unit is used to acquire a target element located in the target area according to the center point coordinates.

[0099] In some embodiments, the web page element acquisition component of the present application can be used as an independent tool in conjunction with a front-end automated testing tool. The target element is acquired through the web page element acquisition component, and then the test case corresponding to the target element is completed through the front-end automated testing tool. In some embodiments, the web page element acquisition component of the present application can be embedded in a front-end automated testing tool. The front-end automated testing tool acquires the target element by calling the web page element acquisition component, and then completes the test case corresponding to the target element.

[0100] In a specific embodiment, the element matching unit is used to perform template matching on the template element image and the web page image to obtain a matching area, which specifically includes: performing contour extraction on the template elements in the template element image to obtain a template contour map; performing contour extraction on each web page element in the web page image to obtain a web page contour map including the contours of each web page element; screening out web page element contours similar to the template contour map in the web page contour map as target contours, and taking the area occupied by the target contour in the web page contour map as the area to be matched; performing template matching on the template contour map in the area to be matched to obtain a matching area.

[0101] In a specific embodiment, the element matching unit is used to perform template matching on the template element image and the web page image to obtain a matching area, which specifically includes: performing feature extraction on the template elements in the template element image to obtain a template feature map; performing feature extraction on each web page element in the web page image to obtain a web page feature map including features of each web page element; screening out web page element features similar to the template feature map in the web page feature map as target features, and using the area occupied by the target features in the web page feature map as the area to be matched; performing template matching on the template feature map in the area to be matched to obtain a matching area.

[0102] In a specific embodiment, the element matching unit is used to perform template matching between the template element image and the web page image to obtain a matching area, specifically including: obtaining a preset sliding step size; obtaining a sliding window with the same size as the template element image; sliding the sliding window on the web page image according to the sliding step size to obtain a matching area.

[0103] In a specific implementation, the element matching unit is used to calculate the coordinates of the center point of the target area, specifically including: establishing a reference pixel coordinate system based on the plane where the web page image is located; obtaining the coordinates of any corner of the target area in the reference pixel coordinate system; obtaining the pixel width and pixel height of the target area; performing coordinate calculation based on the coordinates of the any corner, the pixel width and the pixel height to obtain the coordinates of the center point of the target area.

[0104] In a specific embodiment, after the element acquisition unit acquires the target element located in the target area according to the center point coordinates, it is also used to: acquire the label of the template element corresponding to the template element image, the label is used to define the type of the template element; acquire the label of the target element; if the label of the template element is consistent with the label of the target element, the target element is valid; if the label of the template element is inconsistent with the label of the target element, the target element is not valid.

[0105] In a specific embodiment, after the element acquisition unit acquires the target element located in the target area according to the center point coordinates, it is also used to: draw a border and a center point for the target element; acquire the hypertext markup language content and attribute information of the target element, and store the hypertext markup language content and attribute information in a preset file.

[0106] For the specific definition of the web page element acquisition component, please refer to the definition of the web page element acquisition method above, which will not be repeated here. Each unit in the above web page element acquisition component can be implemented in whole or in part by software, hardware and a combination thereof. The above units can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above units.

[0107] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for obtaining web page elements is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0108] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0110] Get web page images and template element images;

[0111] Performing template matching between the template element image and the web page image to obtain a matching area, and calculating a matching value of the matching area;

[0112] If the matching value is less than a preset matching threshold, returning to the step of performing template matching between the template element image and the web page image;

[0113] If the matching value is greater than or equal to a preset matching threshold, the matching area is taken as a target area, and the coordinates of the center point of the target area are calculated;

[0114] A target element located in the target area is acquired according to the center point coordinates.

[0115] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0116] Extracting the outline of the template element in the template element image to obtain a template outline map;

[0117] Extracting the outline of each web page element in the web page image to obtain a web page outline image including the outlines of each web page element;

[0118] Filtering out web page element contours similar to the template contour map in the web page contour map as target contours, and taking the area occupied by the target contour in the web page contour map as the to-be-matched area;

[0119] The template contour image is subjected to template matching in the area to be matched to obtain a matching area.

[0120] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0121] Extracting features of template elements in the template element image to obtain a template feature map;

[0122] Extracting features of each web page element in the web page image to obtain a web page feature map including features of each web page element;

[0123] Screening out web page element features similar to the template feature map in the web page feature map as target features, and taking the area occupied by the target features in the web page feature map as the to-be-matched area;

[0124] The template feature map is subjected to template matching in the area to be matched to obtain a matching area.

[0125] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0126] Get the preset sliding step size;

[0127] Obtain a sliding window having the same size as the template element image;

[0128] The sliding window is slid on the web page image according to the sliding step length to obtain a matching area.

[0129] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0130] Establishing a reference pixel coordinate system according to the plane where the web page image is located;

[0131] In the reference pixel coordinate system, obtaining the coordinates of any corner of the target area;

[0132] Obtaining the pixel width and pixel height of the target area;

[0133] Coordinate calculation is performed according to the coordinates of the arbitrary angle, the pixel width and the pixel height to obtain the coordinates of the center point of the target area.

[0134] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0135] Obtaining a label of a template element corresponding to the template element image, wherein the label is used to define the type of the element;

[0136] Get the label of the target element;

[0137] If the label of the template element is consistent with the label of the target element, the target element is valid;

[0138] If the label of the template element is inconsistent with the label of the target element, the target element is not valid.

[0139] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0140] Draw a border and a center point for the target element;

[0141] The hypertext markup language content and attribute information of the target element are obtained, and the hypertext markup language content and the attribute information are stored in a preset file.

[0142] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0143] Get web page images and template element images;

[0144] Performing template matching between the template element image and the web page image to obtain a matching area, and calculating a matching value of the matching area;

[0145] If the matching value is less than a preset matching threshold, returning to the step of performing template matching between the template element image and the web page image;

[0146] If the matching value is greater than or equal to a preset matching threshold, the matching area is taken as a target area, and the coordinates of the center point of the target area are calculated;

[0147] A target element located in the target area is acquired according to the center point coordinates.

[0148] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0149] Extracting the outline of the template element in the template element image to obtain a template outline map;

[0150] Extracting the outline of each web page element in the web page image to obtain a web page outline image including the outlines of each web page element;

[0151] Filtering out web page element contours similar to the template contour map in the web page contour map as target contours, and taking the area occupied by the target contour in the web page contour map as the to-be-matched area;

[0152] The template contour image is subjected to template matching in the area to be matched to obtain a matching area.

[0153] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0154] Extracting features of template elements in the template element image to obtain a template feature map;

[0155] Extracting features of each web page element in the web page image to obtain a web page feature map including features of each web page element;

[0156] Screening out web page element features similar to the template feature map in the web page feature map as target features, and taking the area occupied by the target features in the web page feature map as the to-be-matched area;

[0157] The template feature map is subjected to template matching in the area to be matched to obtain a matching area.

[0158] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0159] Get the preset sliding step size;

[0160] Obtain a sliding window having the same size as the template element image;

[0161] The sliding window is slid on the web page image according to the sliding step length to obtain a matching area.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0163] Establishing a reference pixel coordinate system according to the plane where the web page image is located;

[0164] In the reference pixel coordinate system, obtaining the coordinates of any corner of the target area;

[0165] Obtaining the pixel width and pixel height of the target area;

[0166] Coordinate calculation is performed according to the coordinates of the arbitrary angle, the pixel width and the pixel height to obtain the coordinates of the center point of the target area.

[0167] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0168] Obtaining a label of a template element corresponding to the template element image, wherein the label is used to define the type of the element;

[0169] Get the label of the target element;

[0170] If the label of the template element is consistent with the label of the target element, the target element is valid;

[0171] If the label of the template element is inconsistent with the label of the target element, the target element is not valid.

[0172] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0173] Draw a border and a center point for the target element;

[0174] The hypertext markup language content and attribute information of the target element are obtained, and the hypertext markup language content and the attribute information are stored in a preset file.

[0175] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0176] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0177] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for obtaining web page elements, characterized in that: The method comprises: Get web page images and template element images; Performing template matching between the template element image and the web page image to obtain a matching area, and calculating a matching value of the matching area; If the matching value is less than a preset matching threshold, returning to the step of performing template matching between the template element image and the web page image; If the matching value is greater than or equal to a preset matching threshold, the matching area is taken as a target area, and the coordinates of the center point of the target area are calculated; A target element located in the target area is acquired according to the center point coordinates.

2. The web page element acquisition method according to claim 1, characterized in that: Performing template matching between the template element image and the web page image to obtain a matching area includes: Extracting the outline of the template element in the template element image to obtain a template outline map; Extracting the outline of each web page element in the web page image to obtain a web page outline image including the outlines of each web page element; Filtering out web page element contours similar to the template contour map in the web page contour map as target contours, and taking the area occupied by the target contour in the web page contour map as the to-be-matched area; The template contour image is subjected to template matching in the area to be matched to obtain a matching area.

3. The web page element acquisition method according to claim 1, characterized in that: Performing template matching between the template element image and the web page image to obtain a matching area includes: Extracting features of template elements in the template element image to obtain a template feature map; Extracting features of each web page element in the web page image to obtain a web page feature map including features of each web page element; Screening out web page element features similar to the template feature map in the web page feature map as target features, and taking the area occupied by the target features in the web page feature map as the to-be-matched area; The template feature map is subjected to template matching in the area to be matched to obtain a matching area.

4. The web page element acquisition method according to claim 1, characterized in that: Performing template matching between the template element image and the web page image to obtain a matching area includes: Get the preset sliding step size; Obtain a sliding window having the same size as the template element image; The sliding window is slid on the web page image according to the sliding step length to obtain a matching area.

5. The web page element acquisition method according to claim 1, characterized in that: Calculating the coordinates of the center point of the target area includes: Establishing a reference pixel coordinate system according to the plane where the web page image is located; In the reference pixel coordinate system, obtaining the coordinates of any corner of the target area; Obtaining the pixel width and pixel height of the target area; Coordinate calculation is performed according to the coordinates of the arbitrary angle, the pixel width and the pixel height to obtain the coordinates of the center point of the target area.

6. The web page element acquisition method according to claim 1, characterized in that: After acquiring the target element located in the target area according to the center point coordinates, the method further includes: Obtaining a label of a template element corresponding to the template element image, wherein the label is used to define the type of the element; Get the label of the target element; If the label of the template element is consistent with the label of the target element, the target element is valid; If the label of the template element is inconsistent with the label of the target element, the target element is not valid.

7. The web page element acquisition method according to claim 1, characterized in that: After acquiring the target element located in the target area according to the center point coordinates, the method further includes: Draw a border and a center point for the target element; The hypertext markup language content and attribute information of the target element are obtained, and the hypertext markup language content and the attribute information are stored in a preset file.

8. A web page element acquisition component, characterized in that: The components include: An image acquisition unit, used to acquire web page images and template element images; An element matching unit, used to perform template matching between the template element image and the web page image to obtain a matching area, and calculate a matching value of the matching area; The element matching unit is further configured to return to the step of performing template matching between the template element image and the web page image if the matching value is less than a preset matching threshold; The element matching unit is further configured to, if the matching value is greater than or equal to a preset matching threshold, use the matching area as a target area and calculate the coordinates of a center point of the target area; An element acquisition unit is used to acquire a target element located in the target area according to the center point coordinates.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the web page element acquisition method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the web page element acquisition method according to any one of claims 1 to 7 are implemented.

Images