Webpage element automatic detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a webpage element automatic detection method and device, electronic equipment and storage medium, the method comprises the following steps: polling at least one to-be-detected element in a to-be-detected page to obtain a to-be-processed image; obtaining two to-be-processed images corresponding to the same to-be-detected element in two adjacent polling times, and determining the similarity attribute between the two to-be-processed images; and detecting the to-be-detected element in the to-be-detected page according to the similarity attribute of the at least one to-be-detected element. The technical scheme of the embodiments of the present application solves the technical problems of checking webpage elements based on artificial subjective experience, resulting in element missed detection, low detection accuracy and high cost, and realizes automatic detection of webpage elements while reducing detection cost, thereby improving the comprehensiveness and accuracy of element detection.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer processing, and particularly relate to a webpage element automatic detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] Nowadays, many webpages will usually detect the elements displayed in the webpage before going online or during application in order to improve the user experience of the webpage. The element can be understood as the content displayed by each control in the display interface, for example, the displayed content can be pictures, buttons, text, etc. The current detection method of the element is usually to manually check the elements displayed in the webpage. For example, checking whether each element in the webpage is normally displayed, or whether the displayed content is consistent with the set display requirements, etc.

[0003] The inventor found the following problems when implementing the technical solution based on the above method.

[0004] Based on the manual checking of the webpage elements, the displayed elements need to be checked by relying on manual subjective experience, which not only easily leads to the omission of checking elements, but also has the problems of low checking accuracy and high detection cost. SUMMARY

[0005] The present application provides a webpage element automatic detection method, device, electronic equipment and storage medium, which realizes automatic detection of webpage elements, reduces detection cost, and achieves the technical effect of improving the comprehensiveness and accuracy of element detection.

[0006] In a first aspect, the embodiments of the present application provide a webpage element automatic detection method, which comprises:

[0007] Polling to obtain a to-be-processed image of at least one to-be-detected element in a to-be-detected page;

[0008] Obtaining two to-be-processed images corresponding to the same to-be-detected element in adjacent two polling times, and determining the similar attributes between the two to-be-processed images;

[0009] Detecting the to-be-detected elements in the to-be-detected page according to the similar attributes of the at least one to-be-detected element.

[0010] Further, the method further comprises:

[0011] Setting a to-be-detected page, at least one to-be-detected element in the to-be-detected page, and a detection order corresponding to the to-be-detected element, to sequentially obtain the to-be-processed image of the at least one to-be-detected element based on the detection order.

[0012] Further, the method further comprises:

[0013] Polling the pre-configured timing task to obtain the to-be-processed image of at least one to-be-detected element in the to-be-detected page;

[0014] The timing task comprises a polling frequency.

[0015] Further, the method further comprises:

[0016] Respectively determine the pixel information corresponding to each to-be-processed image; and determine the similarity attribute of the two to-be-processed images based on the pixel information.

[0017] Further, the pixel information comprises a total number of pixels, and the method further comprises:

[0018] Determining the total number of pixels of each to-be-processed image according to the length and width of the to-be-processed image;

[0019] Correspondingly, the determining the similarity attribute of the two to-be-processed images based on the pixel information comprises:

[0020] If the total number of pixels is different, determining the ratio of the two total numbers of pixels to obtain the similarity attribute.

[0021] Further, the method further comprises:

[0022] If the total number of pixels is the same, respectively determining the gray scale map of each to-be-processed image;

[0023] Correspondingly, the pixel information comprises a gray scale value of each pixel point, and the determining the similarity attribute of the two to-be-processed images based on the pixel information comprises:

[0024] Determining the number of different pixels by comparing the gray scale values of the same pixel points in the two gray scale maps;

[0025] Determining the similarity attribute based on the number of different pixels and the total number of pixels.

[0026] Further, the method further comprises:

[0027] If the similarity attribute of at least one to-be-detected element is greater than a preset similarity attribute threshold, it is determined that the to-be-detected page is abnormal, and a warning prompt is given.

[0028] Further, the method further comprises:

[0029] Highlighting the warning area in the to-be-detected page;

[0030] The pre-warning region is determined based on a pixel position corresponding to a similar attribute greater than a preset similar attribute threshold.

[0031] Further, the method further comprises:

[0032] The similar attribute is also displayed in the pre-warning region.

[0033] In a second aspect, an embodiment of the present application further provides an automatic detection device of a webpage element, the device comprising:

[0034] A to-be-processed image acquisition module is configured to poll a to-be-processed image of at least one to-be-detected element in a to-be-detected page;

[0035] A similar attribute determination module is configured to acquire two to-be-processed images corresponding to a same to-be-detected element in two adjacent polling times, and determine a similar attribute between the two to-be-processed images;

[0036] An element detection module is configured to detect the to-be-detected element in the to-be-detected page according to the similar attribute of the at least one to-be-detected element.

[0037] In a third aspect, an embodiment of the present application further provides an electronic device, the electronic device comprising:

[0038] One or more processors;

[0039] A storage device configured to store one or more programs,

[0040] When the one or more programs are executed by the one or more processors, the one or more processors implement the automatic detection method of the webpage element according to any of the embodiments of the present application.

[0041] In a fourth aspect, an embodiment of the present application further provides a storage medium containing computer executable instructions, which are used to execute the automatic detection method of the webpage element according to any of the embodiments of the present application when executed by a computer processor.

[0042] The technical scheme of the embodiment of the present application acquires the to-be-processed images of at least one to-be-detected element in a to-be-detected page through polling; acquires two to-be-processed images corresponding to the same to-be-detected element in two adjacent polling operations, and determines the similar attributes between the two to-be-processed images; and detects the to-be-detected element in the to-be-detected page according to the similar attributes of the at least one to-be-detected element. The technical scheme of the embodiment of the present application solves the technical problems of the prior art, such as element missing detection, low detection accuracy, and high cost, caused by checking web page elements based on artificial subjective experience, and realizes the acquisition of to-be-processed images of each to-be-detected element in a to-be-detected page through automatic polling, the comparison and determination of similar attributes of two to-be-processed images corresponding to the same to-be-detected element in two adjacent polling operations, and the automatic detection of the to-be-detected element based on the similar attributes, thereby reducing the detection cost and improving the comprehensiveness and accuracy of element detection. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical scheme of the exemplary embodiments of the present application, the drawings needed in the description of the embodiments will be briefly introduced. Obviously, the drawings introduced are only a part of the drawings of the embodiments to be described by the present application, and not all the drawings. Those skilled in the art can obtain other drawings from these drawings without creating labor.

[0044] Figure 1 A flowchart of an automatic detection method of a web page element provided by the embodiment of the present application;

[0045] Figure 2 A flowchart of an automatic detection method of a web page element provided by the embodiment of the present application;

[0046] Figure 3 A to-be-processed image provided by the embodiment of the present application;

[0047] Figure 4 A to-be-processed image provided by the embodiment of the present application;

[0048] Figure 5 A flowchart of an automatic detection method of a web page element provided by the embodiment of the present application;

[0049] Figure 6 A flowchart of an automatic detection method of a web page element provided by the embodiment of the present application;

[0050] Figure 7 A flowchart of an automatic detection method of a web page element provided by the embodiment of the present application;

[0051] Figure 8This is a flowchart illustrating an automated detection method for web page elements provided in an embodiment of the present invention.

[0052] Figure 9 This is a schematic diagram of the stitched image provided in an embodiment of the present invention;

[0053] Figure 10 This is a schematic diagram of the structure of an automated webpage element detection device provided in an embodiment of the present invention;

[0054] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0055] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0056] Before introducing this technical solution, let's first explain the application scenario. For example, in practical applications, CSS (Cascading Style Sheets) is typically used to render elements on a webpage, controlling the display effects of text, images, and other elements to obtain the rendered webpage. Before the webpage goes live or during application, to improve the user experience, elements on the webpage can be periodically detected. In this case, the webpage can be used as the page to be detected, and the elements to be detected are the elements to be detected. For example, the elements to be detected can be images, text, numbers, icons, labels, wireframes, links, etc. Element detection can be implemented using the technical solution provided in this embodiment of the invention.

[0057] Figure 1 This is a flowchart illustrating an automated webpage element detection method provided in an embodiment of the present invention. This embodiment is applicable to the detection of webpage elements. The method can be executed by an automated webpage element detection device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server.

[0058] like Figure 1 As shown, the method includes:

[0059] S110. Poll to obtain the image of at least one element to be detected in the page to be detected.

[0060] The to-be-detected element can be a pre-set element that needs to be detected, and the number of to-be-detected elements can be one or more. For example, if there is one element that needs to be detected in the to-be-detected page, the number of to-be-detected elements is one, and if there are multiple elements that need to be detected, the number of to-be-detected elements is multiple. In the process of detecting the to-be-detected element, an image containing the to-be-detected element can be collected or intercepted, and this image is the to-be-processed image. The polling can be a polling function triggered at a specified time

[0061] In actual application, the polling function for acquiring images can be triggered after detecting that the user triggers the element detection control, or the polling function for acquiring images can be automatically triggered at a specified time. After the polling function for acquiring images is started, the to-be-processed image of at least one to-be-detected element in the to-be-detected page can be requested at a specified time. For example, a screenshot instruction can be sent to a screenshot tool, or the screenshot function can be automatically triggered, and then the image including the to-be-detected element in the to-be-detected page is intercepted by the screenshot tool as the to-be-processed image. The to-be-processed images of the to-be-detected elements in the to-be-detected page acquired by one request can be taken as the to-be-processed images of one round. The to-be-processed images acquired by each round are sequentially uploaded to the system, so that the to-be-detected elements are detected based on the to-be-processed images.

[0062] In order to improve the accuracy and comprehensiveness of the web element detection and prevent missing detection, the to-be-detected page, at least one to-be-detected element in the to-be-detected page, and the detection order corresponding to the to-be-detected element can be set before the polling function for acquiring the to-be-processed images of at least one to-be-detected element in the to-be-detected page is started, so that the to-be-processed images of at least one to-be-detected element are sequentially acquired based on the detection order.

[0063] The to-be-detected page, the to-be-detected element in the to-be-detected page, and the detection order corresponding to the to-be-detected element can all be determined by a technician according to actual working conditions, which is not limited herein.

[0064] In this embodiment, the web pages that need to be detected can be pre-set as to-be-detected pages. The elements in the to-be-detected page that need to be detected can be pre-set as to-be-detected elements, and the detection order of the to-be-detected elements can be set. For example, the detection order of the to-be-detected elements can be set according to the order from left to right and from top to bottom of the to-be-detected elements in the to-be-detected page, and the to-be-detected elements can be numbered, such as element 1-element 2-element 3-element 4. After starting the detection task, the to-be-processed images of at least one to-be-detected element in the to-be-detected page can be sequentially acquired based on the detection order.

[0065] For example, refer to Figure 2Firstly, information such as a page to be detected, an element to be detected in the page to be detected, a CSS selector of each element to be detected, a preset similar attribute threshold corresponding to the element to be detected, and a polling frequency can be set. The CSS selector is used to select the element to be detected for operations such as screenshot and click. Further, an automated polling task is started, and images of each element to be detected in the page to be detected are periodically intercepted as images to be processed, and the images to be processed are stored.

[0066] It should be noted that, in order to improve the automation and accuracy of element detection, in the process of polling the images to be processed of at least one element to be detected in the page to be detected, the images to be processed of at least one element to be detected in the page to be detected are polled based on a preconfigured timing task.

[0067] The timing task can be a preconfigured task for periodically performing element detection. The timing task includes a polling frequency and can also include an element detection method. For example, the task can specifically include a task of accessing a page to obtain an image to be processed every certain time, and then comparing the image obtained this time with the image obtained in the last round. The polling frequency can be set based on the importance of the page to be detected, for example, the time length of polling the page each time can be set shorter for a page to be detected with high importance, and the time length of polling the page each time can be set longer for a page to be detected with low importance. The polling frequency can also be set by a technician according to actual working conditions. For example, the polling frequency can be set by a cron expression, such as polling every 3 seconds, every 1 minute, or every 5 minutes.

[0068] Specifically, the start of the timing task can be automatically triggered, or the start of the timing task can be triggered after a user triggers an element detection control. After triggering the timing task, the images to be processed of at least one element to be detected in the page to be detected can be polled at a preset polling frequency. For example, the timing task is started, and the images to be processed of at least one element to be detected in the page to be detected are obtained every 3 seconds.

[0069] For example, the playwright open source framework can be used to perform automated polling screenshots by writing an automated polling program, such as opening a page, selecting an element to be detected according to a CSS selector, taking a screenshot, and saving. The timing task can be integrated into a UI (User Interface, user interface) automation system to automatically perform the task at a timing.

[0070] It should be noted that, in order to improve the accuracy of subsequent element image comparison, ensure the correctness of element detection, and prevent the problem of comparing the adjacent two rounds of different to-be-detected elements of the to-be-processed image. When the to-be-detected element of the to-be-processed image is automatically intercepted, the to-be-processed image can be named and saved according to the set naming method. For example, the naming method can be task_round_element acquisition step_suffix. The element acquisition step can represent the step of the element in the detection sequence, and each element has a unique acquisition step. The suffix can be jpg, png, etc. For example, the to-be-processed images of the four elements obtained in the first round are named 1_1_1.jpg, 1_1_2.jpg, 1_1_3.jpg, and 1_1_4.jpg, respectively. The to-be-processed images of the four elements obtained in the second round are named 1_2_1.jpg, 1_2_2.jpg, 1_2_3.jpg, and 1_2_4.jpg, respectively. The to-be-processed images are named and saved in this naming manner to ensure that the images of different tasks, different rounds, and different elements are not repeated.

[0071] S120, acquiring two to-be-processed images corresponding to the same to-be-detected element in adjacent two rounds of polling, and determining the similarity attribute between the two to-be-processed images.

[0072] The similarity attribute can be used to represent the similarity between the image contents. If the similarity is higher, the two images are more similar. If the similarity is lower, the two images are less similar.

[0073] It should be noted that the to-be-processed images of the same to-be-detected element in adjacent rounds can be the same or different. For example, the font color, background, border line, background color, and text effect of element 1 in adjacent rounds are different.

[0074] In practical applications, the two images corresponding to the same element to be detected in adjacent polling can be obtained by naming the images to be processed in the process of detecting the element to be detected. For example, after obtaining the image to be processed in the second round, the image to be processed in the second round can be compared with the image to be processed in the first round, such as 1_1_1.jpg of element 1 in the first round and 1_2_1.jpg of element 1 in the second round as the two images to be processed of element 1 in adjacent rounds. Further, the two images to be processed of the same element to be detected in adjacent polling can be compared and processed to determine the similarity between the two images to be processed as the similarity attribute. Accordingly, the similarity attribute of each element to be detected can be obtained. For example, the way to determine the similarity attribute can be based on image recognition technology to recognize the two images to be processed, compare the feature point information between the two images to be processed to obtain the similarity attribute; or, the similarity between the two images to be processed can also be determined based on similarity calculation method (such as cosine similarity, hash algorithm, histogram calculation method or structural similarity measurement method, etc.), if the similarity value is higher, the similarity is higher, if the similarity value is lower, the similarity is lower, to obtain the similarity attribute; or, the difference between the two images to be processed can also be determined, if the difference value is higher, the similarity is lower, if the difference value is lower, the similarity is higher, the difference value represents the similarity attribute.

[0075] For example, it can be continued to refer to Figure 2 After obtaining the image to be processed in a round, it can be judged whether the round is the first round, if it is the first round, the image to be processed obtained in the first round does not perform task comparison, and waits for the next execution. Further, the image to be processed in the second round is obtained, at this time, it is the image to be processed for the first time, the image to be processed of the same element to be detected in the last round can be obtained. For example, it can be referred to Figure 3 and Figure 4 The image to be processed 1_1_1.jpg can be as shown in Figure 3 , the image to be processed 1_2_1.jpg is as shown in Figure 4 , the numbers of the arrows pointing from 000 to 111 and the arrows pointing from 111 to 222 in the two images to be processed are different, and the sizes of the line boxes to which 222 belongs are inconsistent. The two images to be processed of the same element to be detected can be compared to determine the similarity attribute. After obtaining the image to be processed in the third round, the image to be processed of the same element to be detected can be compared with the image to be processed of the element to be detected obtained in the second round to determine the similarity attribute. Then, the two images to be processed corresponding to the same element to be detected in adjacent polling are compared to obtain the similarity attribute corresponding to each element to be detected.

[0076] S130, detecting the to-be-detected element in the to-be-detected page according to the similarity attribute of the at least one to-be-detected element.

[0077] In the embodiment, after comparing the two to-be-processed images corresponding to the same to-be-detected element in the adjacent two pollings and determining the similarity attribute of the two to-be-processed images, it is analyzed whether the similarity attribute meets the preset display requirement. For example, if the similarity attribute represents a high similarity degree, it is considered that the element is displayed correctly, and if the similarity attribute represents a low similarity degree, it is considered that the element is not displayed correctly. Specifically, a threshold value can be set in advance to determine whether the similarity attribute reaches the set threshold value. If it reaches, it is considered that the similarity degree of the two to-be-processed images is high, and it is considered that the element is displayed correctly. If it does not reach, it is considered that the similarity degree of the two to-be-processed images is low, and the difference degree is large, and there can be a problem that the element is not displayed correctly. In this way, the to-be-detected element in the to-be-detected page is detected to determine whether the element is displayed correctly.

[0078] The technical scheme of the embodiment of the application acquires the to-be-processed image of at least one to-be-detected element in a to-be-detected page through polling, acquires two to-be-processed images corresponding to the same to-be-detected element in adjacent two pollings, determines the similarity attribute between the two to-be-processed images, and detects the to-be-detected element in the to-be-detected page according to the similarity attribute of the at least one to-be-detected element. The technical scheme of the embodiment of the application solves the technical problems of element missed detection, low detection accuracy, and high cost caused by checking the web page element based on artificial subjective experience in the prior art, realizes automatic polling of the to-be-processed image of each to-be-detected element in the to-be-detected page, comparison and determination of the similarity attribute of the two to-be-processed images corresponding to the same to-be-detected element in adjacent two pollings, and automatic detection of the to-be-detected element based on the similarity attribute, thereby reducing the detection cost and improving the comprehensiveness and accuracy of element detection.

[0079] Figure 5 The flowchart of the automatic detection method of the web page element provided in the embodiment of the application is shown in the foregoing embodiment, and S120 is further refined. The specific implementation can be referred to the technical scheme of the embodiment. The same or corresponding technical terms as in the foregoing embodiment are not described herein.

[0080] As shown in the embodiment, the method specifically includes the following steps: Figure 5

[0081] S210, acquiring the to-be-processed image of at least one to-be-detected element in a to-be-detected page through polling.

[0082] S220, acquiring two to-be-processed images corresponding to the same to-be-detected element in adjacent two pollings. ​

[0083] S230, respectively determine the pixel information corresponding to each of the images to be processed.

[0084] The pixel information can include the number, position, brightness, color and the like of the pixels.

[0085] Specifically, each of the images to be processed can be analyzed to obtain the pixel information of the pixels in each of the images to be processed.

[0086] S240, based on the pixel information, determine the similar attribute of the two images to be processed.

[0087] In this embodiment, the similar attribute of the two images to be processed can be determined by comparing the pixel information of the two images to be processed. For example, the brightness values of the same pixel points in the two images to be processed can be compared. If the number of pixels with similar brightness values is larger, it indicates that the similarity is higher and the difference is lower. Alternatively, the number of pixels in the two images to be processed can be compared. If the number of pixels in the two images is more similar, it indicates that the similarity is higher and the difference is lower.

[0088] S250, detecting the element to be detected in the page to be detected according to the similar attribute of the at least one element to be detected.

[0089] The technical scheme of the embodiment of the application determines the pixel information corresponding to each of the images to be processed, and then calculates the similar attribute between the two images to be processed based on the pixel information, so as to detect the element to be detected based on the similar attribute, thereby improving the detection accuracy and convenience.

[0090] Figure 6 The flowchart of the automatic detection method of the web page element provided by the embodiment of the application is based on the foregoing embodiment. The pixel information can include the total number of pixels, and the similar attribute of the two images to be processed can be determined based on the total number of pixels in the process of determining the similar attribute of the two images to be processed based on the pixel information. The specific implementation can be referred to the technical scheme of this embodiment. The same or corresponding technical terms as the above embodiments are not repeated here.

[0091] As shown in the flowchart of the automatic detection method of the web page element provided by the embodiment of the application, the method comprises the steps of: Figure 6

[0092] S310, polling to obtain the images to be processed of at least one element to be detected in the page to be detected.

[0093] S320, obtaining two images to be processed corresponding to the same element to be detected in two adjacent polling.

[0094] ​S330. Determine the total number of pixels in each image to be processed based on its length and width.

[0095] Specifically, after obtaining two images of the same element to be detected in two consecutive polls, the dimensions of these two images can be parsed to obtain the length and width of each image. Furthermore, the length and width of each image can be multiplied to obtain a product value, which can then be used as the total number of pixels in the image. Accordingly, the total number of pixels in both images can be obtained.

[0096] S340. If the total number of pixels is different, determine the ratio of the total number of pixels of the two pixels to obtain similarity attributes.

[0097] In this embodiment, after obtaining the total number of pixels in two images of the same element to be detected in two consecutive polls, these two total pixel counts can be compared. If the two total pixel counts are inconsistent, it indicates that the two images are not identical. In this case, the two total pixel counts can be divided to obtain a ratio. The similarity attribute between the two images can be evaluated using this ratio. It should be noted that, to clearly represent the similarity attribute, it can be expressed as a percentage, decimal, or fraction, for example... Figure 1 The total number of pixels is 100*100. Figure 2 The total number of pixels is 120*100. We can divide the smaller value by the larger value, i.e., (100*100) / (120*100) = 83.33%. 83.33% can be used as a similarity attribute; a larger ratio indicates higher similarity, and a smaller ratio indicates lower similarity. Alternatively, we can use 1-83.33% as a similarity attribute; a larger value indicates lower similarity, and a smaller value indicates higher similarity. Or, when determining the similarity attribute when the total number of pixels is different, we can analyze the pixels with different pixel information at the same location in the two images to be processed, and then count the total number of pixels with different pixel information in one image relative to the other. The ratio of the total number of different pixels in the corresponding image to the total number of pixels in its own image can then be used as the similarity attribute. For example, we can obtain the similarity attribute by multiplying the image's length and width. Figure 2 After the total number of pixels is 120*100, it can be calculated. Figure 2 Zhongyu Figure 1 The percentage of pixels with differences (120*100-100*100) to the total number of pixels is 16.67%. As a similarity attribute, the larger the value of this similarity attribute, the higher the similarity; the smaller the value of the similarity attribute, the lower the similarity.

[0098] S350, detecting the to-be-detected element in the to-be-detected page according to the similarity attribute of the at least one to-be-detected element.

[0099] The technical scheme of the embodiment of the present application determines the total number of pixels of each to-be-processed image according to the length and width of the to-be-processed image, and then determines the ratio of the total number of pixels of two to-be-processed images when the total number of pixels is different, to obtain the similarity attribute. The method of comparing the total number of pixels is used to preliminarily determine whether the two to-be-processed images are inconsistent, and when the total number of pixels is different, it is determined that the two to-be-processed images are inconsistent, thereby improving the speed and accuracy of element detection.

[0100] Figure 7 The flowchart of the automatic detection method of the webpage element provided by the embodiment of the present application is based on the foregoing embodiment. After the total number of pixels of each to-be-processed image is determined according to the length and width of the to-be-processed image, if the total number of pixels is the same, it can be concluded that the pixels are consistent, and the grayscale information of the pixels in the two to-be-processed images can be further analyzed to determine the similarity attribute of the two to-be-processed images, thereby improving the accuracy of webpage element detection. The specific implementation can be referred to the technical scheme of the present embodiment. The same or corresponding technical terms as the foregoing embodiment are not described herein.

[0101] As shown in the method, the method comprises the following steps. Figure 7

[0102] S410, polling to obtain a to-be-processed image of at least one to-be-detected element in a to-be-detected page.

[0103] S420, obtaining two to-be-processed images corresponding to the same to-be-detected element in two adjacent polling operations.

[0104] S430, determining the total number of pixels of each to-be-processed image according to the length and width of the to-be-processed image.

[0105] S440, if the total number of pixels is the same, then determining the grayscale map of each to-be-processed image.

[0106] The pixel information includes the grayscale value of each pixel.

[0107] In the present embodiment, after obtaining the total number of pixels of the two to-be-processed images of the same to-be-detected element in two adjacent polling operations, if the total number of pixels of the two to-be-processed images is the same, then each to-be-processed image can be further processed into a grayscale map, and the pixel information in the image is represented by a grayscale value of 0 to 255, so that the similarity attribute is determined by the grayscale value.

[0108] ​For example, if the total number of pixels in two images is the same, then the grayscale value of each pixel can be calculated from left to right and from top to bottom to obtain the grayscale image of the two images.

[0109] S450. By comparing the gray values ​​of the same pixel in two grayscale images, the number of dissimilar pixels with different gray values ​​is determined.

[0110] In this embodiment, the grayscale values ​​of the same pixel in two grayscale images can be compared. If the grayscale values ​​are different, it indicates that the same pixel in the two grayscale images has different pixel information. The number of such pixels can be counted as the number of dissimilar pixels. For example, the grayscale value of the pixel at position A in image 1 is 20, and the grayscale value of the pixel at position A in image 2 is 21. The two grayscale values ​​are different, and this pixel is a dissimilar pixel. The sum of the number of all dissimilar pixels can be used as the number of dissimilar pixels to analyze the differences between the two images and obtain similarity attributes.

[0111] S460. Based on the number of dissimilar pixels and the total number of pixels, determine the similarity attributes.

[0112] In this embodiment, the quotient is obtained by dividing the number of dissimilar pixels by the total number of pixels. This quotient can be used as a similarity attribute to evaluate the differences between two images to be processed.

[0113] S470. Based on the similarity attribute of at least one element to be detected, detect the element to be detected in the page to be detected.

[0114] The technical solution of this invention determines the grayscale image of each image to be processed when the total number of pixels is the same; compares the grayscale values ​​of the same pixel in the two grayscale images to determine the number of dissimilar pixels with different grayscale values; and determines similarity attributes based on the number of dissimilar pixels and the total number of pixels. This realizes the calculation of similarity attributes through the grayscale values ​​of pixels, so as to characterize the difference between the two images to be processed based on the similarity attributes, thereby improving the accuracy of element detection.

[0115] Figure 8 This is a flowchart illustrating an automated webpage element detection method provided in an embodiment of the present invention. Based on the foregoing embodiments, step S130 is further refined. Specific implementation details can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0116] like Figure 8 As shown, the method includes:

[0117] S510. Poll to obtain the image of at least one element to be detected in the page to be detected.

[0118] S520, acquire two images to be processed corresponding to the same element to be detected in the adjacent two times of polling, and determine the similar attribute between the two images to be processed.

[0119] S530, if the similar attribute of at least one element to be detected is greater than the preset similar attribute threshold, it is determined that the page to be detected is abnormal, and a warning prompt is given.

[0120] It should be noted that in actual application, each element to be detected in the page to be detected may be the same or different. For example, the element type of element 1 and element 2 is both text, but the text content, word count, color, font, and area size may be different. Correspondingly, the display standard of the elements to be detected in different pages to be detected may be different, and the detection standard of different elements to be detected in the same page to be detected may also be different. In order to improve the accuracy of the detection of the elements to be detected in the page, a corresponding detection threshold can be configured for each element to be detected in the page to be detected, that is, as a preset similar attribute threshold. For example, the preset similar attribute threshold can be set based on the element attribute of the element to be detected and / or the page attribute of the page to be detected to which the element to be detected belongs.

[0121] It should be further noted that when the preset similar attribute threshold is configured, the similar attribute can be further combined to set it. If the similar attribute value is higher, the degree of dissimilarity is higher, and the degree of similarity is lower. Correspondingly, in actual application, the lower the similar attribute value of the element to be detected, the better the display effect of the element, and the higher the preset similar attribute threshold set at this time, the higher the degree of dissimilarity, and the lower the degree of similarity. For example, the preset similar attribute threshold corresponding to the element to be detected with high importance can be set to 0.2%; the preset similar attribute threshold corresponding to the element to be detected with complex display effect can be set to 0.3%. If the similar attribute value is lower, the degree of similarity is higher, and the degree of dissimilarity is lower. Correspondingly, in actual application, the lower the similar attribute value of the element to be detected, the worse the display effect of the element, and the smaller the preset similar attribute threshold set at this time, the higher the degree of similarity, and the lower the degree of dissimilarity. For example, the preset similar attribute threshold corresponding to the element to be detected with high importance can be set to 98%; the preset similar attribute threshold corresponding to the element to be detected with complex display effect can be set to 97%. Based on the preset similar attribute threshold to determine whether the similar attribute meets the display standard.

[0122] In actual application, after the similarity attribute of the to-be-detected element is determined, the similarity attribute is compared with a preset similarity attribute threshold corresponding to the to-be-detected element. If the similarity attribute is greater than the preset similarity attribute threshold, it indicates that the difference between the two to-be-processed images corresponding to the to-be-detected element exceeds the set requirement, and the to-be-detected element does not meet the set display standard. At this time, it is considered that the to-be-detected page is abnormal, and a pre-warning prompt can be automatically triggered. For example, the similarity attribute of element A in page 1 is 3.41%, which is greater than the preset similarity attribute threshold 0.3%. It indicates that the difference between the two images of element A is large, and element A may not be normally displayed. At this time, it is determined that page 1 is abnormal.

[0123] For example, continuing to refer to Figure 2 , it is judged whether the similarity attribute is greater than the preset similarity attribute threshold. If yes, a pre-warning prompt is given. If no, the comparison in this round ends.

[0124] In this embodiment, when the pre-warning prompt is given, the pre-warning area in the to-be-detected page can be highlighted.

[0125] The pre-warning area is determined based on the pixel point position corresponding to the case that the similarity attribute is greater than the preset similarity attribute threshold.

[0126] Specifically, when the pre-warning prompt is triggered, the pre-warning area can be marked based on the pixel point position where the similarity attribute is greater than the preset similarity attribute threshold. At this time, the pre-warning area is the area where the difference exists between the two images. Further, the pre-warning area can be highlighted. The highlighting manner can be that the pre-warning area is framed and displayed by using a line, or the information in the pre-warning area is highlighted, or the pre-warning area is highlighted in the form of text, icon, etc. The specific highlighting manner is not limited. For example, the pre-warning area can be framed by a red-green line, so that it is known which elements in the area are abnormal, and the subsequent repair is facilitated.

[0127] It should be noted that, in order to enable the working user to clearly know the difference information from the pre-warning area highlighted in the two to-be-processed images, the information is compared. The two to-be-processed images in which the pre-warning area is highlighted can also be spliced to obtain a spliced image, so as to send the spliced image to the user terminal device. For example, referring to Figure 9 , it is assumed that the similarity attribute (such as 15%) between to-be-processed image 1_1_1.jpg and to-be-processed image 1_2_1.jpg is greater than the preset similarity attribute threshold (such as 2%). The pre-warning area corresponding to the arrow pointing from 000 to 111 and the arrow pointing from 111 to 222 in the two images, and the pre-warning area corresponding to the line frame to which 222 belongs can be highlighted by a dashed frame.

[0128] The plurality of to-be-processed images contained in the to-be-detected page with the abnormality can also be spliced to obtain page images of two adjacent rounds respectively, and the page images can contain to-be-processed images of one to-be-detected element or can contain to-be-processed images of a plurality of to-be-detected elements. The pre-warning region in the to-be-processed image with the similar attribute greater than the preset similar attribute threshold can be highlighted in the page image, and the two highlighted page images can be spliced to obtain a spliced image, so that all to-be-detected elements with display abnormalities in the to-be-detected page can be clearly known based on the spliced image.

[0129] In order to make the working user clearly know the difference degree between the two to-be-processed images of the to-be-detected element, and facilitate the user to determine whether the repair operation is needed. The similar attribute can also be displayed in the pre-warning region, or the similar attribute can be displayed at a preset first position in the to-be-detected page, or the similar attribute can be displayed at a preset second position in the spliced image. For example, the image difference percentage (i.e., the similar attribute) can be marked at the center position of the to-be-detected page, for example, the similar attribute is 3.41%, which represents that the two images have a pixel inconsistency of 3.41%.

[0130] For example, continue to participate Figure 9 The similar attribute of 15% difference can be displayed in the spliced image.

[0131] It should be noted that when the pre-warning prompt is given, the spliced image in which the pre-warning region and the similar attribute are highlighted can also be sent to the user terminal device, so that the working staff corresponding to the user terminal device can confirm whether the page is abnormal based on the spliced image after receiving the spliced image, so as to timely perform page repair when the abnormality is confirmed.

[0132] For example, the sending can be performed in the form of an email, a short message, a message, a webpage, etc. The information sent can include a detection task of the to-be-detected page, to-be-processed images of each to-be-detected element, and / or a front-and-back comparison image (i.e., a spliced image) of the to-be-detected element with the similar attribute greater than the preset similar attribute threshold, etc. so that the working staff who knows the pre-warning prompt can determine whether the CSS is abnormal according to the received information, so as to perform subsequent repair work. The correctness of the CSS is confirmed by manual work, the accuracy of detection is improved, and detection errors are avoided.

[0133] The technical scheme of the embodiment of the application determines that the to-be-detected page has an abnormality when the similar attribute of at least one to-be-detected element is greater than the preset similar attribute threshold, and gives a pre-warning prompt, so that the timeliness of monitoring the abnormal to-be-detected page is realized, and the execution of subsequent repair work is facilitated.

[0134] Figure 10A structural schematic diagram of an automatic detection device for a webpage element provided by an embodiment of the present application, the device comprising: a to-be-processed image acquisition module 610, a similar attribute determination module 620, and an element detection module 630.

[0135] The to-be-processed image acquisition module 610 is configured to acquire to-be-processed images of at least one to-be-detected element in a to-be-detected page through polling; the similar attribute determination module 620 is configured to acquire two to-be-processed images corresponding to a same to-be-detected element in two adjacent polling times, and determine a similar attribute between the two to-be-processed images; and the element detection module 630 is configured to detect the to-be-detected element in the to-be-detected page according to the similar attribute of the at least one to-be-detected element.

[0136] On the basis of the above device, the device can further comprise a setting module.

[0137] The setting module is configured to set a to-be-detected page, at least one to-be-detected element in the to-be-detected page, and a detection order corresponding to the to-be-detected element, so as to acquire to-be-processed images of the at least one to-be-detected element in sequence based on the detection order.

[0138] On the basis of the above device, the to-be-processed image acquisition module 610 can be specifically configured to acquire to-be-processed images of at least one to-be-detected element in the to-be-detected page through polling based on a pre-configured timing task.

[0139] The timing task comprises a polling frequency.

[0140] On the basis of the above device, the similar attribute determination module 620 can comprise a pixel information determination unit and a similar attribute determination unit.

[0141] The pixel information determination unit is configured to determine pixel information corresponding to each to-be-processed image respectively.

[0142] The similar attribute determination unit is configured to determine a similar attribute of the two to-be-processed images based on the pixel information.

[0143] On the basis of the above device, the pixel information can comprise a total number of pixels, and the pixel information determination unit can comprise a total number of pixels determination subunit configured to determine a total number of pixels of a corresponding to-be-processed image according to a length and a width of each to-be-processed image.

[0144] Correspondingly, the similar attribute determination unit is configured to determine a ratio of two total numbers of pixels to obtain the similar attribute if the total numbers of pixels are different.

[0145] On the basis of the above device, optionally, the similar attribute determining unit is further configured to determine a gray scale image of each of the to-be-processed images if the total number of pixels is the same.

[0146] The dissimilar pixel number determining subunit is configured to determine the number of dissimilar pixels by comparing the gray scale values of the same pixel points in the two gray scale images.

[0147] The similar attribute determining subunit is configured to determine the similar attribute based on the number of dissimilar pixels and the total number of pixels.

[0148] On the basis of the above device, optionally, the element detection module 630 is specifically configured to determine that the to-be-detected page is abnormal and give a warning prompt if the similar attribute of at least one to-be-detected element is greater than the preset similar attribute threshold.

[0149] On the basis of the above device, optionally, the element detection module 630 is further configured to highlight a warning area in the to-be-detected page when giving the warning prompt, wherein the warning area is determined based on the pixel positions corresponding to the case that the similar attribute is greater than the preset similar attribute threshold.

[0150] On the basis of the above device, optionally, the element detection module 630 is further configured to display the similar attribute in the warning area.

[0151] The technical scheme of the embodiment of the present application acquires the to-be-processed images of at least one to-be-detected element in the to-be-detected page by polling; acquires two to-be-processed images corresponding to the same to-be-detected element in adjacent two times of polling and determines the similar attribute between the two to-be-processed images; and detects the to-be-detected element in the to-be-detected page according to the similar attribute of the at least one to-be-detected element. The technical scheme of the embodiment of the present application solves the technical problems of element omission, low detection accuracy and high cost in the prior art caused by checking the web page elements based on subjective experience, realizes automatic polling of the to-be-processed images of each to-be-detected element in the to-be-detected page, comparison and determination of the similar attribute of the two to-be-processed images corresponding to the same to-be-detected element in adjacent two times of polling, and automatic detection of the to-be-detected element based on the similar attribute, thereby reducing the detection cost and improving the comprehensiveness and accuracy of element detection.

[0152] The automatic detection device for web page elements provided in the embodiment of the present application can execute the automatic detection method for web page elements provided in any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0153] It is worth noting that the units and modules included in the above system are only divided according to the functional logic, and are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy mutual differentiation, and do not serve to limit the protection scope of the embodiments of the present application.

[0154] Figure 11 A structural schematic diagram of an electronic device provided by the embodiments of the present application. Figure 11 A block diagram of an exemplary electronic device 70 suitable for implementing the embodiments of the present application is shown. Figure 11 The electronic device 70 shown is merely an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0155] As Figure 11 shown, the electronic device 70 is in the form of a general-purpose computing device. The components of electronic device 70 can include, but are not limited to, one or more processors or processing units 701, a system memory 702, and a bus 703 that couples various system components including system memory 702 and processing unit 701.

[0156] Bus 703 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics bus, a processor or local bus using any of a variety of bus architectures including an industry standard architecture (ISA), micro-channel architecture (MAC), enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, and a peripheral component interconnect (PCI) bus.

[0157] Electronic device 70 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by electronic device 70 and includes both volatile and non-volatile media, removable and non-removable media.

[0158] System memory 702 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 704 and / or cache memory 705. Electronic device 70 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 706 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a "hard drive"). Figure 11 not shown, commonly referred to as a "hard disk drive"). Although Figure 11A disk drive, a floppy disk drive, a CD-ROM drive, a DVD-ROM drive, or other removable media drive, a flash memory card drive (such as a compact flash drive), and a DVD RAM, a DVD RAM2, Blu- ray Disc (BD) or other disc drives, a hard-disk drive, or other device that enables a mobile nonvolatile memory to be read from and / or written to by a mobile device, can be provided as an interface for a removable nonvolatile memory (such as a "floppy disk" or a "flash drive") to read from and / or write to. In these instances, each drive can be connected to the bus 703 by one or more data media interfaces. The storage 702 can include one or more program products pertaining to one or more applications used by a mobile device, for example.

[0159] Program / utility 708 having a set (at least one) of program modules 707 can be stored in, for example, storage 702 by way of example, and includes an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, can include an implementation of a network environment, for example.

[0160] The electronic device 70 can also communicate with one or more external devices 709 such as a keyboard or pointing device, a display 710, etc.; one or more devices that enable a user to interact with the electronic device 70; and / or one or more devices that enable the electronic device 70 to communicate with one or more other computing devices. Such communication can be via an input / output (I / O) interface 711. Similarly, the electronic device 70 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network such as the Internet, via a network adapter 712. As indicated, the network adapter 712 can be communicatively coupled to the other components of the electronic device 70 via the bus 703. It will be appreciated that various components (e.g., those not specifically shown) that are well known to those skilled in the art can be included in the electronic device 70, such as power supplies, clock circuits, a cache or other memory, etc. Figure 11 Other hardware and / or software modules that can be used in conjunction with the electronic device 70 can include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0161] The processing unit 701 performs various functions and processing of data by executing programs stored in the system memory 702, such as implementing the automated detection method of a web page element.

[0162] The embodiments of the present application also provide a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to perform an automated detection method of a web page element, the method comprising:

[0163] Polling to obtain a to-be-processed image of at least one to-be-detected element in a to-be-detected page;

[0164] acquire two images to be processed corresponding to the same element to be detected in two adjacent polling, and determine the similar attribute between the two images to be processed;

[0165] detect the element to be detected in the page to be detected according to the similar attribute of the at least one element to be detected.

[0166] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, device or apparatus.

[0167] The computer readable signal medium can include a data signal propagated in a baseband or as a part of a carrier wave, in which a computer readable program code is carried. Such a propagated data signal can take multiple forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in connection with an instruction execution system, device or apparatus.

[0168] The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0169] Computer program code for carrying out operations of embodiments of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0170] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the application. The word "comprising" does not exclude the presence of elements or steps other than those listed and the word "a" or "an" preceding the usage of an element does not exclude the presence of a plurality of such elements. It is anticipated that, depending on the specific design choices made for a given embodiment, the application can be implemented in a variety of ways, all of which are intended to fall within the scope of the present application as defined by the subjoined claims.

Claims

1. An automated detection method for web page elements, characterized in that, include: Poll to obtain the image to be processed of at least one element to be detected in the page to be detected; Obtain two images to be processed corresponding to the same element to be detected in two consecutive polls, and determine the similarity attribute between the two images to be processed; Determine the pixel information corresponding to each image to be processed; The pixel information includes the total number of pixels; The total number of pixels is determined based on the length and width of the image to be processed; Based on the pixel information, the similarity attribute of the two images to be processed is determined, including: if the total number of pixels is different, the ratio of the total number of pixels of the two images is determined to obtain the similarity attribute; Based on the similarity attributes of the at least one element to be detected, the elements to be detected in the page to be detected are detected.

2. The method according to claim 1, characterized in that, Before polling to obtain the image of at least one element to be detected in the page to be detected, the method further includes: Set the page to be detected, at least one element to be detected on the page to be detected, and the detection order corresponding to the element to be detected, so as to obtain the image to be processed of the at least one element to be detected in sequence based on the detection order.

3. The method according to claim 1, characterized in that, The polling process for obtaining the image of at least one element to be detected in the page to be detected includes: The image to be processed is obtained by polling at least one element to be detected in the page to be detected based on a pre-configured timed task; The timed task includes a polling frequency.

4. The method according to claim 1, characterized in that, The step of determining the pixel information corresponding to each image to be processed includes: The total number of pixels in each image to be processed is determined based on its length and width.

5. The method according to claim 1, characterized in that, Also includes: If the total number of pixels is the same, then the grayscale image of each image to be processed is determined separately; Accordingly, the pixel information includes the grayscale value of each pixel, and determining the similarity attribute of the two images to be processed based on the pixel information includes: By comparing the gray values ​​of the same pixel in the two grayscale images, the number of dissimilar pixels with different gray values ​​is determined. The similarity attribute is determined based on the number of dissimilar pixels and the total number of pixels.

6. The method according to claim 1, characterized in that, The step of detecting elements in the page to be detected based on the similarity attributes of the at least one element to be detected includes: If at least one element to be detected has a similarity attribute greater than a preset similarity attribute threshold, then the page to be detected is determined to be abnormal, and an early warning is issued.

7. The method according to claim 6, characterized in that, The warning notification also includes: The warning area will be highlighted on the page to be detected; The warning area is determined based on the pixel position corresponding to the similarity attribute being greater than a preset similarity attribute threshold.

8. The method according to claim 7, characterized in that, Also includes: Similar attributes are also displayed in the warning area.

9. An automated detection device for web page elements, characterized in that, include: The image acquisition module is used to poll and acquire images of at least one element to be detected in the page to be detected. The similarity attribute determination module is used to obtain two images to be processed corresponding to the same element to be detected in two adjacent polls, and to determine the similarity attribute between the two images to be processed. The similarity attribute determination module includes: A pixel information determination unit is used to determine the pixel information corresponding to each image to be processed; the pixel information includes the total number of pixels; the total number of pixels is determined based on the length and width of the image to be processed; A similarity attribute determination unit is used to determine the similarity attributes of two images to be processed based on the information of each pixel. A similarity attribute determination unit is used to determine the ratio of the total number of two pixels if the total number of pixels is different, so as to obtain the similarity attribute; An element detection module is used to detect elements in the page to be detected based on the similarity attributes of the at least one element to be detected.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the automated detection method for web page elements as described in any one of claims 1-8.

11. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the automated detection method for web page elements as described in any one of claims 1-8.

Images