A page detection method, device and device for page detection

Abstract

The embodiment of the present invention provides a page detection method, device and device for page detection. The method specifically includes: modifying the cascading style sheet CSS of the first page according to the preset exception type to obtain a second page containing an abnormal element; matching the first image corresponding to the first page and the second image corresponding to the second page to determine the position information of the abnormal element in the second image; generating the annotation information corresponding to the second image according to the exception type and position information corresponding to the abnormal element; training an anomaly detection model according to the batch of second images and the annotation information corresponding to the second images; and performing anomaly detection on the page images acquired in real time through the anomaly detection model. The embodiment of the present invention can improve the efficiency and accuracy of page UI anomaly detection.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a page detection method, device, and a device for page detection. Background Art

[0002] With the rapid development of the Internet, people's daily life, such as shopping, reading, entertainment, etc., can be easily completed online. As people are increasingly dependent on the Internet, the availability and stability of websites are particularly important.

[0003] To ensure the availability and stability of the website, it is necessary to detect the website pages, such as whether there are any UI (User Interface) anomalies, to ensure that the anomalies of the website pages are discovered in time and reduce the impact on users.

[0004] Currently, there is still no good technical solution for detecting UI (User Interface) anomalies. Detection is usually done manually, for example, by writing test cases for test coverage. However, in actual applications, there are many causes of UI anomalies. For example, it may be caused by page code issues or browser compatibility issues. Therefore, the numerous types of UI anomalies and complex scenarios make methods such as manually writing test cases difficult to implement, resulting in low efficiency and accuracy of UI anomaly detection. Summary of the Invention

[0005] The embodiments of the present invention provide a page detection method, an apparatus, and an apparatus for page detection, which can improve the efficiency and accuracy of UI anomaly detection.

[0006] In order to solve the above problem, an embodiment of the present invention discloses a page detection method, which includes:

[0007] Modify the Cascading Style Sheet (CSS) of the first page according to the preset exception type to obtain a second page containing the exception element;

[0008] matching a first image corresponding to the first page with a second image corresponding to the second page to determine position information of the abnormal element in the second image;

[0009] Generate annotation information corresponding to the second image according to the abnormality type and location information corresponding to the abnormal element;

[0010] Training an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images;

[0011] The anomaly detection model is used to detect anomalies on page images acquired in real time.

[0012] On the other hand, an embodiment of the present invention discloses a page detection device, comprising:

[0013] A style modification module is used to modify the cascading style sheet CSS of the first page according to a preset exception type to obtain a second page containing the exception element;

[0014] a position determination module, configured to match a first image corresponding to the first page with a second image corresponding to the second page, and determine position information of the abnormal element in the second image;

[0015] a labeling generation module, configured to generate labeling information corresponding to the second image according to the abnormality type and location information corresponding to the abnormal element;

[0016] A model training module, configured to train an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images;

[0017] The online detection module is used to perform anomaly detection on the page images obtained in real time through the anomaly detection model.

[0018] In another aspect, an embodiment of the present invention discloses an apparatus for page detection, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations:

[0019] Modify the Cascading Style Sheet (CSS) of the first page according to the preset exception type to obtain a second page containing the exception element;

[0020] matching a first image corresponding to the first page with a second image corresponding to the second page to determine position information of the abnormal element in the second image;

[0021] Generate annotation information corresponding to the second image according to the abnormality type and location information corresponding to the abnormal element;

[0022] Training an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images;

[0023] The anomaly detection model is used to detect anomalies on page images acquired in real time.

[0024] In another aspect, an embodiment of the present invention discloses a machine-readable medium having instructions stored thereon, which, when executed by one or more processors, causes a device to perform one or more of the page detection methods described above.

[0025] The embodiments of the present invention include the following advantages:

[0026] The embodiment of the present invention modifies the CSS of the first page without UI abnormality according to the preset abnormality type to obtain the second page with UI abnormality; and matches the first image corresponding to the first page with the second image corresponding to the second page to determine the position information of the abnormal element in the second image. Based on the above modification process, a batch of training samples can be formed with the assistance of an automated process. Each training sample can include a second image and annotation information of the second image, and the annotation information includes the abnormality type and position information corresponding to the abnormal element in the second image. Through batch training samples, an abnormality detection model can be trained, and then the abnormality detection model can be used to perform abnormality detection on the page images obtained in real time.

[0027] In the process of constructing training samples, the embodiments of the present invention modify the CSS of the real page based on the preset anomaly types obtained through pre-analysis and summary, resulting in a second page containing anomaly elements of the preset anomaly types. This basis for constructing training samples makes the constructed training samples more consistent with real-world scenarios, thereby improving the accuracy of the anomaly detection model in identifying UI anomalies.

[0028] In addition, the process of constructing training samples in the embodiment of the present invention can be implemented through an automated process, and the anomaly detection model obtained by training based on a large number of training samples can continuously improve the recognition accuracy during the training process. After the training is completed, the anomaly detection model can be used to automatically detect UI anomalies in the page, which can avoid problems such as writing a large number of test cases resulting in low testing efficiency and incomplete test case coverage resulting in inaccurate detection, and can improve the efficiency and accuracy of page UI anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0030] Figure 1 is a flowchart of a page detection method embodiment of the present invention;

[0031] Figure 2 It is a structural block diagram of an embodiment of a page detection device of the present invention;

[0032] Figure 3 is a block diagram of an apparatus 800 for page detection according to the present invention; and

[0033] Figure 4 It is a schematic diagram of the structure of the server in some embodiments of the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] Method Example

[0036] Reference Figure 1 , shows a flowchart of a page detection method embodiment of the present invention, which may specifically include the following steps:

[0037] Step 101: Modify the CSS (Cascading Style Sheets) of the first page according to the preset exception type to obtain a second page containing the exception element;

[0038] Step 102: Match the first image corresponding to the first page with the second image corresponding to the second page to determine the position information of the abnormal element in the second image;

[0039] Step 103: Generate annotation information corresponding to the second image based on the abnormality type and location information corresponding to the abnormal element;

[0040] Step 104: training an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images;

[0041] Step 105: Perform anomaly detection on the page image acquired in real time using the anomaly detection model.

[0042] The page detection method of the present invention can be used to detect whether there are UI anomalies on a website page. If a UI anomaly is present, the type of UI anomaly and the location of the UI anomaly on the page can be identified. This allows the UI anomaly information to be recorded and corresponding error information data to be generated, allowing technicians to adjust or optimize the website page based on this data.

[0043] The page detection method of the embodiment of the present invention can be applied to electronic devices, including but not limited to: servers, smart phones, voice recorders, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, car computers, desktop computers, set-top boxes, smart TVs, wearable devices, etc.

[0044] To improve the efficiency and accuracy of UI anomaly detection, embodiments of the present invention can pre-acquire log data of various types of UI anomalies actually generated online, analyze and summarize the log data, and obtain relatively common preset UI anomaly types. Training samples are constructed for various preset anomaly types, and then an anomaly detection model can be trained based on the training samples. In actual applications, the anomaly detection model can be used to automatically detect whether there are UI anomalies on the page. This can avoid problems such as low test efficiency caused by writing a large number of test cases and inaccurate detection due to incomplete test case coverage, thereby improving the efficiency and accuracy of page UI anomaly detection.

[0045] In an optional embodiment of the present invention, the preset abnormality type may include at least any one of the following: font overlap, abnormal line break, and abnormal covering.

[0046] Font overlap refers to the overlap between the previous and next lines of text on a page. Abnormal line break refers to the splitting of content that should appear on the same line onto two lines. Abnormal obscuration refers to the partial obstruction of content within certain elements on a page by other elements.

[0047] It is understood that the above three exception types are merely examples of the present invention. The present invention does not limit the preset exception types. For example, the exception types may also include image drawing errors on a page, incomplete expansion of a button on a page, or an input box on a page that is too small. For ease of description, the present invention uses the exception type of font overlap as an example. The processing procedures for other exception types are similar and can be referenced accordingly.

[0048] CSS provides a style rule for HTML (HyperText Markup Language) pages, which defines the display mode of each element in the page. According to the embodiment of the present invention, the CSS of the first page is modified according to the preset exception type, and a second page containing an abnormal element can be obtained, and the abnormal element conforms to the preset exception type. Among them, the first page refers to a normal page that does not contain a UI abnormality, and the second page refers to an abnormal page containing a UI abnormality obtained after the CSS of the first page is modified. For example, for the first page, the CSS of the first page is modified according to the exception type of font overlap, and the second page obtained contains an abnormal element, and the UI abnormality type of the abnormal element is font overlap.

[0049] In an optional embodiment of the present invention, step 101 modifies the cascading style sheet CSS of the first page according to a preset exception type to obtain a second page containing the exception element, which may specifically include:

[0050] Step S11: randomly determine a target element in the CSS of the first page;

[0051] Step S12: modify the style attribute value of the target element according to the preset exception type, modify the target element to an exception element of the exception type, and obtain a second page.

[0052] CSS style rules can be used to describe elements on a web page. Style rules consist of one or more style attributes and their values. Style attributes can include font, font size, background color, line spacing, line height, etc. In this embodiment of the present invention, by modifying the style attribute values ​​of a target element on a first page, the target element can be modified into an abnormal element.

[0053] In specific applications, the div tag is often used to define a separate block or area in an HTML page. A separate block or area in an HTML page is called a div element. The div element is often used with CSS to layout web pages, so that these div elements can be formatted using CSS. The elements in the page mentioned in the embodiments of the present invention may refer to the div element in the HTML page.

[0054] Since a page typically contains a large number of div elements, an embodiment of the present invention can randomly select a div element in the CSS of the first page as a target element. Based on a preset exception type, the style attribute value of the target element is modified to modify the target element into an abnormal element that conforms to the exception type, thereby obtaining a second page. For example, the style attribute value of the target element can be modified based on the exception type of font overlap, such as by reducing the line spacing of the target element, so that the target element can be modified into an abnormal element with font overlap, thereby obtaining a second page.

[0055] In an optional embodiment of the present invention, the method may further include: before modifying the CSS of the first page, obtaining a first image corresponding to the first page; and after modifying the cascading style sheet CSS of the first page to obtain a second page containing abnormal elements, obtaining a second image corresponding to the second page.

[0056] The first image may be a screenshot of the first page, and the first image does not contain abnormal elements of the UI abnormality. The second image may be a screenshot of the second page, and the second image contains abnormal elements that conform to the preset UI abnormality type obtained by modifying the target elements of the first page.

[0057] After obtaining the first picture corresponding to the first page and the second picture corresponding to the second page, the first picture and the second picture may be matched to determine the position information of the abnormal element in the second picture.

[0058] In an optional embodiment of the present invention, step 102 of matching the first image corresponding to the first page with the second image corresponding to the second page to determine the location information of the abnormal element in the second image may specifically include:

[0059] Step S21: using edge detection technology to identify structural elements in the first image and the second image respectively;

[0060] Step S22: Compare the structuring elements in the first image and the second image one by one, and determine that the structuring elements in the second image and the corresponding positions in the first image that do not match are abnormal elements;

[0061] Step S23: Match the area image corresponding to the abnormal element with the second image to determine the position information of the abnormal element in the second image.

[0062] Edge detection is a fundamental problem in image processing and computer vision. Its purpose is to identify points in an image where significant changes occur. Significant changes in an image typically reflect important events or changes in properties, such as discontinuities in depth, surface orientation, changes in material properties, and changes in scene lighting. Edge detection can be used to divide an image into regions where pixels within the same region share common or similar characteristics.

[0063] An HTML page is generally composed of a large number of div elements. From the image point of view, there are generally significant structural changes such as depth discontinuity and scene lighting changes between different div elements. Therefore, the structuring elements described in the embodiment of the present invention refer to the div elements in the first page and the second page. According to the display area of ​​each div element in the first page and the second page in the first picture and the second picture, the edge detection is performed on the first picture and the second picture, and the structuring elements in the first picture and the second picture can be identified. The structuring elements are the div elements in the first page and the second page.

[0064] Thus, the embodiment of the present invention can divide the first image and the second image into several regions, each region corresponding to a structured element (such as a div element). The edge detection algorithm can use operators such as Canny and Sobel, which are not limited in the embodiment of the present invention.

[0065] Alternatively, if the first and second images are color images, they can be grayscaled to obtain grayscale images before edge detection. The resulting grayscale images are color-independent, and points with sudden grayscale changes within their neighborhoods can be used as edges to improve edge detection accuracy. The grayscale images can then be downsampled using an adaptive downsampling rate determined based on their size. Edge detection can then be performed on the downsampled images to improve edge detection efficiency.

[0066] After identifying the structuring elements in the first and second images respectively through edge detection, the structuring elements in the two images can be compared one by one to determine that the structuring elements in the second image that do not match the corresponding positions in the first image are abnormal elements. In this way, the abnormal elements can be found in the second image.

[0067] In an optional embodiment of the present invention, after step S22 compares the structural elements in the first image and the second image one by one, the method may further include:

[0068] If it is determined that all structuring elements in the second picture match the structuring elements in the first picture, or if it is determined that the number of structuring elements in the second picture that do not match those in the first picture exceeds a preset value, the second picture is determined to be a failed picture.

[0069] In a specific application, after the CSS of the first page is modified, there may be a situation where the modification fails. For example, a target element is randomly determined in the CSS of the first page, and the style attribute value of the target element is modified according to a preset exception type (such as font overlap) to obtain a second page. If the difference in the modified style attribute value of the target element is small, it may result in a situation where the target element in the second page does not have font overlap. In this way, after comparing the structured elements in the first image and the second image one by one, it is found that the structured elements in the second image all match the structured elements in the first image, and there are no unmatched structured elements, that is, after modifying the CSS of the first page, the first page does not change, indicating that this modification has failed. In this embodiment of the present invention, the second image is determined as a failed image, the failed image can be discarded, and the CSS of the first page can be modified again.

[0070] In specific applications, there may be another situation where the modification fails. For example, the target element is randomly determined in the CSS of the first page, and the style attribute value of the target element is modified according to the preset exception type (such as font overlap). Since the target element is randomly determined, the target element may be a div element in the outer structure of the HTML page. The div elements in the HTML page are usually nested in layers. Changes to the div elements of the inner structure may have a smaller impact on the overall structure of the page, while changes to the div elements of the outer structure may have a greater impact on the overall structure of the page. For example, changes in the style attribute value of a div element may cause the positions of multiple other div elements to be misaligned, thereby affecting the overall structure of the page. Alternatively, if the difference in the modified style attribute value of the target element is large, it may also affect the display of other div elements.

[0071] Thus, after comparing the structural elements in the first and second images one by one, it may be found that the second image contains a large number of structural elements that do not match the first image. This also indicates that the modification has failed. In this embodiment of the present invention, the second image is determined to be a failed image, the failed image can be discarded, and the CSS of the first page can be modified again.

[0072] Optionally, in the process of comparing the structuring elements in the first image and the second image one by one, the unmatched structuring elements can be counted. If it is determined that the number of structuring elements in the second image that do not match the first image exceeds a preset value, the second image is determined to be a failed image.

[0073] To minimize modification failures, the embodiment of the present invention can optionally determine the target element from a div element within the inner structure of the first page when randomly determining the target element in the CSS of the first page. This can minimize the possibility that modifications to the outer div element will cause other div elements to be misaligned.

[0074] In addition, the number of target elements modified each time may be limited. For example, only one target element in the CSS of the first page may be modified each time to reduce the probability of modification failure.

[0075] After finding the abnormal element in the second image, it is necessary to determine the location of the abnormal element in the second image. Specifically, a screenshot of the area where the abnormal element is located can be taken to obtain an image of the area corresponding to the abnormal element. By matching the image of the area corresponding to the abnormal element with the second image, the location of the abnormal element in the second image can be determined.

[0076] In an embodiment of the present invention, template matching can be performed between the regional image corresponding to the abnormal element and the image corresponding to the second image. Template matching is a technique used to find a given template image T (i.e., the regional image corresponding to the abnormal element) in a source image S (i.e., the image corresponding to the second image). The principle is to measure the similarity (S, T) between the two images using some similarity criteria. Template matching technology can be used to locate the location information of the regional image corresponding to the abnormal element in the second image. This location information can be the coordinates of the abnormal element in the second image, such as the coordinates of the upper left corner of the abnormal element in the second image.

[0077] After determining the position information of the abnormal element in the second image, annotation information corresponding to the second image can be generated according to the abnormal type and position information corresponding to the abnormal element to obtain training samples for training the abnormality detection model.

[0078] In an optional embodiment of the present invention, step 103 generates annotation information corresponding to the second image based on the abnormal type and position information corresponding to the abnormal element, which may specifically include: writing the abnormal type and position information corresponding to the abnormal element into a preset file, and generating a annotation file corresponding to the second image.

[0079] Specifically, after determining the location information of the abnormal element in the second image, the abnormality type and location information corresponding to the abnormal element can be written into a preset file to generate a labeling file corresponding to the second image. The labeling file is used to record the labeling information corresponding to the second image, and the labeling information includes the abnormality type and location information corresponding to the abnormal element in the second image. The preset file may include a JSON file or an XML file, etc.

[0080] Therefore, a second image and a labeling file corresponding to the second image may be used as a training sample, where the labeling file includes abnormality type and position information corresponding to abnormal elements in the second image.

[0081] In an optional embodiment of the present invention, step 104 may include training an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images. Specifically, the training may include:

[0082] Step S31: Use the second image and the annotation file corresponding to the second image as training samples;

[0083] Step S32: construct a training sample set based on the batch of second images and the annotation files corresponding to the second images;

[0084] Step S33: train an anomaly detection model based on the training sample set.

[0085] The embodiment of the present invention can collect a large number of first pages, and automatically perform steps 101 to 103 on the collected first pages, thereby automatically obtaining a batch of second images and the annotation information corresponding to the second images.

[0086] It should be noted that the embodiment of the present invention does not limit the specific method of modifying the CSS of the first page during the automatic execution of steps 101 to 103. In an example of the present invention, it is assumed that n types of exceptions are preset, and m first pages are collected, and n and m are both positive integers. Steps 101 to 103 can be automatically executed for these m first pages. For example, for the i-th first page (1≤i≤m), one type can be selected from the n types of exceptions in turn to modify the CSS of the i-th first page to obtain the i-th second page. For example, if the first exception type is selected to modify the CSS of the first first page, the first second page is obtained; if the second exception type is selected to modify the CSS of the second first page, the second second page is obtained; and so on, until the modification of the m first pages is completed, m second pages are obtained.

[0087] For another example, n threads can be created, each corresponding to a different exception type. The n threads can simultaneously modify the CSS of n first pages according to the n exception types, and then second pages corresponding to the n exception types can be generated simultaneously until m first pages are modified, resulting in m second pages.

[0088] It should be noted that the embodiments of the present invention all use the example of determining a target element on the first page and modifying this target element to an abnormal element of a preset abnormal type. In actual application, the embodiments of the present invention do not limit the number of target elements that need to be modified on the first page, nor do they limit the type and number of preset abnormal types that can be modified for a target element.

[0089] In one example of the present invention, for a first page A1, assuming that the target element is div1 and the exception type to be modified is text overlap, the style attribute values ​​of div1 in the CSS of the first page A1 are modified so that div1 is modified into an abnormal element with text overlap. A second page A2 is obtained, and A2 includes an abnormal element with an exception type of text overlap.

[0090] In another example of the present invention, for a first page A, it is determined that the target elements therein include elements div1 and div2, and the exception type to be modified is determined to be text overlap. The style attribute values ​​of div1 and div2 in the CSS of the first page A are modified so that div1 and div2 are modified to abnormal elements with text overlap. A second page A3 is obtained, and A3 includes two abnormal elements, both of which have the exception type of text overlap.

[0091] In another example of the present invention, for a first page A, it is assumed that the target elements therein include elements div1 and div2, and that the abnormal types to be modified are text overlap and abnormal line break. The style attribute values ​​of div1 and div2 in the CSS of the first page A are modified so that div1 is modified to an abnormal element with text overlap, and div2 is modified to an abnormal element with abnormal line break, resulting in a second page A4. A4 then includes two abnormal elements, and the abnormal types of the two abnormal elements are text overlap and abnormal line break, respectively.

[0092] Each time the CSS of a first page is automatically modified according to a preset exception type to obtain a second page, the first image corresponding to the first page and the second image corresponding to the second page can be automatically matched to determine the position information of the abnormal element in the second page, and then the abnormal type and position information corresponding to the abnormal element can be used as the annotation information of the second image and automatically written into the preset file. Thus, the embodiment of the present invention can automatically obtain a batch of second images and the annotation information corresponding to the second images, and then a training sample set can be constructed to train an anomaly detection model. Each training sample in the training sample set includes a second image and the annotation information corresponding to the second image. The annotation information can be a json file or an xml file, which records the abnormal type and position information corresponding to the abnormal element in the second image.

[0093] The anomaly detection model can be obtained by supervised training of an existing neural network based on a large number of training samples and machine learning methods. It should be noted that the embodiment of the present invention does not limit the model structure and training method of the anomaly detection model. The anomaly detection model can be a classification model that integrates multiple neural networks. The neural network includes but is not limited to a combination, superposition, or nesting of at least one or at least two of the following: CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory) network, RNN (Simple Recurrent Neural Network), attention neural network, etc. In an optional embodiment of the present invention, the anomaly detection model may include a neural network model based on YOLO.

[0094] First, the embodiment of the present invention can construct and initialize an anomaly detection model based on the YOLO model structure and set the model parameters of the initial model; then, the training samples are input into the initial model one by one, and the initial model is iteratively optimized according to the difference between the output results of the initial model and the annotation information in the training samples and the gradient descent algorithm, and the model parameters are adjusted. When the optimized model meets the preset conditions, the iterative optimization is stopped, and the model obtained by the last optimization is used as the trained anomaly detection model.

[0095] It should be noted that the embodiments of the present invention do not restrict the type of YOLO model. For example, any of YOLOv1, YOLOv2, and YOLOv3 can be used. Preferably, the embodiments of the present invention use the YOLOv3 model structure. YOLOv3 is the third version of the YOLO series of object detection algorithms. Compared to previous algorithms, it has significantly improved accuracy in detecting small objects.

[0096] It should be noted that, in embodiments of the present invention, a training sample set of a single anomaly type can be constructed to train an anomaly detection model for detecting a certain type of UI anomaly. For example, a training sample set consisting of training samples of the font overlap anomaly type can be constructed, and an anomaly detection model trained based on this training sample set can be used to detect UI anomalies caused by font overlap on a page. Alternatively, a training sample set consisting of training samples of the abnormal line break anomaly type can be constructed, and an anomaly detection model trained based on this training sample set can be used to detect UI anomalies caused by abnormal line break on a page, etc.

[0097] Alternatively, embodiments of the present invention can construct a training sample set that includes multiple anomaly types and train an anomaly detection model for detecting multiple anomaly types. For example, if the constructed training sample set includes five anomaly types, the trained anomaly detection model can output the probability values ​​of anomaly elements corresponding to each of the five anomaly types, as well as the location information corresponding to the anomaly elements of each anomaly type.

[0098] In an optional embodiment of the present invention, the second image in the training sample set may be preprocessed before using it to train the anomaly detection model. This preprocessing may include binarization, which sets the grayscale values ​​of pixels in the image to 0 or 255, effectively rendering the entire image in a distinct black and white effect. Binarization significantly reduces the amount of data in the image, thereby highlighting the outlines of each element in the second image. Training the anomaly detection model based on the preprocessed training sample set can further improve the anomaly detection model's recognition accuracy.

[0099] In an optional embodiment of the present invention, step 105 performs anomaly detection on the page image acquired in real time using the anomaly detection model, which may specifically include:

[0100] Step S41: Acquire the page to be detected in real time;

[0101] Step S42: taking a screenshot of the page to be detected to obtain a page image corresponding to the page to be detected;

[0102] Step S43: input the page image into a trained anomaly detection model, and output the anomaly type and location information corresponding to the abnormal elements in the page image through the anomaly detection model.

[0103] After the anomaly detection model is trained, the trained anomaly detection model can be used to detect whether there is a UI anomaly on the page. If there is a UI anomaly, the anomaly detection model can output the anomaly type corresponding to the abnormal element with the UI anomaly and the position of the abnormal element on the page.

[0104] Specifically, the page to be detected can be obtained in real time, a screenshot of the page to be detected can be taken, and a page image corresponding to the page to be detected can be obtained; the page image is input into the trained anomaly detection model, and the anomaly detection model can output the anomaly type and location information of the UI anomaly corresponding to the abnormal element in the page image.

[0105] Optionally, after performing online real-time detection of pages using an anomaly detection model, embodiments of the present invention can collect anomaly detection results. The anomaly detection results include the detected abnormal pages containing UI anomalies, as well as the anomaly type and location information corresponding to the abnormal elements in the abnormal pages. The anomaly detection results can then be used as training samples to further train the anomaly detection model, thereby continuously optimizing the anomaly detection model.

[0106] In summary, the embodiment of the present invention modifies the CSS of the first page without UI abnormality according to the preset abnormality type to obtain the second page with UI abnormality; and matches the first image corresponding to the first page with the second image corresponding to the second page to determine the position information of the abnormal element in the second image. Based on the above modification process, batches of training samples can be formed with the assistance of an automated process. Each training sample can include a second image and annotation information of the second image, and the annotation information includes the abnormality type and position information corresponding to the abnormal element in the second image. Through batch training samples, an abnormality detection model can be trained, and then the abnormality detection model can be used to perform abnormality detection on page images acquired in real time.

[0107] In the process of constructing training samples, the embodiments of the present invention modify the CSS of the real page based on the preset anomaly types obtained through pre-analysis and summary, resulting in a second page containing anomaly elements of the preset anomaly types. This basis for constructing training samples makes the constructed training samples more consistent with real-world scenarios, thereby improving the accuracy of the anomaly detection model in identifying UI anomalies.

[0108] In addition, the process of constructing training samples in the embodiment of the present invention can be implemented through an automated process, and the anomaly detection model obtained by training based on a large number of training samples can continuously improve the recognition accuracy during the training process. After the training is completed, the anomaly detection model can be used to automatically detect UI anomalies in the page, which can avoid problems such as writing a large number of test cases resulting in low testing efficiency and incomplete test case coverage resulting in inaccurate detection, and can improve the efficiency and accuracy of page UI anomaly detection.

[0109] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0110] Device embodiment

[0111] Reference Figure 2 , shows a structural block diagram of an embodiment of a page detection device of the present invention, the device may specifically include:

[0112] The style modification module 201 is used to modify the cascading style sheet CSS of the first page according to a preset exception type to obtain a second page containing the exception element;

[0113] a position determination module 202, configured to match a first image corresponding to the first page with a second image corresponding to the second page, and determine position information of the abnormal element in the second image;

[0114] Annotation generating module 203, configured to generate annotation information corresponding to the second image according to the abnormality type and location information corresponding to the abnormal element;

[0115] A model training module 204 is configured to train an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images;

[0116] The online detection module 205 is used to perform anomaly detection on the page image obtained in real time using the anomaly detection model.

[0117] Optionally, the style modification module 201 may specifically include:

[0118] a target determination submodule, configured to randomly determine a target element in the CSS of the first page;

[0119] The target modification submodule is used to modify the style attribute value of the target element according to the preset exception type, modify the target element into an exception element of the exception type, and obtain a second page.

[0120] Optionally, the location determination module 202 may specifically include:

[0121] an edge detection submodule, configured to use edge detection technology to respectively identify structural elements in the first image and the second image;

[0122] an element matching submodule, configured to compare the structured elements in the first image and the second image one by one, and determine that the structured elements in the second image and the corresponding positions in the first image that do not match are abnormal elements;

[0123] The template matching submodule is used to match the area image corresponding to the abnormal element with the second image to determine the position information of the abnormal element in the second image.

[0124] Optionally, the device may further include:

[0125] A failed image determination module is used to determine that the second image is a failed image if it is determined that all structuring elements in the second image match the structuring elements in the first image, or if it is determined that the number of structuring elements in the second image that do not match those in the first image exceeds a preset value.

[0126] Optionally, the annotation generating module 203 is specifically configured to write the abnormality type and position information corresponding to the abnormal element into a preset file, and generate an annotation file corresponding to the second image.

[0127] Optionally, the model training module 204 may specifically include:

[0128] a sample determination submodule, configured to use the second image and the annotation file corresponding to the second image as training samples;

[0129] A set construction submodule, configured to construct a training sample set based on the batch of second images and the annotation files corresponding to the second images;

[0130] The model training submodule is used to train the anomaly detection model based on the training sample set.

[0131] Optionally, the online detection module 205 may specifically include:

[0132] The page acquisition submodule is used to obtain the page to be detected in real time;

[0133] The image acquisition submodule is used to take a screenshot of the page to be detected and obtain a page image corresponding to the page to be detected;

[0134] The model detection submodule is used to input the page image into a trained anomaly detection model, and output the anomaly type and location information corresponding to the abnormal elements in the page image through the anomaly detection model.

[0135] Optionally, the anomaly detection model includes a YOLO-based neural network model.

[0136] Optionally, the preset abnormality type includes at least any one of the following: font overlap, abnormal line break, and abnormal covering.

[0137] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0138] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0139] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0140] An embodiment of the present invention provides a device for page detection, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: modifying the cascading style sheet (CSS) of a first page according to a preset exception type to obtain a second page containing an abnormal element; matching a first image corresponding to the first page with a second image corresponding to the second page to determine position information of the abnormal element in the second image; generating annotation information corresponding to the second image according to the exception type and position information corresponding to the abnormal element; training an anomaly detection model based on a batch of second images and the annotation information corresponding to the second images; and performing anomaly detection on page images acquired in real time using the anomaly detection model.

[0141] Figure 3 FIG1 is a block diagram of an apparatus 800 for page detection according to an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0142] Reference Figure 3 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0143] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0144] The memory 804 is configured to store various types of data to support the operations of the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0145] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0146] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0147] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice information processing mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0148] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0149] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor assembly 814 can also detect changes in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and temperature changes of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0150] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency information processing (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0151] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0152] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the apparatus 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0153] Figure 4 1 is a schematic diagram of the structure of a server in some embodiments of the present invention. The server 1900 may vary greatly due to different configurations or performance, and may include one or more central processing units (CPUs) 1922 (for example, one or more processors) and memory 1932, and one or more storage media 1930 (for example, one or more mass storage devices) storing application programs 1942 or data 1944. Among them, the memory 1932 and the storage medium 1930 can be temporary storage or permanent storage. The program stored in the storage medium 1930 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1922 can be configured to communicate with the storage medium 1930 to execute a series of instruction operations in the storage medium 1930 on the server 1900.

[0154] The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input and output interfaces 1958, one or more keyboards 1956, and / or one or more operating systems 1941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0155] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of a device (server or terminal), enables the device to perform Figure 1 The page detection method shown.

[0156] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a device (server or terminal), enables the device to perform a page detection method, the method comprising: modifying the cascading style sheet (CSS) of a first page according to a preset exception type to obtain a second page containing an abnormal element; matching a first image corresponding to the first page with a second image corresponding to the second page to determine the position information of the abnormal element in the second image; generating annotation information corresponding to the second image according to the exception type and position information corresponding to the abnormal element; training an anomaly detection model based on batches of second images and the annotation information corresponding to the second images; and performing anomaly detection on page images acquired in real time using the anomaly detection model.

[0157] The embodiment of the present invention discloses A1, a page detection method, comprising:

[0158] Modify the Cascading Style Sheet (CSS) of the first page according to the preset exception type to obtain a second page containing the exception element;

[0159] matching a first image corresponding to the first page with a second image corresponding to the second page to determine position information of the abnormal element in the second image;

[0160] Generate annotation information corresponding to the second image according to the abnormality type and location information corresponding to the abnormal element;

[0161] Training an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images;

[0162] The anomaly detection model is used to detect anomalies on page images acquired in real time.

[0163] A2. The method according to A1, wherein the cascading style sheet (CSS) of the first page is modified according to a preset exception type to obtain a second page containing the exception element, including:

[0164] Randomly determine a target element in the CSS of the first page;

[0165] According to the preset exception type, the style attribute value of the target element is modified, and the target element is modified into an exception element of the exception type to obtain a second page.

[0166] A3. According to the method of A1, matching the first image corresponding to the first page with the second image corresponding to the second page to determine the position information of the abnormal element in the second image includes:

[0167] Using edge detection technology, identifying structural elements in the first image and the second image respectively;

[0168] Comparing the structuring elements in the first image and the second image one by one, and determining that the structuring elements at corresponding positions in the second image and the first image that do not match are abnormal elements;

[0169] Match the area image corresponding to the abnormal element with the second image to determine the position information of the abnormal element in the second image.

[0170] A4. The method according to A3, wherein after comparing the structural elements in the first image and the second image one by one, the method further comprises:

[0171] If it is determined that all structuring elements in the second picture match the structuring elements in the first picture, or if it is determined that the number of structuring elements in the second picture that do not match those in the first picture exceeds a preset value, the second picture is determined to be a failed picture.

[0172] A5. The method according to A1, wherein generating annotation information corresponding to the second image based on the abnormality type and location information corresponding to the abnormal element includes:

[0173] The abnormality type and location information corresponding to the abnormal element are written into a preset file to generate a labeling file corresponding to the second image.

[0174] A6. The method according to A5, wherein the training of the anomaly detection model based on the batch of second images and the annotation information corresponding to the second images includes:

[0175] using the second image and the annotation file corresponding to the second image as training samples;

[0176] Constructing a training sample set based on the batch of second images and the annotation files corresponding to the second images;

[0177] An anomaly detection model is trained based on the training sample set.

[0178] A7. The method according to A1, wherein the anomaly detection model is used to perform anomaly detection on the page image acquired in real time, including:

[0179] Get the page to be tested in real time;

[0180] Taking a screenshot of the page to be detected to obtain a page image corresponding to the page to be detected;

[0181] The page image is input into a trained anomaly detection model, and the anomaly detection model outputs anomaly type and location information corresponding to the abnormal elements in the page image.

[0182] A8. According to any one of the methods described in A1 to A7, the preset abnormality type includes at least one of the following: font overlap, abnormal line break, and abnormal masking.

[0183] The embodiment of the present invention discloses B9, a page detection device, comprising:

[0184] A style modification module is used to modify the cascading style sheet CSS of the first page according to a preset exception type to obtain a second page containing the exception element;

[0185] a position determination module, configured to match a first image corresponding to the first page with a second image corresponding to the second page, and determine position information of the abnormal element in the second image;

[0186] a labeling generation module, configured to generate labeling information corresponding to the second image according to the abnormality type and location information corresponding to the abnormal element;

[0187] A model training module, configured to train an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images;

[0188] The online detection module is used to perform anomaly detection on the page images obtained in real time through the anomaly detection model.

[0189] B10. The apparatus according to B9, wherein the style modification module comprises:

[0190] a target determination submodule, configured to randomly determine a target element in the CSS of the first page;

[0191] The target modification submodule is used to modify the style attribute value of the target element according to the preset exception type, modify the target element into an exception element of the exception type, and obtain a second page.

[0192] B11. The apparatus according to B9, wherein the location determination module comprises:

[0193] an edge detection submodule, configured to use edge detection technology to respectively identify structural elements in the first image and the second image;

[0194] an element matching submodule, configured to compare the structured elements in the first image and the second image one by one, and determine that the structured elements in the second image and the corresponding positions in the first image that do not match are abnormal elements;

[0195] The template matching submodule is used to match the area image corresponding to the abnormal element with the second image to determine the position information of the abnormal element in the second image.

[0196] B12. The device according to B11, further comprising:

[0197] A failed image determination module is used to determine that the second image is a failed image if it is determined that all structuring elements in the second image match the structuring elements in the first image, or if it is determined that the number of structuring elements in the second image that do not match those in the first image exceeds a preset value.

[0198] B13. According to the device described in B9, the annotation generation module is specifically used to write the abnormality type and location information corresponding to the abnormal element into a preset file to generate an annotation file corresponding to the second image.

[0199] B14. The apparatus according to B13, wherein the model training module comprises:

[0200] a sample determination submodule, configured to use the second image and the annotation file corresponding to the second image as training samples;

[0201] A set construction submodule, configured to construct a training sample set based on the batch of second images and the annotation files corresponding to the second images;

[0202] The model training submodule is used to train the anomaly detection model based on the training sample set.

[0203] B15. The apparatus according to B9, wherein the model training module comprises:

[0204] The page acquisition submodule is used to obtain the page to be detected in real time;

[0205] The image acquisition submodule is used to take a screenshot of the page to be detected and obtain a page image corresponding to the page to be detected;

[0206] The model detection submodule is used to input the page image into a trained anomaly detection model, and output the anomaly type and location information corresponding to the abnormal elements in the page image through the anomaly detection model.

[0207] B16. According to any one of the devices described in B9 to B15, the preset abnormality type includes at least one of the following: font overlap, abnormal line break, and abnormal masking.

[0208] An embodiment of the present invention discloses C17, a device for page detection, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations:

[0209] Modify the Cascading Style Sheet (CSS) of the first page according to the preset exception type to obtain a second page containing the exception element;

[0210] matching a first image corresponding to the first page with a second image corresponding to the second page to determine position information of the abnormal element in the second image;

[0211] Generate annotation information corresponding to the second image according to the abnormality type and location information corresponding to the abnormal element;

[0212] Training an anomaly detection model based on the batch of second images and the annotation information corresponding to the second images;

[0213] The anomaly detection model is used to detect anomalies on page images acquired in real time.

[0214] C18. The apparatus according to C17, wherein the step of modifying the Cascading Style Sheet (CSS) of the first page according to the preset exception type to obtain a second page containing the exception element comprises:

[0215] Randomly determine a target element in the CSS of the first page;

[0216] According to the preset exception type, the style attribute value of the target element is modified, and the target element is modified into an exception element of the exception type to obtain a second page.

[0217] C19. The apparatus according to C17, wherein matching the first image corresponding to the first page with the second image corresponding to the second page to determine the location information of the abnormal element in the second image includes:

[0218] Using edge detection technology, identifying structural elements in the first image and the second image respectively;

[0219] Comparing the structuring elements in the first image and the second image one by one, and determining that the structuring elements at corresponding positions in the second image and the first image that do not match are abnormal elements;

[0220] Match the area image corresponding to the abnormal element with the second image to determine the position information of the abnormal element in the second image.

[0221] C20. The device according to C19, wherein the device is further configured to execute, by one or more processors, the one or more programs including instructions for performing the following operations:

[0222] If it is determined that all structuring elements in the second picture match the structuring elements in the first picture, or if it is determined that the number of structuring elements in the second picture that do not match those in the first picture exceeds a preset value, the second picture is determined to be a failed picture.

[0223] C21. The apparatus according to C17, wherein generating annotation information corresponding to the second image based on the abnormality type and location information corresponding to the abnormal element comprises:

[0224] The abnormality type and location information corresponding to the abnormal element are written into a preset file to generate a labeling file corresponding to the second image.

[0225] C22. The apparatus according to C21, wherein the training of the anomaly detection model based on the batch of second images and the annotation information corresponding to the second images comprises:

[0226] using the second image and the annotation file corresponding to the second image as training samples;

[0227] Constructing a training sample set based on the batch of second images and the annotation files corresponding to the second images;

[0228] An anomaly detection model is trained based on the training sample set.

[0229] C23. The apparatus according to C17, wherein the abnormality detection model is used to perform abnormality detection on the page image acquired in real time, comprising:

[0230] Get the page to be tested in real time;

[0231] Taking a screenshot of the page to be detected to obtain a page image corresponding to the page to be detected;

[0232] The page image is input into a trained anomaly detection model, and the anomaly detection model outputs anomaly type and location information corresponding to the abnormal elements in the page image.

[0233] C24. According to any one of the devices described in C17 to C23, the preset abnormality type includes at least one of the following: font overlap, abnormal line break, and abnormal masking.

[0234] An embodiment of the present invention discloses D25, a machine-readable medium having instructions stored thereon, which, when executed by one or more processors, causes a device to perform a page detection method as described in one or more of A1 to A8.

[0235] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

[0236] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0237] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0238] The above is a detailed introduction to a page detection method, a page detection device and a device for page detection provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A page detection method, characterized in that: The method comprises: According to the preset exception type, the cascading style sheet CSS of the first page is modified to obtain a second page containing the exception element, including: Randomly determining a target element in the CSS of the first page includes: randomly determining a target element in a div element of an inner structure of the first page; According to the preset exception type, modify the style attribute value of the target element, modify the target element to an exception element of the exception type, and obtain a second page; the preset exception type is obtained by analyzing log data; grayscale the first picture corresponding to the first page and the second picture corresponding to the second page to obtain a grayscale image, and adaptively determine a downsampling rate according to the size of the grayscale image to downsample the grayscale image; Performing edge detection on the downsampled image to respectively identify structural elements in the first image and the second image; Matching the structural elements in the first picture and the second picture to determine the position information of the abnormal element in the second picture; Generate annotation information corresponding to the second image according to the abnormal type and position information corresponding to the abnormal element; According to the batch of second images and the annotation information corresponding to the second images, the anomaly detection model is trained; the training of the anomaly detection model includes: constructing training samples through an automated process, and training the anomaly detection model according to the batch of training samples; the construction of the training samples through an automated process includes: presetting n types of anomalies, and collecting m first pages, where n and m are both positive integers; for the i-th first page, 1≤i≤m, selecting one of the n types of anomalies in turn to modify the CSS of the i-th first page to obtain the i-th second page; or, establishing n threads, each thread corresponding to an anomaly type; modifying the CSS of the n first pages according to the n types of anomalies through n threads at the same time, and generating second pages corresponding to the n types of anomalies at the same time, until the modification of the m first pages is completed, and m second pages are obtained; automatically obtaining the batch of second images and the annotation information corresponding to the second images; the anomaly detection model is a classification model that integrates multiple neural networks; Before using the training sample set to train the anomaly detection model, preprocessing the second image in the training sample set, wherein the preprocessing includes binarization processing; The anomaly detection model is used to perform anomaly detection on the page images acquired in real time.

2. The method according to claim 1, characterized in that The matching of the structural elements in the first picture and the second picture to determine the position information of the abnormal element in the second picture includes: Using edge detection technology to identify structural elements in the first picture and the second picture respectively; Compare the structured elements in the first image and the second image one by one, and determine that the structured elements in the second image and the corresponding positions in the first image that do not match are abnormal elements; The regional image corresponding to the abnormal element is matched with the second image to determine the position information of the abnormal element in the second image.

3. The method according to claim 2, characterized in that After comparing the structural elements in the first picture and the second picture one by one, the method further includes: If it is determined that all the structuring elements in the second picture match the structuring elements in the first picture, or if it is determined that the number of structuring elements in the second picture that do not match the first picture exceeds a preset value, the second picture is determined to be a failed picture.

4. The method according to claim 1, characterized in that: The generating, according to the abnormal type and position information corresponding to the abnormal element, annotation information corresponding to the second image includes: The abnormal type and position information corresponding to the abnormal element are written into a preset file, and a labeling file corresponding to the second image is generated.

5. The method according to claim 4, characterized in that The training of the anomaly detection model according to the batch of second images and the annotation information corresponding to the second images includes: Using the second image and the annotation file corresponding to the second image as training samples; Constructing a training sample set according to the batch of second images and the annotation files corresponding to the second images; An anomaly detection model is trained based on the training sample set.

6. The method according to claim 1, characterized in that The abnormality detection model is used to perform abnormality detection on the page image acquired in real time, including: Get the page to be detected in real time; Taking a screenshot of the page to be detected to obtain a page image corresponding to the page to be detected; The page image is input into a trained anomaly detection model, and the anomaly detection model outputs anomaly types and location information corresponding to abnormal elements in the page image.

7. A page detection device, characterized in that: The device comprises: The style modification module is used to modify the cascading style sheet CSS of the first page according to the preset exception type to obtain a second page containing the exception element, including: The target determination submodule is used to randomly determine the target element in the CSS of the first page, including: randomly determining the target element in the div element of the inner structure of the first page; A target modification submodule is used to modify the style attribute value of the target element according to the preset exception type, modify the target element to an exception element of the exception type, and obtain a second page; the preset exception type is obtained by analyzing log data; a position determination module, configured to grayscale a first image corresponding to a first page and a second image corresponding to a second page to obtain a grayscale image, and adaptively determine a downsampling rate according to a size of the grayscale image to downsample the grayscale image; perform edge detection on the downsampled image to respectively identify structural elements in the first image and the second image; match structural elements in the first image and the second image to determine position information of the abnormal element in the second image; A labeling generation module, used to generate labeling information corresponding to the second image according to the abnormal type and position information corresponding to the abnormal element; A model training module is used to train an anomaly detection model according to a batch of second images and the annotation information corresponding to the second images; the training of the anomaly detection model includes: constructing training samples through an automated process, and training the anomaly detection model according to the batch of training samples; the constructing of the training samples through an automated process includes: presetting n types of anomalies, and collecting m first pages, where n and m are both positive integers; for the i-th first page, 1≤i≤m, selecting one of the n types of anomalies in turn to modify the CSS of the i-th first page to obtain the i-th second page; or, establishing n threads, each thread corresponding to an anomaly type; modifying the CSS of the n first pages according to the n types of anomalies through n threads at the same time, and generating second pages corresponding to the n types of anomalies at the same time, until the modification of the m first pages is completed, and obtaining m second pages; automatically obtaining a batch of second images and the annotation information corresponding to the second images; the anomaly detection model is a classification model that integrates multiple neural networks; before using the training sample set to train the anomaly detection model, preprocessing the second images in the training sample set, and the preprocessing includes binarization processing; The online detection module is used to perform anomaly detection on the page images acquired in real time through the anomaly detection model.

8. A device for page detection, characterized in that: The system comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: According to a preset abnormality type, modifying the cascading style sheet CSS of the first page to obtain a second page containing abnormal elements, including: randomly determining a target element in the CSS of the first page, including: randomly determining a target element in a div element of an inner structure of the first page; according to the preset abnormality type, modifying the style attribute value of the target element, modifying the target element to an abnormal element of the abnormal type, and obtaining a second page; the preset abnormality type is obtained by analyzing log data; grayscale the first picture corresponding to the first page and the second picture corresponding to the second page to obtain a grayscale image, and adaptively determine a downsampling rate according to the size of the grayscale image to downsample the grayscale image; Performing edge detection on the downsampled image to respectively identify structural elements in the first image and the second image; Matching the structural elements in the first picture and the second picture to determine the position information of the abnormal element in the second picture; Generate annotation information corresponding to the second image according to the abnormal type and position information corresponding to the abnormal element; According to the batch of second images and the annotation information corresponding to the second images, the anomaly detection model is trained; the training of the anomaly detection model includes: constructing training samples through an automated process, and training the anomaly detection model according to the batch of training samples; the construction of the training samples through an automated process includes: presetting n types of anomalies, and collecting m first pages, where n and m are both positive integers; for the i-th first page, 1≤i≤m, selecting one of the n types of anomalies in turn to modify the CSS of the i-th first page to obtain the i-th second page; or, establishing n threads, each thread corresponding to an anomaly type; modifying the CSS of the n first pages according to the n types of anomalies through n threads at the same time, and generating second pages corresponding to the n types of anomalies at the same time, until the modification of the m first pages is completed, and m second pages are obtained; automatically obtaining the batch of second images and the annotation information corresponding to the second images; the anomaly detection model is a classification model that integrates multiple neural networks; before using the training sample set to train the anomaly detection model, preprocessing the second images in the training sample set; the preprocessing includes binarization processing; The anomaly detection model is used to perform anomaly detection on the page images acquired in real time.

9. A machine-readable medium having instructions stored thereon, which, when executed by one or more processors, causes an apparatus to perform the page detection method as claimed in one or more of claims 1 to 6.

Images