Page abnormality monitoring method, system, device, electronic device and computer readable medium

By comparing the similarity and variance of web page screenshots, we automatically detect web page abnormalities, which solves the problem that R&D personnel finds difficult to understand the problem in a timely manner when page errors are made, and achieves rapid repair.

CN110704772BActive Publication Date: 2024-11-26BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN201810654762.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2018-06-22
Publication Date
2024-11-26
Estimated Expiration
2038-06-22

AI Technical Summary

Technical Problem

In website service applications, when page data or rendering errors are wrong, it is difficult for R&D personnel to understand the error situation in a timely manner, resulting in an extended page repair time.

Method used

By screenshotting the current web page, the current image is generated, and the similarity comparison is performed with the reference image. Whether there are abnormalities is judged through the similarity curve and variance, and the abnormalities on the web page are automatically detected.

Benefits of technology

It realizes timely detection of web page exceptions, shortens page repair time, and improves page reliability and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110704772B_ABST
    Figure CN110704772B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a page anomaly monitoring method, system, device, electronic device and computer-readable medium. The present disclosure relates to the field of computer information processing, and the method comprises: taking a screenshot of a current web page to generate a current image; performing a similarity comparison between the current image and a reference image to obtain a first similarity; performing a similarity comparison between the current image and a screenshot of the page at a previous moment to obtain a second similarity; and determining whether the current web page has an abnormality by using the first similarity and the second similarity. The page anomaly monitoring method, system, device, electronic device and computer-readable medium involved in the present disclosure can automatically detect abnormalities of web pages so as to repair web page defects in a timely manner.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer information processing, and in particular to a page exception monitoring method, system, device, electronic device and computer-readable medium. Background Art

[0002] In website service applications, users use the auxiliary functions provided by different pages to complete information interaction with the website service system, thereby achieving the ultimate purpose of users visiting the website. Taking shopping websites as an example, users will browse a large number of different pages from selecting products to final payment, such as: store homepage, event page, product details page, shopping cart page, and payment page, etc.

[0003] In general, if an error occurs in the data or rendering of a page, the R&D personnel will not be able to know the error immediately after it occurs. Generally, the error needs to be discovered by the business department before it is notified to the R&D department. The time period between the business department notifying the R&D department of the page anomaly and the time when the page anomaly occurs can be long or short. How to enable the page R&D personnel to know the page anomaly in a timely and rapid manner so as to quickly repair the page is a difficult problem in the prior art.

[0004] Therefore, a new page exception monitoring method, system, device, electronic device and computer-readable medium are needed.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] In view of this, the present disclosure provides a page exception monitoring method, system, device, electronic device and computer-readable medium, which can automatically detect abnormal conditions of web pages so as to repair web page defects in a timely manner.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to one aspect of the present disclosure, a page anomaly monitoring method is proposed, the method comprising: taking a screenshot of a current web page to generate a current image; performing a similarity comparison between the current image and a reference image to obtain a first similarity; performing a similarity comparison between the current image and a screenshot of the page at a previous moment to obtain a second similarity; and determining whether there is an abnormality in the current web page through the first similarity and the second similarity.

[0009] In an exemplary embodiment of the present disclosure, taking a screenshot of the current web page to generate a current image includes: taking a screenshot of the current web page to generate a first image; and graying the first image to generate the current image.

[0010] In an exemplary embodiment of the present disclosure, taking a screenshot of the current web page to generate a current image further comprises: segmenting the current image in units of pixels to obtain a predetermined number of segmented image units.

[0011] In an exemplary embodiment of the present disclosure, performing a similarity comparison between the current image and a reference image to obtain a first similarity includes: performing a similarity comparison between segmented image units in the current image and segmented image units in the reference image to obtain a first similarity.

[0012] In an exemplary embodiment of the present disclosure, each segmented image unit is respectively compared with the segmented image unit in the reference image for similarity, and obtaining the first similarity includes: recording the number of similar units between the segmented image unit in the current image and the segmented image unit in the reference image as M; recording the number of dissimilar units between the segmented image unit in the current image and the segmented image unit in the reference image as N; and determining the first similarity between the current image and the reference image by a similarity formula.

[0013] In an exemplary embodiment of the present disclosure, the similarity formula includes:

[0014] S = M / N*100%;

[0015] Among them, S is the similarity, M is the number of similar units, and N is the number of dissimilar units.

[0016] In an exemplary embodiment of the present disclosure, determining whether there is an abnormality in the current web page through the first similarity and the second similarity includes: generating a first similarity curve and a second similarity curve through the first similarity and the second similarity; and determining whether there is an abnormality in the current web page through the first similarity curve and the second similarity curve.

[0017] In an exemplary embodiment of the present disclosure, determining whether there is an abnormal situation in the current web page through the first similarity curve and the second similarity curve includes: when the first similarity curve and the second similarity curve meet the first feature, determining that the current web page is in a normal state; and when the first similarity curve and the second similarity curve meet the second feature, determining that the current web page is in a normal state.

[0018] In an exemplary embodiment of the present disclosure, determining whether there is an abnormality in the current web page through the first similarity curve and the second similarity curve includes: when the first similarity curve and the second similarity curve meet the third feature, calculating and obtaining the similarity variance; when the similarity variance is less than a predetermined threshold, determining that the current web page is in a normal state; and when the similarity variance is greater than a predetermined threshold, determining that there is an abnormality in the current web page.

[0019] In an exemplary embodiment of the present disclosure, user behavior is analyzed to generate page exception information; based on the page exception information, a screenshot of the current web page is taken to generate a current image.

[0020] According to one aspect of the present disclosure, a page anomaly monitoring system is proposed, which includes: a screenshot server, used to take a screenshot of a current web page to generate a current image; a crawler server, used to periodically obtain the current image, and compare the current image with a reference image to obtain a first similarity; compare the current image with a screenshot of a page at a previous moment to obtain a second similarity; and a network server, used to determine whether there is an abnormality in the current web page through the first similarity and the second similarity.

[0021] According to one aspect of the present disclosure, a page anomaly monitoring device is proposed, which includes: a screenshot module, used to take a screenshot of a current web page to generate a current image; a first comparison module, used to compare the similarity of the current image with a reference image to obtain a first similarity; a second comparison module, used to compare the similarity of the current image with a screenshot of the page at a previous moment to obtain a second similarity; and a judgment module, used to determine whether there is an abnormality in the current web page through the first similarity and the second similarity.

[0022] According to one aspect of the present disclosure, an electronic device is proposed, which includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0023] According to one aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0024] According to the page anomaly monitoring method, system, device, electronic device and computer-readable medium disclosed in the present invention, abnormal conditions of web pages can be automatically detected so as to repair web page defects in a timely manner.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail its exemplary embodiments with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present disclosure, and it is clear to a person skilled in the art that other accompanying drawings can be obtained from these accompanying drawings without creative work.

[0027] Figure 1 The present invention is a system block diagram showing a method and device for monitoring page abnormality according to an exemplary embodiment.

[0028] Figure 2 The figure is a flow chart of a method for monitoring page abnormality according to an exemplary embodiment.

[0029] Figure 3 The figure is a flow chart of a method for monitoring page abnormality according to another exemplary embodiment.

[0030] Figure 4 The figure is a schematic diagram of a statistical curve in a page abnormality monitoring method according to another exemplary embodiment.

[0031] Figure 5 The figure is a schematic diagram of a statistical curve in a page abnormality monitoring method according to another exemplary embodiment.

[0032] Figure 6 The figure is a schematic diagram of a statistical curve in a page abnormality monitoring method according to another exemplary embodiment.

[0033] Figure 7 The figure is a schematic diagram of a statistical curve in a page abnormality monitoring method according to another exemplary embodiment.

[0034] Figure 8 The figure is a schematic diagram of a page abnormality monitoring system according to an exemplary embodiment.

[0035] Fig. 9 The invention is a block diagram of a system for monitoring page abnormality according to an exemplary embodiment.

[0036] Fig.10 is a block diagram of a device for monitoring page abnormality according to another exemplary embodiment.

[0037] Fig.11 It is a block diagram of an electronic device according to an exemplary embodiment.

[0038] Fig.12 The present invention is a schematic diagram showing a computer-readable storage medium according to an exemplary embodiment. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0040] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0041] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0042] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0043] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the disclosed concepts. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0044] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present disclosure, and therefore cannot be used to limit the protection scope of the present disclosure.

[0045] Figure 1 The present invention is a system block diagram showing a method and device for monitoring page abnormality according to an exemplary embodiment.

[0046] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0047] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0048] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0049] The server 105 may be a server that provides various services, such as a background management server that provides page support for shopping websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as product information query requests, and feed back the processing results to the terminal device.

[0050] The server 105 may, for example, take a screenshot of the current web page to generate a current image; the server 105 may, for example, compare the current image with a reference image for similarity to obtain a first similarity; the server 105 may, for example, compare the current image with a screenshot of the page at a previous moment for similarity to obtain a second similarity; the server 105 may, for example, determine whether there is any abnormality in the current web page through the first similarity and the second similarity.

[0051] The server 105 may be a physical server, or may be composed of a plurality of servers. The server 105 may be composed of a screenshot server, a crawler server, and a network server, for example.

[0052] The screenshot server can be used to take a screenshot of the current web page to generate a current image; the crawler server can be used to periodically obtain the current image and compare the current image with a reference image to obtain a first similarity; compare the current image with a screenshot of the page at a previous moment to obtain a second similarity; and the network server can be used to determine whether there is an abnormality in the current web page through the first similarity and the second similarity.

[0053] It should be noted that the page anomaly monitoring method provided in the embodiment of the present disclosure can be executed by the server 105, and accordingly, the page anomaly monitoring device can be set in the server 105. The web page end provided for users to browse products and the monitoring end for taking page screenshots are generally located in the terminal devices 101, 102, and 103.

[0054] Figure 2 2 is a flow chart showing a method for monitoring page anomalies according to an exemplary embodiment. The method 20 for monitoring page anomalies includes at least steps S202 to S208.

[0055] like Figure 2 As shown, in S202, the current web page is screenshoted to generate a current image. For example, the current web page is screenshoted to generate a first image; and the first image is grayed to generate the current image.

[0056] Graying refers to converting the values ​​of the three channels R, G, and B of a pixel in the image into a brightness value. Since it is easy to make the coordinates and size of each screenshot of a web page completely consistent, there is no need to consider displacement when calculating the similarity of two screenshots. Moreover, web page screenshots are different from real photos and are not affected by lighting conditions. Therefore, after graying the image, the grayscale value of the pixel in the screenshot is used alone to calculate the similarity, which can meet the scenario described in the present invention.

[0057] In one embodiment, the current web page is screenshotted, and the generation of the current image further includes: segmenting the current image in pixels to obtain a predetermined number of segmented image units. The current page is screenshotted and saved as A1. A1 is first grayed, and then the image is segmented in units of X pixels horizontally and Y pixels vertically. For example, a picture with a resolution of PX*PY can be segmented in units of 10 (px)*10 (px), and a total of N units are segmented, each with 100 pixels.

[0058] In S204, the current image is compared with the reference image for similarity to obtain a first similarity. For example, the segmented image units in the current image are compared with the segmented image units in the reference image for similarity to obtain the first similarity.

[0059] In one embodiment, for example, the same page can be screenshotted twice at different times and saved as A1 and A2. A1 represents the current image and A2 represents the reference image. A1 and A2 are first grayed out, and then the images are segmented into units of X pixels horizontally and Y pixels vertically, and then the segmented units are compared in turn. A picture with a resolution of PX*PY is segmented into 10 (px)*10 (px) as the minimum unit, and a total of N units are segmented, each with 100 pixels.

[0060] The reference image may be, for example, an image of the current web page captured at any time, or may be an image of the current web page captured at a fixed time period every day, but the present application is not limited thereto.

[0061] In one embodiment, each segmented image unit is compared with the segmented image unit in the reference image for similarity, and obtaining the first similarity includes: recording the number of similar units between the segmented image unit in the current image and the segmented image unit in the reference image as M; recording the number of dissimilar units between the segmented image unit in the current image and the segmented image unit in the reference image as N; and determining the first similarity between the current image and the reference image by a similarity formula.

[0062] For example, if the number of pixels with different gray values ​​in a unit U1 in the A1 image and the corresponding unit U2 in the A2 image exceeds a preset threshold, then U1 and U2 are marked as dissimilar. The number of all similar units is M, and the number of dissimilar units is N. Then the page similarity between A1 and A2 is:

[0063] S=M / N*100%.

[0064] In S206, the current image is compared with the previous page screenshot for similarity to obtain a second similarity. For example, the segmented image units in the current image are compared with the segmented image units in the previous page screenshot for similarity to obtain the second similarity.

[0065] In one embodiment, for example, the same page can be screenshotted in two consecutive time periods and saved as A1 and A3. A1 represents the current image, and A3 represents the screenshot of the page at the previous moment. A1 and A3 are first grayed out, and then the images are segmented according to X pixels horizontally and Y pixels vertically, and then the segmented units are compared in turn. A picture with a resolution of PX*PY is segmented according to 10 (px)*10 (px) as the minimum unit, and a total of N units are segmented, each with 100 pixels.

[0066] For example, if the number of pixels with different gray values ​​in a unit U1 in the A1 image and the corresponding unit U3 in the A3 image exceeds a preset threshold, then U1 and U3 are marked as dissimilar. The number of all similar units is M, and the number of dissimilar units is N. Then the page similarity between A1 and A3 is:

[0067] S=M / N*100%.

[0068] In S208, whether the current webpage has an abnormality is determined by the first similarity and the second similarity. For example, a first similarity curve and a second similarity curve may be generated by the first similarity and the second similarity; and whether the current webpage has an abnormality is determined by the first similarity curve and the second similarity curve.

[0069] In one embodiment, when the first similarity curve and the second similarity curve satisfy a first feature, it is determined that the current webpage is in a normal state.

[0070] In one embodiment, when the first similarity curve and the second similarity curve satisfy a second characteristic, it is determined that the current webpage is in a normal state.

[0071] In one embodiment, when the first similarity curve and the second similarity curve meet the third feature, a similarity variance is calculated; when the similarity variance is less than a predetermined threshold, it is determined that the current webpage is in a normal state.

[0072] In one embodiment, when the first similarity curve and the second similarity curve meet the third feature, a similarity variance is calculated; when the similarity variance is greater than a predetermined threshold, it is determined that an abnormality exists in the current webpage.

[0073] According to the page anomaly monitoring method disclosed in the present invention, by comparing the similarity data of page screenshots taken at different times, and then judging whether there is an abnormality on the current page, it is possible to automatically detect the abnormality of the web page so as to repair the web page defects in time.

[0074] It should be clearly understood that the present disclosure describes how to form and use specific examples, but the principles of the present disclosure are not limited to any details of these examples. On the contrary, based on the teachings of the contents disclosed in the present disclosure, these principles can be applied to many other embodiments.

[0075] Based on the similarity calculation formula introduced above, the idea of ​​establishing similarity statistics for a fixed page is as follows: take a screenshot of the tracked page (denoted as P) as the baseline image (B0), then take screenshots of P at a fixed frequency, calculate the similarity between the screenshot and B0 (denoted as Scurrent), record all Scurrent values, and calculate the fluctuation range of page P based on the recorded Scurrent (such as Figure 3 As shown), if the Scurrent of the next screenshot exceeds this range, it can be preliminarily considered that the page P is abnormal.

[0076] However, due to the limitations of existing technology and the fact that pages contain dynamic content, it is difficult to achieve 100% accuracy in similarity unless the page is completely static (i.e., the page has no dynamically changing parts). Therefore, page similarity generally fluctuates. In this embodiment, such fluctuations are minimized by graying the pages. However, the similarity curve of pages in a real environment may present a more complex situation. Different situations are classified and explained here.

[0077] The following parameter definitions are made: the similarity between the current page screenshot and the reference image is recorded as S1; the similarity between the current page screenshot and the previous screenshot is recorded as S2.

[0078] In one embodiment, the page does not change over time (static page, or only a small part of the page changes dynamically). For example, the "User Personal Information" page. Except for the user balance, number of messages, and number of shopping carts, which change dynamically, the rest of the page does not change over time. The similarity statistical curve of such pages is as follows: Figure 4 As shown in the figure: Since the changing part does not show periodic changes over time, such as the user's balance, it is also related to the user's purchase behavior. Technical means can be used to exclude this part of the area when taking screenshots, so that the similarity curves S1 and S2 are simplified to quasi-straight lines.

[0079] In one embodiment, the fixed part of the page changes periodically (search result page, page with BI function).

[0080] For example, when you search for "apple" on the homepage of a shopping website, the product recommendations on one side of the page may be different each time, but the specific content of the search results is fixed for a period of time. The S1 and S2 similarity curves of such pages are as follows: Figure 5 As shown: Figure 5In the similarity curve of S1, the search results change due to the change in the relevance value of the search background items, and the similarity drops from about 80% to about 30%. However, since the change in search results is fixed in a plane range, after the first change, even if the search results change again, the similarity will remain stable at about 30%. If the page rendering is abnormal during this period, and the abnormal content is located in the plane range of the search result change, it cannot be perceived through the S1 similarity statistics.

[0081] The S2 similarity curve is introduced. Figure 5 It can be seen that the similarity change of the S2 curve only appears in the first screenshot calculation after the search results change. Because S2 is a comparison between the current screenshot and the previous screenshot, the similarity level will be restored to the previous level later. For such pages, the similarity fluctuation range can be fixed in the two intervals of [30,31] and [80,81] based on the S2 curve, or even the similarity value during the change can be eliminated, and only the interval of [80,81] can be retained.

[0082] The two types of pages mentioned above can detect abnormal situations of the pages by establishing similarity statistical curves, and use technical means to eliminate interference elements to achieve the purpose of simplifying the curve. However, this method is only applicable to static pages or pages with relatively regular changes. The following describes the judgment scenario when the similarity curve does not conform to the similarity curves in the above two embodiments.

[0083] In one embodiment, the non-fixed part of the page changes non-periodically (the merchant homepage, the channel homepage). For example, on the homepage of "XX Shopping", most of the content is dynamic, and the change frequency of different parts of the page is also different (the change frequency of the large carousel image is lower than that of the bottom purchase). This results in different changes in the similarity of the overall page screenshots each time. In addition, the change frequency of the dynamic part is unpredictable, which also results in no regularity in the change frequency of the overall page similarity (such as Figure 6 In general, such pages cannot be statistically regularized or detected using the S1 and S2 curves.

[0084] Although it is impossible to detect page anomalies through the similarity curve, the fluctuation range of similarity is still of reference value, because generally speaking, the more frequently a page changes, the greater the probability of error, and such pages need to be paid special attention. This application introduces the concept of similarity variance to describe this change, namely:

[0085] Similarity Variance

[0086] Among them, D(S) is the variance value, S k is the current similarity, S ave is the average similarity, and n is the number of screenshots.

[0087] Combining the three types of page similarity curves mentioned above, if D(S2) is calculated, the variance of type 3 pages is obviously larger than that of types 1 and 2. In this way, the variance can be used to quantify the change range of the page. When the variance is greater than the predetermined threshold, it is determined that the current web page is abnormal.

[0088] With the help of the description of different situations above, the method of judging page anomalies is introduced in detail below.

[0089] Figure 7 The page anomaly monitoring method 70 is a flow chart of a page anomaly monitoring method according to another exemplary embodiment. The page anomaly monitoring method 70 is a specific description of step S208 of the page anomaly monitoring method 20, "determining whether the current webpage has an anomaly by using the first similarity and the second similarity".

[0090] like Figure 7 As shown, in S702, the first similarity and the second similarity are obtained.

[0091] In S704, a first similarity curve and a second similarity curve are generated by using the first similarity and the second similarity.

[0092] In S706, when the first similarity curve and the second similarity curve satisfy a first feature, it is determined that the current webpage is in a normal state. The first feature may be, for example, the curve feature described in 4.

[0093] In S708, when the first similarity curve and the second similarity curve satisfy a second feature, it is determined that the current webpage is in a normal state. The second feature may be, for example, the curve feature described in step 5.

[0094] In S710, when the first similarity curve and the second similarity curve meet the third feature, a similarity variance is calculated. The third feature may be, for example, the curve feature described in step 6.

[0095] In S712, when the similarity variance is less than a predetermined threshold, it is determined that the current webpage is in a normal state.

[0096] In S714, when the similarity variance is greater than a predetermined threshold, it is determined that an abnormality exists in the current webpage.

[0097] According to the disclosed page anomaly monitoring method, not only can the anomaly monitoring of static pages and pages with relatively regular page changes be performed, but also the webpages with more dynamic content can be judged by variance judgment. That is, by actively detecting page anomalies through similarity statistics, the time from the occurrence of page anomalies to their discovery can be greatly shortened.

[0098] Figure 8 This is a schematic diagram of a page anomaly monitoring system according to an exemplary embodiment. In one embodiment, page anomaly information can be generated, for example, by analyzing user behavior; based on the page anomaly information, a screenshot of the current web page is processed to generate the current image. For example, after determining that the current web page is abnormal based on the similarity curve, the abnormality of the web page can be further confirmed by monitoring the user's specified operation. This application is not limited to this.

[0099] Among them, user behavior analysis refers to injecting scripts into the monitored page. When the user browses the page and completes certain specific operations (such as clicking, refreshing), a specific request is triggered and sent to the server, and the server then completes the corresponding data analysis for different requests.

[0100] like Figure 8 As shown, the anomaly monitoring system 800 may be composed of, for example, a similarity analysis system 802 and a user behavior analysis system 804 .

[0101] like Figure 8 As shown, the similarity analysis system 802 includes: a screenshot server 8022, a crawler server 8024, and a network server 8026. The current web page is screenshoted to generate a current image.

[0102] The crawler server 8022 is used to periodically obtain the current image, and compare the current image with the reference image to obtain a first similarity; and compare the current image with the screenshot of the page at the previous moment to obtain a second similarity.

[0103] The network server 8024 is used to determine whether there is any abnormality in the current web page through the first similarity and the second similarity.

[0104] Database 8026 stores similarity data.

[0105] like Figure 8 As shown, the similarity analysis system 804 includes: a proxy server 8041, a load balancing server 8043, an analysis server 8045, a data structure server 8047, and a distributed database 8049.

[0106] The proxy server 8041 is used to monitor the user's specified operation, generate a first request, and the specified operation includes a page refresh operation; and filter the first request to remove invalid requests;

[0107] The load balancing server 8043 is used to parse the first request to obtain user information and access information; the load balancing server 6043 includes multiple logical collection units 60431 for processing and calculating the first request.

[0108] The analysis server 8045 is used to determine the time interval between the current specified operation and the last specified operation of the user based on the user information and the access information; and when the time interval is less than a predetermined threshold, generate page exception information.

[0109] The data structure server 8047 is used to store the user information and access information;

[0110] The distributed database 8049 is used to store page exception information.

[0111] For example, it may also include: a message queue server 8040, which is an intermediate server for storing filtered valid requests.

[0112] In the similarity analysis system 802, the crawler server 8022 (Crawler Server) is responsible for periodically crawling screenshots of the monitored pages, calculating and generating similarity values. The network server 8024 (Web Server) is responsible for the visualization of similarity statistics, the management of similarity data, the management of crawling strategies, and provides a similarity data interface for the analysis server 8045 (Analyze worker) to call in order to generate abnormal monitoring data.

[0113] The cycle of the analysis server 8045 (Analyze worker) in the user analysis system 804 can be, for example, 1 minute, so when a page anomaly occurs, the problem page can be detected in time by the user's refresh behavior. Of course, if a page anomaly occurs, but no user visits the page, it will naturally not trigger user behavior analysis. In this case, the detection time for the page anomaly will be extended to the cycle time of the similarity statistics. If the similarity statistics frequency is one day, the page anomaly will not be discovered until the next similarity statistics. In fact, this situation rarely occurs, because pages with such a low user access frequency are not worth monitoring.

[0114] The page anomaly monitoring method of the present invention converts the deterministic problem of page anomaly into a probabilistic problem, quantifies the probability of anomaly through similarity statistics and user behavior analysis, and provides data for the final page anomaly judgment.

[0115] The page anomaly monitoring method of the present invention solves a deterministic problem through probability: that is, it actively detects page anomalies through similarity statistics and user behavior analysis, and combines the data from the two aspects to provide a basis for the final judgment of page anomalies. It greatly shortens the time from the occurrence of page anomalies to their discovery.

[0116] Those skilled in the art will appreciate that all or part of the steps for implementing the above embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, the above functions defined by the above method provided by the present disclosure are performed. The program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0117] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0118] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0119] Fig. 9 1 is a block diagram of a page exception monitoring system according to an exemplary embodiment. The page exception monitoring system 90 includes: a screenshot server 902 , a crawler server 904 , and a network server 906 .

[0120] The screenshot server 902 is used to take a screenshot of the current web page to generate a current image.

[0121] The crawler server 904 is used to periodically obtain the current image, and compare the current image with the reference image to obtain a first similarity; and compare the current image with the screenshot of the page at the previous moment to obtain a second similarity.

[0122] The network server 906 is used to determine whether there is any abnormality in the current web page according to the first similarity and the second similarity.

[0123] According to the page anomaly monitoring system disclosed in the present invention, by comparing the similarity data of page screenshots taken at different times and then judging whether the current page has an abnormality, it is possible to automatically detect the abnormality of the web page so as to repair the web page defects in time.

[0124] Fig.10 1 is a block diagram of a device for monitoring page anomalies according to another exemplary embodiment. The device for monitoring page anomalies 1000 comprises: a screenshot module 1002 , a first comparison module 1004 , a second comparison module 1006 , and a judgment module 1008 .

[0125] The screenshot module 1002 is used to take a screenshot of the current web page to generate a current image;

[0126] The first comparison module 1004 is used to compare the similarity between the current image and the reference image to obtain a first similarity;

[0127] The second comparison module 1006 is used to compare the current image with the screenshot of the page at the previous moment, and obtain a second similarity;

[0128] The determination module 1008 is used to determine whether there is an abnormality in the current webpage according to the first similarity and the second similarity.

[0129] According to the page anomaly monitoring device disclosed in the present invention, by comparing the similarity data of page screenshots at different times and then judging whether the current page has an abnormality, it is possible to automatically detect the abnormality of the web page so as to repair the web page defects in time.

[0130] Fig.11 It is a block diagram of an electronic device according to an exemplary embodiment.

[0131] Refer to the following Fig.11 The electronic device 200 according to this embodiment of the present disclosure is described. Fig.11 The electronic device 200 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0132] like Fig.11 As shown, the electronic device 200 is in the form of a general computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one storage unit 220, a bus 230 connecting different system components (including the storage unit 220 and the processing unit 210), a display unit 240, etc.

[0133] The storage unit stores a program code, which can be executed by the processing unit 210, so that the processing unit 210 executes the steps according to various exemplary embodiments of the present disclosure described in the electronic prescription circulation processing method section of this specification. For example, the processing unit 210 can execute the following steps: Figure 2 , Figure 7 Follow the steps shown in .

[0134] The storage unit 220 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 2201 and / or a cache memory unit 2202 , and may further include a read-only memory unit (ROM) 2203 .

[0135] The storage unit 220 may also include a program / utility 2204 having a set (at least one) of program modules 2205, such program modules 2205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0136] Bus 230 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0137] The electronic device 200 may also communicate with one or more external devices 300 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 200, and / or communicate with any device that enables the electronic device 200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 250. Furthermore, the electronic device 200 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 260. The network adapter 260 may communicate with other modules of the electronic device 200 via the bus 230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0138] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the implementation of the present disclosure.

[0139] Fig.12 A schematic diagram of a computer-readable storage medium in an exemplary embodiment of the present disclosure is schematically shown.

[0140] refer to Fig.12As shown, a program product 400 for implementing the above method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0141] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0142] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0143] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0144] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by a device, the computer-readable medium realizes the following functions: taking a screenshot of the current web page to generate a current image; performing a similarity comparison between the current image and a reference image to obtain a first similarity; performing a similarity comparison between the current image and a screenshot of the page at a previous moment to obtain a second similarity; and determining whether there is an abnormality in the current web page through the first similarity and the second similarity.

[0145] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0146] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0147] The exemplary embodiments of the present disclosure are specifically shown and described above. It should be understood that the present disclosure is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present disclosure is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

[0148] In addition, the structures, proportions, sizes, etc. shown in the drawings of the specification are only used to match the contents disclosed in the specification for the understanding and reading of those skilled in the art, and are not used to limit the limiting conditions for the implementation of the present disclosure, so they have no substantial technical significance. Any modification of the structure, change of the proportional relationship or adjustment of the size, without affecting the technical effects and purposes that can be achieved by the present disclosure, should still fall within the scope of the technical contents disclosed by the present disclosure. At the same time, the terms such as "on", "first", "second" and "one" quoted in this specification are only for the convenience of description, and are not used to limit the scope of the implementation of the present disclosure. The change or adjustment of their relative relationship should also be regarded as the scope of the implementation of the present disclosure without substantially changing the technical contents.

Claims

1. A page abnormality monitoring method, characterized in that: include: Take a screenshot of the current web page to generate the current image; Comparing the current image with the reference image for similarity to obtain a first similarity; Comparing the current image with the screenshot of the page at the previous moment for similarity, to obtain a second similarity; as well as Generating a first similarity curve and a second similarity curve through the first similarity and the second similarity; Determine whether there is an abnormality on the current webpage through the first similarity curve and the second similarity curve; The first similarity curve is determined according to a plurality of first similarities, and the second similarity curve is determined according to a plurality of second similarities.

2. The method according to claim 1, characterized in that Take a screenshot of the current web page and generate the current image including: Taking a screenshot of the current webpage to generate a first image; and The first image is grayed to generate the current image.

3. The method according to claim 1, characterized in that Taking a screenshot of the current web page, generating the current image also includes: The current image is segmented in units of pixels to obtain a predetermined number of segmented image units.

4. The method according to claim 3, characterized in that Comparing the current image with the reference image for similarity, and obtaining a first similarity includes: The segmented image units in the current image are respectively compared with the segmented image units in the reference image to obtain a first similarity.

5. The method according to claim 4, characterized in that Comparing each segmented image unit with the segmented image unit in the reference image for similarity, and obtaining a first similarity includes: The number of similar units between the segmented image unit in the current image and the segmented image unit in the reference image is recorded as M; The number of dissimilar units between the segmented image units in the current image and the segmented image units in the reference image is recorded as N; and The first similarity between the current image and the reference image is determined by a similarity formula.

6. The method according to claim 5, characterized in that The similarity formula includes: S = MN * 100%; Among them, S is the similarity, M is the number of similar units, and N is the number of dissimilar units.

7. The method according to claim 1, characterized in that Determining whether the current webpage has an abnormality by using the first similarity curve and the second similarity curve includes: When the first similarity curve and the second similarity curve satisfy a first feature, determining that the current webpage is in a normal state; and When the first similarity curve and the second similarity curve satisfy the second characteristic, it is determined that the current webpage is in a normal state.

8. The method according to claim 1, characterized in that Determining whether the current webpage has an abnormality by using the first similarity curve and the second similarity curve includes: When the first similarity curve and the second similarity curve meet the third feature, calculating and obtaining a similarity variance; When the similarity variance is less than a predetermined threshold, determining that the current webpage is in a normal state; and When the similarity variance is greater than a predetermined threshold, it is determined that an abnormality exists in the current web page.

9. The method according to claim 1, characterized in that Also includes: Analyze user behavior to generate page exception information; According to the page exception information, the current web page is screenshoted to generate the current image.

10. A page abnormality monitoring system, characterized in that: include: The screenshot server is used to take a screenshot of the current web page and generate the current image; The crawler server is used to periodically obtain the current image, and compare the current image with a reference image to obtain a first similarity; Performing a similarity test between the current image and the screenshot of the page at the previous moment, to obtain a second similarity; as well as The network server is used to determine whether there is an abnormality in the current web page through the first similarity and the second similarity; The network server is also used to generate a first similarity curve and a second similarity curve through the first similarity and the second similarity; determine whether there is an abnormality in the current web page through the first similarity curve and the second similarity curve; wherein the first similarity curve is determined according to multiple first similarities, and the second similarity curve is determined according to multiple second similarities.

11. A page abnormality monitoring device, characterized in that: include: The screenshot module is used to take a screenshot of the current web page and generate the current image; A first comparison module, used for comparing the similarity between the current image and the reference image to obtain a first similarity; A second comparison module is used to compare the current image with the screenshot of the page at the previous moment to obtain a second similarity; as well as A judgment module, used for determining whether there is an abnormality in the current web page through the first similarity and the second similarity; The judgment module is also used to generate a first similarity curve and a second similarity curve through the first similarity and the second similarity; determine whether there is an abnormality in the current web page through the first similarity curve and the second similarity curve; wherein the first similarity curve is determined according to multiple first similarities, and the second similarity curve is determined according to multiple second similarities.

12. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

13. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Citation Information

Patent Citations

  • Page code testing method and device

    CN103678105A

  • Method and system for webpage state monitoring based on web page

    CN107832428A

  • Color image ray transform technique for detecting phishing web pages

    US20180115584A1