A website performance optimization supervision system and method based on multi-source data fusion

Abstract

The invention discloses a website performance optimization supervision system and method based on multi-source data fusion, which relates to the field of Internet technology. The system obtains access feedback data of all web pages in a website to be inspected, analyzes the running status of the web pages, and determines the abnormal situation of the web pages; generates a directed weighted graph of the website according to the jump relationship between the web pages; analyzes the abnormal sources of the web pages; obtains the quality score of the web pages based on the abnormal sources of the web pages; obtains the quality score of the website based on the quality score of the web pages and the directed weighted graph of the website; monitors the access feedback data of the web pages, finds the web pages related to the abnormal sources, and analyzes them according to the abnormal sources, so that the quality evaluation of the web pages and the website is more reasonable; timely finds and processes the web pages related to the abnormal sources, so that the website obtains better index scores when facing inspection, which is helpful to improve the image of the website; inspects and processes the abnormal sources, and improves the efficiency and speed of website detection.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a website performance optimization and supervision system and method based on multi-source data fusion. Background Art

[0002] The significance of website performance optimization is to comprehensively and objectively evaluate the performance and user experience of the website, help website operators discover and solve problems existing on the website, and improve the quality of the website and user experience. By monitoring the website, you can understand the pros and cons of the website, discover potential problems and directions for improvement, and then formulate corresponding improvement measures to improve the website's operational efficiency and effectiveness; currently, most websites are tested manually, which consumes a lot of manpower and time, and cannot be tested in real time, and the immediacy is poor; the index scoring results obtained after the test can only reflect the quality of the website within a period of time, and the real-time performance is poor; therefore, how to improve the efficiency of website monitoring and optimize website performance in a timely manner according to the monitoring results has become an urgent problem to be solved. Summary of the invention

[0003] The object of the present invention is to provide a pipeline steel fracture safety analysis system and method based on multi-source data to solve the problems raised in the prior art.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a website performance optimization and supervision system based on multi-source data fusion, comprising a data receiving module, a website evaluation module, a data storage module, a website monitoring module and an output module; the output end of the data receiving module is connected to the input end of the data storage module, and is used to obtain access feedback data of all web pages in the website, and the access feedback data includes access volume and access time; the output end of the website evaluation module is connected to the input end of the output module, and the quality score of the website is obtained according to the weight of the web page and the quality score of the web page; the output end of the data storage module is connected to the input end of the website evaluation module and the website monitoring module, and is used to store the access log data of the website and the index score of the web page; the website detection module is used to monitor the access feedback data of the web page and determine whether an abnormal situation occurs on the web page; if so, the abnormal source of the web page is analyzed, otherwise the monitoring of the access feedback data of the web page is maintained; the output module is used to output the abnormal source information of the web page and the quality score of the website.

[0005] The website monitoring module also includes a probability density estimation unit, an access feedback data analysis unit, an unsupervised classification unit and an abnormal source analysis unit; the probability density estimation unit is used to generate a probability density function of the access feedback data of the web page; the access feedback data analysis unit is used to perform instant analysis on the access feedback data of the web page item by item; the unsupervised classification unit is used to analyze whether the access feedback data of the web page match each other, if they match, there is no abnormality in the web page, otherwise there is an abnormality in the web page; the abnormal source analysis unit is used to analyze the abnormal source of the web page with abnormality. The website evaluation module also includes a directed graph unit, a weight analysis unit, a web page evaluation unit and a website evaluation unit; the directed graph unit obtains the directed graph of the website according to the jump relationship of the web pages in the website; the weight analysis unit obtains the weight of the edge according to the access path data of the user in the website, and obtains the weight of the web page according to the weight of the edge; the web page evaluation unit is used to determine the quality score of the web page; the website evaluation unit obtains the quality score of the website according to the weight of the web page and the quality score of the web page.

[0006] The abnormal source analysis unit analyzes the abnormal source through the following steps:

[0007] Obtain the current visit volume y1 and visit time y2 data of the target webpage; according to the probability density function of the visit volume and the visit time, obtain the probability P1 that the visit volume is in the interval [(1-d1)y1, (1+d1)y1] and the probability P2 that the visit time is in the interval [(1-d2)y2, (1+d2)y2]; d1 and d2 are preset constants;

[0008] Under the condition that the target webpage has an abnormal situation, if P2 is less than the first threshold and P1 is not less than the first threshold, then the abnormal source is related to the target webpage itself;

[0009] If both P1 and P2 are less than the first threshold, or P2 is not less than the first threshold and P1 is less than the first threshold, then the weights of all edges pointing to the target web page are obtained from the directed weighted graph of the website, and the proportional relationship between the weights of all edges pointing to the target web page is calculated; at the same time, the user's access path to the target web page under the current visit volume y1 of the target web page is obtained, and the proportional relationship between different edges of the number of times the user enters the target web page on the access path is obtained; it is determined whether there is an edge e with an abnormally low proportion in the proportional relationship between different edges, if so, the edge e with the abnormally low proportion is eliminated, and the proportional relationship g1 between the remaining edges of the number of times the user enters the target web page on the access path is calculated, and at the same time, the proportional relationship g2 between the weights of the remaining edges pointing to the target web page is obtained, if the error between g1 and g2 does not exceed the set threshold, it is determined that g1 and g2 are similar, and it is determined that the abnormal source is related to the web page pointing to the target web page through the edge e; if there is no edge e with an abnormally low proportion in the proportional relationship between different edges, then the abnormal source is related to the target web page itself.

[0010] To achieve the above object, the present invention provides the following technical solution: a website performance optimization and supervision method based on multi-source data fusion, comprising the following steps:

[0011] S11, obtaining access feedback data of all web pages in the website to be checked, wherein the access feedback data includes the number of visits and the access time of the web pages;

[0012] S12, analyzing the operation status of the web page based on the page visit volume and access time data, and determining the abnormality of the web page;

[0013] S13, obtaining the jump relationship between all web pages in the website, generating a directed weighted graph of the website according to the jump relationship between the web pages; analyzing the abnormal source of the web pages according to the directed weighted graph of the website;

[0014] S14, obtaining a quality score of the web page based on the abnormal source of the web page; and obtaining a quality score of the website based on the quality score of the web page and the directed weighted graph of the website.

[0015] Specifically, in step S12, the analysis of the operation status of the web page based on the page visit volume and access time data to determine whether there is an abnormality on the web page also includes the following steps:

[0016] S21, obtaining historical access feedback data of a web page within a fixed time period T, where the time period T is set according to the situation of the website;

[0017] S22, generating a probability density function of the web page access feedback data according to the historical access feedback data of the web page: firstly, selecting a kernel function which is non-negative and symmetric and whose integral over the real number field R is 1;

[0018] S23, setting a constant greater than zero as the bandwidth h of the kernel function K; scaling the kernel function according to the bandwidth to obtain Kh, where Kh(u)=1 / h×K(u / h), and u is the input of the kernel function;

[0019] S24, obtain the contribution rate Kh(x-xi) of the data point xi in the historical access feedback data to the estimated point x; add the contribution rates of all data points in the historical access feedback data to the estimated point x to obtain the kernel density estimation value f(x) at the estimated point x; f(x)=1 / n∑Kh(x-xi), where n is the number of data points in the historical access feedback data; change the position of the estimated point x, re-execute step S24, and obtain the kernel density estimation of the access feedback data on the entire data set; the data set is the interval between the minimum and maximum values ​​of the historical access feedback data;

[0020] S25, verifying the effect of the kernel function using historical access feedback data, executing step S24 for different bandwidths h, and selecting the bandwidth h with the best verification effect.

[0021] Specifically, in step S12, the following steps are also included:

[0022] S31, obtain the current visit volume y1 and visit time y2 data of the target webpage; according to the probability density function of the visit volume and the visit time, obtain the probability P1 that the visit volume is in the interval [(1-d1)y1, (1+d1)y1] and the probability P2 that the visit time is in the interval [(1-d2)y2, (1+d2)y2]; d1 and d2 are preset constants; perform anomaly detection on the visit volume and the visit time respectively, if P1 and P2 are not less than the first threshold, enter step S32 to determine whether the visit volume and the visit time match; otherwise, there is an abnormality in the webpage and the process ends;

[0023] S32, perform feature concatenation on the access volume y1 and the access time y2 to obtain a feature vector [y1, y2], and obtain the feature vector of the historical data from the historical access feedback data of the web page; in the historical access feedback data of the web page, if there is an error in the link that jumps to the web page, then the historical access feedback data of the web page within the time period containing the error is marked; perform unsupervised classification on the feature vector [y1, y2] and the feature vector of the historical data, and according to the unsupervised classification result, if the feature vector [y1, y2] is an outlier, then it is determined that there is an abnormality in the target web page; if the feature vector [y1, y2] is not an outlier, then obtain the feature vector information of the historical data in the classification cluster to which the feature vector [y1, y2] belongs, and calculate the proportion of the feature vector information of the historical data with annotations in the classification cluster to which it belongs. If the proportion is not less than the second threshold, then it is determined that there is an abnormality in the target web page, otherwise, there is no abnormality in the target web page.

[0024] Monitor the web page through the number of visits and the access time. If there is an abnormal decrease in the number of visits and the access time, it means that there may be an abnormal problem with the web page, such as the link error to enter the web page makes the user unable to access the web page, resulting in a decrease in the number of visits to the web page; the web page has not been updated for a long time, resulting in a decrease in the access time; if there is no abnormal decrease in the number of visits and the access time, it is necessary to analyze whether the number of visits and the access time match. The probability density function is for the case of continuous distribution. For this reason, two smaller values ​​d1 and d2 are set to obtain the probabilities P1 and P2.

[0025] Specifically, in step S13, the step of obtaining the jump relationship between all web pages in the website and generating a directed weighted graph of the website according to the jump relationship between the web pages further includes the following steps:

[0026] Get all web pages in the website and use the web pages as nodes; if there is a link on web page a that jumps to web page b, generate an edge from web page a to web page b; connect the web page nodes that have a jump relationship in the website through the edge to obtain a directed graph of the website;

[0027] Get the number of visits T1 and the visit time T2 of web page a within a fixed time period T, and get the total weight Wa of the edge pointing to web page a based on the number of visits and the visit time, Wa=U1×T1 / ST1+U2×T2 / ST2, where U1 and U2 are the weights of the number of visits and the visit time, and ST1 and ST2 are the number of visits and the visit time of the website; get the user's access path within the website from the historical access log of the website, determine all the access paths containing web page a from the access paths within the website, and record the access path containing web page a as the target path; get the jth path in the target path The number of times pathj and the total number of times the target path appears path, the partial weight Uc→a from web page c to web page a is obtained, Uc→a=pathj / path, where web page c is the previous web page of web page a on the j-th path in the target path; for all paths in the target path, the partial weights of web page c pointing to web page a are added to obtain the weight of the edge from web page c to web page a; the position of web page c is changed to obtain the weights of all edges pointing to web page a; the position of web page a is changed, and the above steps are repeated to obtain the weights of all edges in the directed graph of the website, thereby generating a directed weighted graph of the website.

[0028] Specifically, in step S13, the analyzing the abnormal source of the web page according to the directed weighted graph of the website further includes the following steps:

[0029] If P2 is less than the first threshold and P1 is not less than the first threshold, then the anomaly source is related to the target web page itself;

[0030] If both P1 and P2 are less than the first threshold, or P2 is not less than the first threshold and P1 is less than the first threshold, then the weights of all edges pointing to the target web page are obtained from the directed weighted graph of the website, and the proportional relationship between the weights of all edges pointing to the target web page is calculated; at the same time, the user's access path to the target web page under the current visit volume y1 of the target web page is obtained, and the proportional relationship between different edges of the number of times the user enters the target web page on the access path is obtained; it is determined whether there is an edge e with an abnormally low proportion in the proportional relationship between different edges, if so, the edge e with the abnormally low proportion is eliminated, and the proportional relationship g1 between the remaining edges of the number of times the user enters the target web page on the access path is calculated, and at the same time, the proportional relationship g2 between the weights of the remaining edges pointing to the target web page is obtained, if the error between g1 and g2 does not exceed the set threshold, it is determined that g1 and g2 are similar, and it is determined that the abnormal source is related to the web page pointing to the target web page through the edge e; if there is no edge e with an abnormally low proportion in the proportional relationship between different edges, then the abnormal source is related to the target web page itself.

[0031] If the number of visits does not decrease abnormally, but the time users spend on a webpage decreases abnormally, it means that users can access the webpage normally, but stay on the webpage for a short time, which indicates that there are problems with the webpage itself, such as obvious typos, unsightly interface, or long-term non-update.

[0032] In the case of abnormal decrease in the number of visits, first obtain the proportional relationship between different edges of the number of times users enter the target web page on the access path; determine whether there is an edge e with an abnormally low ratio in the proportional relationship between different edges. If the proportional relationship of other edges is similar to that in normal situations without considering edge e, it means that the reason for the decrease in the number of visits is related to edge e; if there is no edge e with an abnormally low ratio in the proportional relationship between different edges, it means that the web page is less attractive to users and the abnormal source is related to the web page itself. By analyzing the user feedback data of the target web page, problems in the website can be discovered in time. For example, when the link of the target web page changes, the jump entrance of other web pages is not updated in time, making it impossible for users to enter the target web page. By detecting the number of visits and access time of the target web page, the abnormal source can be discovered in time and solved; at the same time, in addition to its own reasons, the changes in the user feedback data of the target web page are also affected by other web pages. It is not appropriate to attribute the deterioration of user feedback data entirely to the target web page. Timely processing of abnormal source web pages can obtain better indicator scores when the website faces inspection, which helps to improve the image of the website.

[0033] Specifically, in step S14, the following steps are also included:

[0034] Obtain the index scores of all web pages, and the index score results are determined by checking the website content; if the web page is related to an abnormal source, reduce the index score of the web page to obtain the quality score of the web page; in the directed weighted graph of the website, obtain the weights of the edges starting from web page a and pointing to other web pages, and add all weights to obtain the weight of web page a; perform weighted summation based on the weight of the web page and the quality score of the web page to obtain the quality score of the website.

[0035] Compared with the prior art, the beneficial effects of the present invention are: monitoring the access feedback data of web pages, discovering web pages related to abnormal sources, and analyzing them according to the abnormal sources, so that the quality evaluation of web pages and websites is more reasonable; timely discovering and processing web pages related to abnormal sources, so that the website can obtain better indicator scores when facing inspection, which is helpful to improve the image of the website; inspecting and processing abnormal sources to improve the efficiency and speed of website detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a structural schematic diagram of a website performance optimization and supervision system based on multi-source data fusion according to the present invention. Detailed implementation mode

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] Embodiment: As Figure 1 shown, the present invention provides a technical solution, a website performance optimization supervision system based on multi-source data fusion, including a data receiving module, a website evaluation module, a data storage module, a website monitoring module, and an output module; the output end of the data receiving module is connected to the input end of the data storage module, and is used to obtain the access feedback data of all web pages in the website, and the access feedback data includes the number of visits and the access time; the output end of the website evaluation module is connected to the input end of the output module, and obtains the quality score of the website according to the weight of the web page and the quality score of the web page; the output end of the data storage module is connected to the input ends of the website evaluation module and the website monitoring module, and is used to store the access log data of the website and the index scores of the web pages; the website detection module is used to monitor the access feedback data of the web pages to determine whether an abnormal situation occurs on the web pages; if so, analyze the abnormal source of the web pages, otherwise keep monitoring the access feedback data of the web pages; the output module is used to output the abnormal source information of the web pages and the quality score of the website.

[0039] The website monitoring module further includes a probability density estimation unit, an access feedback data analysis unit, an unsupervised classification unit, and an abnormal source analysis unit; the probability density estimation unit is used to generate the probability density function of the access feedback data of the web pages; the access feedback data analysis unit is used to immediately analyze each item of the access feedback data of the web pages; the unsupervised classification unit is used to analyze whether the access feedback data of the web pages match each other. If they match, there is no abnormal situation on the web pages, otherwise there is an abnormal situation on the web pages; the abnormal source analysis unit is used to analyze the abnormal source of the web pages with abnormal situations. The website evaluation module further includes a directed graph unit, a weight analysis unit, a web page evaluation unit, and a website evaluation unit; the directed graph unit obtains the directed graph of the website according to the jump relationship of the web pages in the website; the weight analysis unit obtains the weight of the edge according to the access path data of the user in the website, and obtains the weight of the web page according to the weight of the edge; the web page evaluation unit is used to determine the quality score of the web page; the website evaluation unit obtains the quality score of the website according to the weight of the web page and the quality score of the web page.

[0040] The abnormal source analysis unit analyzes the abnormal source through the following steps:

[0041] Obtain the current visit volume y1 and visit time y2 data of the target webpage; according to the probability density function of the visit volume and the visit time, obtain the probability P1 that the visit volume is in the interval [(1-d1)y1, (1+d1)y1] and the probability P2 that the visit time is in the interval [(1-d2)y2, (1+d2)y2]; d1 and d2 are preset constants;

[0042] Under the condition that the target webpage has an abnormal situation, if P2 is less than the first threshold and P1 is not less than the first threshold, then the abnormal source is related to the target webpage itself;

[0043] If both P1 and P2 are less than the first threshold, or P2 is not less than the first threshold and P1 is less than the first threshold, then the weights of all edges pointing to the target web page are obtained from the directed weighted graph of the website, and the proportional relationship between the weights of all edges pointing to the target web page is calculated; at the same time, the user's access path to the target web page under the current visit volume y1 of the target web page is obtained, and the proportional relationship between different edges of the number of times the user enters the target web page on the access path is obtained; it is determined whether there is an edge e with an abnormally low proportion in the proportional relationship between different edges, if so, the edge e with the abnormally low proportion is eliminated, and the proportional relationship g1 between the remaining edges of the number of times the user enters the target web page on the access path is calculated, and at the same time, the proportional relationship g2 between the weights of the remaining edges pointing to the target web page is obtained, if the error between g1 and g2 does not exceed the set threshold, it is determined that g1 and g2 are similar, and it is determined that the abnormal source is related to the web page pointing to the target web page through the edge e; if there is no edge e with an abnormally low proportion in the proportional relationship between different edges, then the abnormal source is related to the target web page itself.

[0044] Embodiment: The present invention provides a technical solution, a website performance optimization and supervision method based on multi-source data fusion, comprising the following steps:

[0045] S11, obtaining access feedback data of all web pages in the website to be checked, wherein the access feedback data includes the number of visits and the access time of the web pages;

[0046] S12, based on the page visit volume and access time data, analyze the operation status of the page to determine the abnormality of the page:

[0047] Obtain the current visit volume y1 and visit time y2 data of the target webpage; according to the probability density function of the visit volume and the visit time, obtain the probability P1 that the visit volume is in the interval [(1-d1)y1, (1+d1)y1] and the probability P2 that the visit time is in the interval [(1-d2)y2, (1+d2)y2]; d1 and d2 are preset constants; perform anomaly detection on the visit volume and the visit time respectively, if P1 and P2 are not less than the first threshold, enter step S32 to determine whether the visit volume and the visit time match; otherwise, there is an abnormality in the webpage and end; the first threshold is related to d1 and d2, the higher the d1 and d2, the higher the first threshold; the first threshold can be set to 5%, when the probability of the current visit volume y1 and the visit time y2 data appearing is less than 5%, it is determined that an abnormality has occurred;

[0048] Perform feature concatenation on the access volume y1 and the access time y2 to obtain a feature vector [y1, y2], and obtain a feature vector of the historical data from the historical access feedback data of the web page; in the historical access feedback data of the web page, if there is an error in the link that jumps to the web page, then the historical access feedback data of the web page in the time period containing the error is marked; perform unsupervised classification on the feature vector [y1, y2] and the feature vector of the historical data, and according to the unsupervised classification result, if the feature vector [y1, y2] is an outlier, then it is determined that there is an abnormality in the target web page; if the feature vector [y1, y2] is not an outlier, then obtain the feature vector information of the historical data in the classification cluster to which the feature vector [y1, y2] belongs, calculate the proportion of the feature vector information of the historical data with annotations in the classification cluster to which it belongs, and if the proportion is not less than the second threshold, then it is determined that there is an abnormality in the target web page, otherwise, there is no abnormality in the target web page.

[0049] The probability density function is determined by steps S21 to S25:

[0050] S21, obtaining historical access feedback data of a web page within a fixed time period T, where the time period T is set according to the situation of the website;

[0051] S22, generating a probability density function of the web page access feedback data according to the historical access feedback data of the web page: firstly, selecting a kernel function which is non-negative and symmetric and whose integral over the real number field R is 1;

[0052] S23, setting a constant greater than zero as the bandwidth h of the kernel function K; scaling the kernel function according to the bandwidth to obtain Kh, where Kh(u)=1 / h×K(u / h), and u is the input of the kernel function;

[0053] S24, obtain the contribution rate Kh(x-xi) of the data point xi in the historical access feedback data to the estimated point x; add the contribution rates of all data points in the historical access feedback data to the estimated point x to obtain the kernel density estimation value f(x) at the estimated point x; f(x)=1 / n∑Kh(x-xi), where n is the number of data points in the historical access feedback data; change the position of the estimated point x, re-execute step S24, and obtain the kernel density estimation of the access feedback data on the entire data set; the data set is the interval between the minimum and maximum values ​​of the historical access feedback data;

[0054] S25, verifying the effect of the kernel function using historical access feedback data, executing step S24 for different bandwidths h, and selecting the bandwidth h with the best verification effect.

[0055] S13, obtaining the jump relationship between all web pages in the website, and generating a directed weighted graph of the website according to the jump relationship between the web pages:

[0056] Get all web pages in the website and use the web pages as nodes; if there is a link on web page a that jumps to web page b, generate an edge from web page a to web page b; connect the web page nodes that have a jump relationship in the website through the edge to obtain a directed graph of the website;

[0057] Get the number of visits T1 and the visit time T2 of web page a within a fixed time period T, and get the total weight Wa of the edge pointing to web page a based on the number of visits and the visit time, Wa=U1×T1 / ST1+U2×T2 / ST2, where U1 and U2 are the weights of the number of visits and the visit time, and ST1 and ST2 are the number of visits and the visit time of the website; get the user's access path within the website from the historical access log of the website, determine all the access paths containing web page a from the access paths within the website, and record the access path containing web page a as the target path; get the jth path in the target path The number of times pathj and the total number of times the target path appears path, the partial weight Uc→a from web page c to web page a is obtained, Uc→a=pathj / path, where web page c is the previous web page of web page a on the j-th path in the target path; for all paths in the target path, the partial weights of web page c pointing to web page a are added to obtain the weight of the edge from web page c to web page a; the position of web page c is changed to obtain the weights of all edges pointing to web page a; the position of web page a is changed, and the above steps are repeated to obtain the weights of all edges in the directed graph of the website, thereby generating a directed weighted graph of the website.

[0058] Analyze the abnormal sources of web pages based on the directed weighted graph of the website:

[0059] If P2 is less than the first threshold and P1 is not less than the first threshold, then the anomaly source is related to the target web page itself;

[0060] If both P1 and P2 are less than the first threshold, or P2 is not less than the first threshold and P1 is less than the first threshold, then the weights of all edges pointing to the target webpage are obtained from the directed weighted graph of the website, and the proportional relationship between the weights of all edges pointing to the target webpage is calculated; at the same time, the user's access path to the target webpage under the current number of visits to the target webpage y1 is obtained, and the proportional relationship between different edges of the number of times the user enters the target webpage on the access path is obtained; it is determined whether there is an edge e with an abnormally low ratio in the proportional relationship between different edges, if so, the edge e with an abnormally low ratio is removed, and the proportional relationship g1 between the remaining edges of the number of times the user enters the target webpage on the access path is calculated, and at the same time, the proportional relationship g2 between the weights of the remaining edges pointing to the target webpage is obtained, if the error between g1 and g2 does not exceed the set threshold, it is determined that g1 and g2 are similar, and it is determined that the abnormal source is related to the webpage pointing to the target webpage through edge e; if there is no edge e with an abnormally low ratio in the proportional relationship between different edges, then the abnormal source is related to the target webpage itself. The error between g1 and g2 can be calculated as root mean square error, mean absolute percentage error, percentage error, etc.

[0061] S14, obtaining a quality score of the web page based on the abnormal source of the web page:

[0062] Obtaining index scores of all web pages, wherein the index score results are determined by checking the website content; if a web page is related to an abnormal source, the index score of the web page is reduced to obtain a quality score of the web page;

[0063] The index score is the content inspection result obtained when all web pages in the website are subject to inspection; usually, the index score of a web page is mainly affected by deduction items. When the information displayed on the web page does not meet the standards, the response is slow, there are typos, there are error messages, and it does not meet the current network security transformation standards, the index score of the web page will be deducted. For web pages related to abnormal sources, the deduction standards of the index score can be referred to and reduced on the basis of the index score. During the operation of the website, timely inspection and processing of web pages with quality scores less than the index score can improve the efficiency of web page detection and enable the website to obtain better index scores when facing inspections.

[0064] The quality score of the website is obtained based on the quality score of the web page and the directed weighted graph of the website:

[0065] In the directed weighted graph of the website, the weights of the edges starting from web page a and pointing to other web pages are obtained, and all weights are added up to get the weight of web page a; the quality score of the website is obtained by weighted summing the weight of the web page and the quality score of the web page. The indicator score results obtained by the website can only reflect the quality of the website at the time of inspection. The real-time quality of the website can be obtained by accessing the feedback data.

[0066] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A website performance optimization and supervision method based on multi-source data fusion, characterized in that: The following steps are involved: S11, obtaining access feedback data of all web pages in the website to be checked, wherein the access feedback data includes the number of visits and the access time of the web pages; S12, based on the page visit volume and access time data, analyzing the operation status of the web page to determine the abnormality of the web page, including the following steps S31 and S32: S31, obtain the current visit volume y1 and visit time y2 data of the target webpage; according to the probability density function of the visit volume and the visit time, obtain the probability P1 that the visit volume is in the interval [(1-d1)y1, (1+d1)y1] and the probability P2 that the visit time is in the interval [(1-d2)y2, (1+d2)y2]; d1 and d2 are preset constants; perform anomaly detection on the visit volume and the visit time respectively, if P1 and P2 are not less than the first threshold, enter step S32 to determine whether the visit volume and the visit time match; otherwise, there is an abnormality in the webpage and the process ends; S32, perform feature concatenation on the access volume y1 and the access time y2 to obtain a feature vector [y1, y2], and obtain a feature vector of the historical data from the historical access feedback data of the web page; in the historical access feedback data of the web page, if there is an error in the link that jumps to the web page, then the historical access feedback data of the web page in the time period containing the error is marked; perform unsupervised classification on the feature vector [y1, y2] and the feature vector of the historical data, and according to the unsupervised classification result, if the feature vector [y1, y2] is an outlier, then it is determined that there is an abnormality in the target web page; if the feature vector [y1, y2] is not an outlier, then obtain the feature vector information of the historical data in the classification cluster to which the feature vector [y1, y2] belongs as an outlier, calculate the proportion of the feature vector information of the historical data with annotations in the classification cluster to which it belongs, and if the proportion is not less than the second threshold, then it is determined that there is an abnormality in the target web page, otherwise there is no abnormality in the target web page; S13, obtaining the jump relationship between all web pages in the website, generating a directed weighted graph of the website according to the jump relationship between the web pages; analyzing the abnormal source of the web pages according to the directed weighted graph of the website; S14, obtaining a quality score of the web page based on the abnormal source of the web page; and obtaining a quality score of the website based on the quality score of the web page and the directed weighted graph of the website.

2. According to claim 1, a website performance optimization and supervision method based on multi-source data fusion is characterized in that: In step S12, the operation status of the web page is analyzed based on the page visit volume and access time data to determine the abnormality of the web page, and the following steps are also included: S21, obtaining historical access feedback data of a web page within a fixed time period T, where the time period T is set according to the situation of the website; S22, generating a probability density function of the web page access feedback data according to the historical access feedback data of the web page: firstly, selecting a kernel function which is non-negative and symmetric and whose integral over the real number field R is 1; S23, setting a constant greater than zero as the bandwidth h of the kernel function K; scaling the kernel function according to the bandwidth to obtain Kh, where Kh(u)=1 / h×K(u / h), and u is the input of the kernel function; S24, obtain the contribution rate Kh(x-xi) of the data point xi in the historical access feedback data to the estimated point x; add the contribution rates of all the data points in the historical access feedback data to the estimated point x to obtain the kernel density estimation value f(x) at the estimated point x; f(x) = 1 / n∑Kh(x-xi), where n is the number of data points in the historical access feedback data; change the position of the estimated point x, re-execute step S24, and obtain the kernel density estimation of the access feedback data on the entire data set; the data set is the interval between the minimum and maximum values ​​of the historical access feedback data; S25, verifying the effect of the kernel function using historical access feedback data, executing step S24 for different bandwidths h, and selecting the bandwidth h with the best verification effect.

3. A website performance optimization and supervision method based on multi-source data fusion according to claim 2, characterized in that: In step S13, the step of obtaining the jump relationship between all web pages in the website and generating a directed weighted graph of the website according to the jump relationship between the web pages further includes the following steps: Get all web pages in the website and use the web pages as nodes; if there is a link on web page a that jumps to web page b, generate an edge from web page a to web page b; connect the web page nodes that have a jump relationship in the website through the edge to obtain a directed graph of the website; Obtain the number of visits T1 and the visit time T2 of web page a within a fixed time period T, and obtain the total weight Wa of the edge pointing to web page a based on the number of visits and the visit time, Wa = U1 × T1 / ST1 + U2 × T2 / ST2, where U1 and U2 are the weights of the number of visits and the visit time, and ST1 and ST2 are the number of visits and the visit time of the website; obtain the user's access path within the website from the historical access log of the website, determine all the access paths containing web page a from the access paths within the website, and record the access path containing web page a as the target path; obtain the jth path in the target path. The number of times pathj and the total number of times the target path appears path, the partial weight Uc→a from web page c to web page a is obtained, Uc→a=pathj / path, where web page c is the previous web page of web page a on the j-th path in the target path; for all paths in the target path, the partial weights of web page c pointing to web page a are added to obtain the weight of the edge from web page c to web page a; the position of web page c is changed to obtain the weights of all edges pointing to web page a; the position of web page a is changed, and the above steps are repeated to obtain the weights of all edges in the directed graph of the website, and a directed weighted graph of the website is generated.

4. A website performance optimization and supervision method based on multi-source data fusion according to claim 3, characterized in that: In step S13, the analysis of the abnormal source of the web page according to the directed weighted graph of the website further includes the following steps: If P2 is less than the first threshold and P1 is not less than the first threshold, then the anomaly source is related to the target web page itself; If both P1 and P2 are less than the first threshold, or P2 is not less than the first threshold and P1 is less than the first threshold, then the weights of all edges pointing to the target web page are obtained from the directed weighted graph of the website, and the proportional relationship between the weights of all edges pointing to the target web page is calculated; at the same time, the user's access path to the target web page under the current visit volume y1 of the target web page is obtained, and the proportional relationship between different edges of the number of times the user enters the target web page on the access path is obtained; it is determined whether there is an edge e with an abnormally low proportion in the proportional relationship between different edges, if so, the edge e with the abnormally low proportion is eliminated, and the proportional relationship g1 between the remaining edges of the number of times the user enters the target web page on the access path is calculated, and at the same time, the proportional relationship g2 between the weights of the remaining edges pointing to the target web page is obtained, if the error between g1 and g2 does not exceed the set threshold, it is determined that g1 and g2 are similar, and it is determined that the abnormal source is related to the web page pointing to the target web page through the edge e; if there is no edge e with an abnormally low proportion in the proportional relationship between different edges, then the abnormal source is related to the target web page itself.

5. A website performance optimization and supervision method based on multi-source data fusion according to claim 4, characterized in that: In step S14, the following steps are also included: Obtain the index scores of all web pages, and the index score results are determined by checking the website content; if the web page is related to an abnormal source, reduce the index score of the web page to obtain the quality score of the web page; in the directed weighted graph of the website, obtain the weights of the edges starting from web page a and pointing to other web pages, and add all weights to obtain the weight of web page a; perform weighted summation based on the weight of the web page and the quality score of the web page to obtain the quality score of the website.

6. A website performance optimization and supervision system based on multi-source data fusion, using a website performance optimization and supervision method based on multi-source data fusion as claimed in any one of claims 1 to 5, characterized in that: It includes a data receiving module, a website evaluation module, a data storage module, a website monitoring module and an output module; the output end of the data receiving module is connected to the input end of the data storage module, and is used to obtain the access feedback data of all web pages in the website, and the access feedback data includes the access volume and the access time; the output end of the website evaluation module is connected to the input end of the output module, and the quality score of the website is obtained according to the weight of the web page and the quality score of the web page; the output end of the data storage module is connected to the input end of the website evaluation module and the website monitoring module, and is used to store the access log data of the website and the index score of the web page; the website monitoring module is used to monitor the access feedback data of the web page and determine whether an abnormal situation occurs in the web page; if so, the abnormal source of the web page is analyzed, otherwise the monitoring of the access feedback data of the web page is maintained; the output module is used to output the abnormal source information of the web page and the quality score of the website.

7. A website performance optimization and supervision system based on multi-source data fusion according to claim 6, characterized in that: The website monitoring module also includes a probability density estimation unit, an access feedback data analysis unit, an unsupervised classification unit and an abnormal source analysis unit; the probability density estimation unit is used to generate a probability density function of the access feedback data of the web page; the access feedback data analysis unit is used to perform instant analysis on the access feedback data of the web page item by item; the unsupervised classification unit is used to analyze whether the access feedback data of the web page match each other, if they match, there is no abnormality in the web page, otherwise there is an abnormality in the web page; the abnormal source analysis unit is used to analyze the abnormal source of the web page with the abnormal situation.

8. A website performance optimization and supervision system based on multi-source data fusion according to claim 6, characterized in that: The website evaluation module also includes a directed graph unit, a weight analysis unit, a web page evaluation unit and a website evaluation unit; the directed graph unit obtains a directed graph of the website according to the jump relationship of the web pages in the website; the weight analysis unit obtains the weight of the edge according to the user's access path data in the website, and obtains the weight of the web page according to the weight of the edge; the web page evaluation unit is used to determine the quality score of the web page; the website evaluation unit obtains the quality score of the website according to the weight of the web page and the quality score of the web page.

9. A website performance optimization and supervision system based on multi-source data fusion according to claim 7, characterized in that: The abnormal source analysis unit analyzes the abnormal source through the following steps: Obtain the current visit volume y1 and visit time y2 data of the target webpage; according to the probability density function of the visit volume and the visit time, obtain the probability P1 that the visit volume is in the interval [(1-d1)y1, (1+d1)y1] and the probability P2 that the visit time is in the interval [(1-d2)y2, (1+d2)y2]; d1 and d2 are preset constants; Under the condition that the target webpage has an abnormal situation, if P2 is less than the first threshold and P1 is not less than the first threshold, then the abnormal source is related to the target webpage itself; If both P1 and P2 are less than the first threshold, or P2 is not less than the first threshold and P1 is less than the first threshold, then the weights of all edges pointing to the target web page are obtained from the directed weighted graph of the website, and the proportional relationship between the weights of all edges pointing to the target web page is calculated; at the same time, the user's access path to the target web page under the current visit volume y1 of the target web page is obtained, and the proportional relationship between different edges of the number of times the user enters the target web page on the access path is obtained; it is determined whether there is an edge e with an abnormally low proportion in the proportional relationship between different edges, if so, the edge e with the abnormally low proportion is eliminated, and the proportional relationship g1 between the remaining edges of the number of times the user enters the target web page on the access path is calculated, and at the same time, the proportional relationship g2 between the weights of the remaining edges pointing to the target web page is obtained, if the error between g1 and g2 does not exceed the set threshold, it is determined that g1 and g2 are similar, and it is determined that the abnormal source is related to the web page pointing to the target web page through the edge e; if there is no edge e with an abnormally low proportion in the proportional relationship between different edges, then the abnormal source is related to the target web page itself.

Images