Webpage identification method and device based on deep data packet detection library and storage medium

Abstract

The invention provides a webpage recognition method and device based on a deep data packet detection library and a storage medium, and relates to the technical field of computer fingerprints. The method comprises the following steps: accessing a target link for multiple times based on at least one known device, generating a plurality of target data object lists, and determining a target fingerprint corresponding to the target link based on the plurality of target data object lists; obtaining a plurality of pieces of depth data packet detection data in a preset range in a depth data packet detection library, and determining a plurality of candidate fingerprints based on the depth data packet detection data; and determining a target webpage list based on the first similarity between the target fingerprint and the plurality of candidate fingerprints. According to the method, the technical effect of improving the webpage recognition accuracy is achieved.

Description

Technical Field

[0001] The present application relates to computer fingerprint technology, and more particularly to a web page identification method, device and storage medium based on a deep data packet inspection library. Background Art

[0002] With the development of network communication technology, websites have become an important carrier of network information interaction activities, and web pages, as the basic elements of websites, play a very important role in the process of network information transmission. In order to protect the privacy and security of network users, most websites use the https protocol to encrypt data, and the extensive use of the https protocol has brought certain challenges to web page identification.

[0003] Existing web page recognition methods collect a large amount of data from network traffic, extract data features from each data, and annotate each data to generate a classification model. Based on the corresponding data features, web page recognition is performed through the classification model.

[0004] Since the number of data objects in a web page is uncertain, a web page may contain several to hundreds of data objects, making it difficult to establish a large and comprehensive feature dimension. Generally, only some high-dimensional and coarse-grained features can be selected for modeling. Moreover, when faced with a complex network environment, changes in the network environment will cause the size and order of data objects to fluctuate to a certain extent, resulting in insufficient accuracy of the established model and inability to achieve fine-grained web page recognition. Therefore, the prior art has the technical problem of low web page recognition accuracy. Summary of the invention

[0005] The present application provides a web page recognition method, device and storage medium based on a deep data packet inspection library, which are used to solve the technical problem of low web page recognition accuracy in the prior art.

[0006] In a first aspect, the present application provides a web page identification method based on a deep data packet inspection library, comprising:

[0007] Based on at least one known device accessing the target link multiple times, generating multiple target data object lists, and determining a target fingerprint corresponding to the target link based on the multiple target data object lists;

[0008] Acquire multiple deep packet inspection data within a preset range in a deep packet inspection library, and determine multiple candidate fingerprints based on the deep packet inspection data;

[0009] Based on the first similarities between the target fingerprint and the plurality of candidate fingerprints, a target web page list is determined.

[0010] Optionally, generating a plurality of target data object lists, and determining a target fingerprint corresponding to the target link based on the plurality of target data object lists includes:

[0011] Parsing the target link, determining the target data object generated in the process of loading the target webpage corresponding to the target link, and generating a corresponding plurality of target data object lists; wherein the target data object list corresponding to each known device is determined based on the device information of each known device and the access time period for accessing the target link;

[0012] Processing the target data objects in each target data object list respectively based on preset rules, and determining each target data object list after processing as a candidate target fingerprint;

[0013] Generate corresponding multiple candidate target fingerprint sets based on multiple target data object lists, calculate the second similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets based on a preset algorithm, and determine the target fingerprint corresponding to the target link based on the second similarity.

[0014] Optionally, processing the target data objects in each target data object list respectively based on a preset rule, and determining each processed target data object list as a candidate target fingerprint, includes:

[0015] Determine a first lower quartile point corresponding to the target data object list based on a first downlink traffic value of the target data object in the target data object list;

[0016] Filter the target data object corresponding to the first downstream traffic value less than the first lower quartile point, and determine the first start access time corresponding to the filtered target data object;

[0017] Based on the first start access time, the target data objects are sorted in time order to generate candidate target fingerprints.

[0018] Optionally, obtaining a plurality of deep packet inspection data within a preset range in a deep packet inspection library includes:

[0019] Determine the target data object that ranks first in the target fingerprint as an anchor point;

[0020] Based on the deep data packet inspection library, multiple deep data packet inspection data within a preset range after the anchor point appears are obtained; wherein the deep data packet inspection data is used to characterize a set of data segments generated by multiple terminal devices within the preset range accessing multiple https links, and each data segment in the deep data packet inspection data includes multiple candidate data objects.

[0021] Optionally, a plurality of candidate fingerprints are determined based on the deep packet inspection data, including:

[0022] Traversing each data segment of the deep packet inspection data, and determining a second lower quartile point corresponding to the data segment based on a second downstream flow value of a candidate data object in the data segment; wherein the candidate data object is used to represent a data object generated during the process of loading an https webpage corresponding to an https link;

[0023] Filter the candidate data objects corresponding to the second downstream traffic value less than the second lower quartile point, and determine the second start access time corresponding to the filtered candidate data objects;

[0024] Based on the second starting access time, the candidate data objects are sorted in time order to generate a corresponding plurality of candidate fingerprints.

[0025] Optionally, determining a target web page list based on the first similarity between the target fingerprint and the plurality of candidate fingerprints includes:

[0026] Calculating a first similarity between the target fingerprint and a plurality of candidate fingerprints based on a preset algorithm;

[0027] A target web page list is determined based on the https links corresponding to the candidate fingerprints whose first similarity is less than a preset threshold.

[0028] Optionally, before the first similarity based on the target fingerprint and the plurality of candidate fingerprints, the method further comprises:

[0029] Generate a fingerprint set to be identified based on the target fingerprint and multiple candidate fingerprints, and perform normalization on the fingerprint set to be identified;

[0030] Determine a target fingerprint and multiple candidate fingerprints in the normalized fingerprint set to be identified.

[0031] In a second aspect, the present application provides a web page recognition device based on a deep data packet inspection library, comprising:

[0032] A first processing module, configured to generate a plurality of target data object lists based on at least one known device accessing the target link multiple times, and determine a target fingerprint corresponding to the target link based on the plurality of target data object lists;

[0033] A second processing module is used to obtain a plurality of deep data packet detection data within a preset range in the deep data packet detection library, and determine a plurality of candidate fingerprints based on the deep data packet detection data;

[0034] The determination module is used to determine a target web page list based on a first similarity between the target fingerprint and a plurality of candidate fingerprints.

[0035] Optionally, the first processing module is used for:

[0036] Parsing the target link, determining the target data object generated in the process of loading the target webpage corresponding to the target link, and generating a corresponding plurality of target data object lists; wherein the target data object list corresponding to each known device is determined based on the device information of each known device and the access time period for accessing the target link;

[0037] Processing the target data objects in each target data object list respectively based on preset rules, and determining each target data object list after processing as a candidate target fingerprint;

[0038] Generate corresponding multiple candidate target fingerprint sets based on multiple target data object lists, calculate the second similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets based on a preset algorithm, and determine the target fingerprint corresponding to the target link based on the second similarity.

[0039] Optionally, the first processing module is further used for:

[0040] Determine a first lower quartile point corresponding to the target data object list based on a first downlink traffic value of the target data object in the target data object list;

[0041] Filter the target data object corresponding to the first downstream traffic value less than the first lower quartile point, and determine the first start access time corresponding to the filtered target data object;

[0042] Based on the first start access time, the target data objects are sorted in time order to generate candidate target fingerprints.

[0043] Optionally, the second processing module is used for:

[0044] Determine the target data object that ranks first in the target fingerprint as an anchor point;

[0045] Based on the deep data packet inspection library, multiple deep data packet inspection data within a preset range after the anchor point appears are obtained; wherein the deep data packet inspection data is used to characterize a set of data segments generated by multiple terminal devices within the preset range accessing multiple https links, and each data segment in the deep data packet inspection data includes multiple candidate data objects.

[0046] Optionally, the second processing module is further used for:

[0047] Traversing each data segment of the deep packet inspection data, and determining a second lower quartile point corresponding to the data segment based on a second downstream flow value of a candidate data object in the data segment; wherein the candidate data object is used to represent a data object generated during the process of loading an https webpage corresponding to an https link;

[0048] Filter the candidate data objects corresponding to the second downstream traffic value less than the second lower quartile point, and determine the second start access time corresponding to the filtered candidate data objects;

[0049] Based on the second starting access time, the candidate data objects are sorted in time order to generate a corresponding plurality of candidate fingerprints.

[0050] Optionally, the determination module is used to:

[0051] Calculating a first similarity between the target fingerprint and a plurality of candidate fingerprints based on a preset algorithm;

[0052] A target web page list is determined based on the https links corresponding to the candidate fingerprints whose first similarity is less than a preset threshold.

[0053] Optionally, the device is used to:

[0054] Generate a fingerprint set to be identified based on the target fingerprint and multiple candidate fingerprints, and perform normalization on the fingerprint set to be identified;

[0055] Determine a target fingerprint and multiple candidate fingerprints in the normalized fingerprint set to be identified.

[0056] In a third aspect of the present application, a web page recognition device based on a deep data packet inspection library is provided, comprising:

[0057] Processor and memory;

[0058] Memory stores computer-executable instructions;

[0059] The processor executes the computer-executable instructions stored in the memory, so that the web page identification device based on the deep data packet inspection library performs any one of the web page identification methods based on the deep data packet inspection library in the first aspect.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed by a processor, they are used to implement a web page identification method based on a deep data packet inspection library as described in any one of the first aspects.

[0061] The web page identification method, device and storage medium based on the deep data packet inspection library provided by the present application are based on at least one known device accessing the target link multiple times, parsing the target link, and determining the target data object generated in the process of loading the target web page corresponding to the target link, thereby avoiding the situation where the target data object is not fixed due to the network environment fluctuation that may exist when directly parsing the target web page; generating multiple target data object lists based on the target data object, and processing the multiple target data object lists based on preset rules to generate corresponding multiple candidate target fingerprint sets, and selecting the target fingerprint corresponding to the target link from the multiple candidate target fingerprint sets, thereby further reducing the impact of network environment uncertainty on web page identification; obtaining multiple deep data packet inspection data within a preset range in the deep data packet inspection library, traversing each data segment of the deep data packet inspection data, generating multiple candidate fingerprints based on the data segment, and determining the target web page list based on the similarity between the target fingerprint and the multiple candidate fingerprints, thereby achieving the technical effect of improving the accuracy of web page identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0063] Figure 1 The web page identification method process based on the deep data packet inspection library provided in the embodiment of the present application Figure 1 ;

[0064] Figure 2 The web page identification method process based on the deep data packet inspection library provided in the embodiment of the present application Figure 2 ;

[0065] Figure 3 A schematic diagram of the structure of a web page recognition device based on a deep data packet inspection library provided in an embodiment of the present application;

[0066] Figure 4 A hardware structure diagram of a web page recognition device based on a deep data packet inspection library provided in an embodiment of the present application.

[0067] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0068] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0069] In the existing web page recognition method, a large amount of data in the network traffic is collected, the data features in each data are extracted, and each data feature is labeled to generate a classification model. Since the number of data objects in the web page is uncertain, it is difficult to establish a large and comprehensive feature dimension, and generally only some high-dimensional and coarse-grained features can be selected for modeling; based on the corresponding data features, web page recognition is performed through the classification model, and the fluctuation of the network environment will cause the size and order of the data objects to fluctuate to a certain extent, resulting in insufficient accuracy of the established model and inability to achieve fine-grained web page recognition. Therefore, the existing technology has the technical problem of low web page recognition accuracy.

[0070] The web page identification method, device and storage medium based on the deep data packet inspection library provided by the present application are based on at least one known device accessing the target link multiple times, parsing the target link, and determining the target data object generated in the process of loading the target web page corresponding to the target link, thereby avoiding the situation where the target data object is not fixed due to the network environment fluctuation that may exist when directly parsing the target web page; generating multiple target data object lists based on the target data object, and processing the multiple target data object lists based on preset rules to generate corresponding multiple candidate target fingerprint sets, and selecting the target fingerprint corresponding to the target link from the multiple candidate target fingerprint sets, thereby further reducing the impact of network environment uncertainty on web page identification; obtaining multiple deep data packet inspection data within a preset range in the deep data packet inspection library, traversing each data segment of the deep data packet inspection data, generating multiple candidate fingerprints based on the data segment, and determining the target web page list based on the similarity between the target fingerprint and the multiple candidate fingerprints, thereby achieving the technical effect of improving the accuracy of web page identification.

[0071] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0072] Figure 1 The web page identification method process based on the deep data packet inspection library provided in the embodiment of the present application Figure 1 .like Figure 1As shown, the present embodiment provides a web page identification method based on a deep data packet inspection library, including:

[0073] S101, generating multiple target data object lists based on at least one known device accessing the target link multiple times, and determining a target fingerprint corresponding to the target link based on the multiple target data object lists;

[0074] In this embodiment, when accessing the same https web page, due to the uncertainty of the network environment, the data objects generated each time in the process of loading the https web page are not fixed. In order to avoid the influence of the network environment, at least one known device accesses the target link multiple times, parses the target link, determines the target web page corresponding to the target link, and determines the target data object generated in the process of loading the target web page, and then generates a corresponding target data object list; since each known device generates a target data object each time it accesses the target link, multiple target data object lists will be obtained, and the multiple target data object lists will be stored in the deep data packet inspection library; wherein, the target data object list corresponding to each known device can be determined based on the device information of the known device and the access time period of each known device accessing the target link.

[0075] In this embodiment, since any https web page contains common public data objects, such as CSS objects, advertising objects, etc., and it is impossible to determine whether the accessed https web page is the target web page based on these public data objects, the public data objects in each target data object list are filtered out; at the same time, based on the downlink traffic value of the target data object in each target data object list, the lower quartile point corresponding to each target data object list is determined, and the target data objects corresponding to the downlink traffic value less than the lower quartile point in each target data object list are filtered out; wherein, the downlink traffic value belongs to the basic attributes of the target data object, and the basic attributes of the target data object also include the website URL, source port, target port, traffic type, uplink traffic value, start access time, and end access time. The present application only involves the two basic attributes of the downlink traffic value and the start access time of the target data object.

[0076] In this embodiment, the starting access time of the filtered target data object is determined, and the target data objects in each target data object list are sorted in time order based on the starting access time to obtain candidate target fingerprints, and corresponding multiple candidate target fingerprint sets can be obtained based on multiple target data object lists; the similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets is calculated according to a preset algorithm, and a candidate target fingerprint with the smallest sum of similarities with other candidate target fingerprints is selected as the target fingerprint; wherein, the existing dynamic time adjustment DTW algorithm can be used to calculate the similarity, and the DTW algorithm can adapt to the problem of inconsistent length of target data object sequences in the candidate target fingerprints, which will not be described in detail here.

[0077] S102, obtaining a plurality of deep data packet detection data within a preset range in a deep data packet detection library, and determining a plurality of candidate fingerprints based on the deep data packet detection data;

[0078] In this embodiment, all web page access records in the network traffic are stored in the deep data packet inspection library, resulting in a very large amount of https links in the deep data packet inspection library. Therefore, it is necessary to process the https links in the deep data packet inspection library, generate corresponding candidate fingerprints, and then match the candidate fingerprints with the target fingerprints to identify the target web page. In order to speed up the matching speed and efficiency, the candidate data objects for generating candidate fingerprints are preliminarily screened with the help of the context rules of the target link, and the target data object ranked first in the target fingerprint is determined as an anchor point, and multiple deep data packet inspection data within a preset range after the anchor point appears are obtained from the deep data packet inspection library; wherein, the deep data packet inspection data is a set of data segments generated by multiple terminal devices accessing multiple https links within a preset range stored in the deep data packet inspection library, and each data segment in the deep data packet inspection data includes multiple candidate data objects.

[0079] In the first exemplary embodiment, the first target data object generated in the process of loading the target web page corresponding to the target link is determined, that is, the target data object ranked first in the target fingerprint; the target web page will be loaded within a preset time period such as 5 seconds after the target data object appears, so the target data object ranked first in the target fingerprint is determined as the anchor point, and multiple deep data packet inspection data within a preset time period such as 5 seconds after the anchor point appears are obtained from the deep data packet inspection library.

[0080] In this embodiment, each data segment in the deep data packet inspection data is traversed, and based on the downlink traffic value of the candidate data object in each data segment, the lower quartile point corresponding to each data segment is determined, and the candidate data objects corresponding to the downlink traffic value less than the lower quartile point in each data segment are filtered out; the starting access time of the filtered candidate data objects is determined, and the candidate data objects in each data segment are sorted in chronological order based on the starting access time to obtain candidate fingerprints, and corresponding multiple candidate fingerprints can be obtained based on multiple data segments.

[0081] S103: Determine a target web page list based on the first similarity between the target fingerprint and the plurality of candidate fingerprints.

[0082] In this embodiment, a fingerprint set to be identified is generated based on a target fingerprint and multiple candidate fingerprints, the fingerprint set to be identified is normalized, and the target fingerprint and multiple candidate fingerprints in the fingerprint set to be identified after the normalization are determined; based on a preset algorithm such as the DTW algorithm, the similarities between the target fingerprint and the multiple candidate fingerprints are calculated one by one, and the https links corresponding to the candidate fingerprints whose similarities are less than a preset threshold are selected to generate a corresponding https link list, which is the target web page list to be searched.

[0083] In the second example, the downlink traffic value of each fingerprint in the fingerprint set to be identified is traversed to determine the maximum downlink traffic value max(x) and the minimum downlink traffic value min(x), and all fingerprints in the fingerprint set to be identified are linearly normalized according to the following formula:

[0084]

[0085] Wherein, x is the downlink traffic value of each fingerprint in the fingerprint set to be identified.

[0086] The web page identification method based on the deep data packet inspection library provided by the present application, based on at least one known device accessing the target link multiple times, parsing the target link, determining the target data object generated in the process of loading the target web page corresponding to the target link, avoiding the situation where the target data object is not fixed due to the network environment fluctuation that may exist when directly parsing the target web page; generating multiple target data object lists based on the target data object, filtering and sorting the target data objects in each target data object list based on preset rules, generating candidate target fingerprints, generating corresponding multiple candidate target fingerprint sets based on the multiple target data objects, selecting the target fingerprint corresponding to the target link from the multiple candidate target fingerprint sets, further reducing the impact of network environment uncertainty on web page identification; based on the target data object ranked first in the target fingerprint, obtaining multiple deep data packet inspection data within a preset range in the deep data packet inspection library, traversing each data segment of the deep data packet inspection data, and filtering and sorting the candidate data objects in each data segment to generate corresponding multiple candidate fingerprints; calculating the similarity between the target fingerprint and the multiple candidate fingerprints, determining the target web page list based on the candidate fingerprints whose similarity is less than a preset threshold, achieving the technical effect of improving the accuracy of web page identification.

[0087] Figure 2 The web page identification method process based on the deep data packet inspection library provided in the embodiment of the present application Figure 2 .like Figure 2 As shown, a web page identification method based on a deep data packet inspection library provided in an embodiment of the present application includes:

[0088] S201, based on at least one known device accessing the target link multiple times, parsing the target link, determining the target data object generated in the process of loading the target webpage corresponding to the target link, and generating corresponding multiple target data object lists; wherein the target data object list corresponding to each known device is determined based on the device information of each known device and the access time period of accessing the target link;

[0089] S202, based on the first downlink traffic value of the target data object in the target data object list, determine the first lower quartile point corresponding to the target data object list; filter the target data objects corresponding to the first downlink traffic value less than the first lower quartile point, and determine the first starting access time corresponding to the filtered target data objects; sort the target data objects in time order based on the first starting access time to generate candidate target fingerprints;

[0090] S203, generating corresponding multiple candidate target fingerprint sets based on multiple target data object lists, calculating a second similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets based on a preset algorithm, and determining a target fingerprint corresponding to the target link based on the second similarity;

[0091] S204, determining the target data object ranked first in the target fingerprint as an anchor point; based on the deep data packet inspection library, obtaining multiple deep data packet inspection data within a preset range after the anchor point appears; wherein the deep data packet inspection data is used to characterize a set of data segments generated by multiple terminal devices accessing multiple https links within the preset range, and each data segment in the deep data packet inspection data includes multiple candidate data objects;

[0092] S205, traversing each data segment of the deep data packet inspection data, and determining the second lower quartile point corresponding to the data segment based on the second downstream traffic value of the candidate data object in the data segment; wherein the candidate data object is used to characterize the data object generated in the process of loading the https webpage corresponding to the https link; filtering the candidate data objects corresponding to the second downstream traffic value less than the second lower quartile point, and determining the second starting access time corresponding to the filtered candidate data objects; and sorting the candidate data objects in time order based on the second starting access time to generate a corresponding plurality of candidate fingerprints;

[0093] S206, generating a fingerprint set to be identified based on the target fingerprint and multiple candidate fingerprints, and performing normalization processing on the fingerprint set to be identified; determining the target fingerprint and multiple candidate fingerprints in the fingerprint set to be identified after the normalization processing;

[0094] S207, calculating a first similarity between the target fingerprint and multiple candidate fingerprints based on a preset algorithm; and determining a target web page list based on https links corresponding to candidate fingerprints whose first similarity is less than a preset threshold.

[0095] By executing S201 to S207, based on at least one known device accessing the target link multiple times, the target link is parsed, and the target data object generated in the process of loading the target web page corresponding to the target link is determined, thereby avoiding the situation where the target data object is not fixed due to the fluctuation of the network environment that may exist when directly parsing the target web page; multiple target data object lists are generated based on the target data object, and the target data objects in each target data object list are filtered and sorted based on preset rules to generate candidate target fingerprints, and multiple corresponding candidate target fingerprint sets are generated based on the multiple target data objects; the second similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets is calculated based on a preset algorithm, and the target fingerprint corresponding to the target link is determined based on the second similarity, thereby further reducing the impact of network environment uncertainty on web page recognition; based on the target data object ranked first in the target fingerprint, multiple deep data packet detection data within a preset range in the deep data packet detection library are obtained, each data segment of the deep data packet detection data is traversed, and multiple candidate fingerprints are generated based on the data segment; the first similarity between the target fingerprint and the multiple candidate fingerprints is calculated, and the target web page list is determined based on the candidate fingerprints whose first similarity is less than a preset threshold, thereby achieving the technical effect of improving the accuracy of web page recognition.

[0096] In the third example, a target link is accessed using a known device, and based on the device information of the known device and the access time period of the target link, a generated target data object list ob_list = {ob1, ob2, ob3, ob4, ob5, ob6, ob7, ..., ob n}, where ob n Represents a specific target data object; based on the preset rules, the target data objects in ob_list are filtered to obtain a new target data object list ob_list_new = {ob1, ob3, ob5, ob6, ob7, ...}, and the target data objects in ob_list_new are sorted in time order according to the start access time of the target data objects, forming a candidate target fingerprint standard_finger = {ob1, ob7, ob3, ob6, ob5, ...}; considering the uncertainty of the network environment, at least one known device is used to repeat the above process to generate a corresponding set of multiple candidate target fingerprints, and the similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets is calculated based on the DTW algorithm, and a candidate target fingerprint with the smallest sum of similarities with other candidate target fingerprints is selected as the target fingerprint. The target data object ranked first in the target fingerprint is used as the anchor point, and a set of data segments C = {c1, c2, c3, c4, ..., c i}, where each data segment ci ={job1, job2, job3, job4, job5, job6, job7,..., job n}, job n Represents a specific candidate data object; based on the preset rules, the data segment c i The candidate data objects in the filtered data segment are filtered, the starting access time of the candidate data objects in the filtered data segment is determined, and the candidate data objects are sorted in time order according to the starting access time to form a candidate fingerprint p i ={job4, job2, job3, job7, ...}, multiple data segments can generate multiple candidate fingerprints P ={p1, p2, p3, p4, ..., p i Based on the target fingerprint and multiple candidate fingerprints, a fingerprint set to be identified is generated M = {standard_finger, p1, p2, p3, p4, ..., p i}, the fingerprint set to be identified is normalized to obtain the normalized fingerprint set to be identified M_NDH = {standard_finger_ndh, p 1_ndh , p 2_ndh , p 3_ndh , p 4_ndh , ..., p i_ndh}; Based on the DTW algorithm, calculate standard_finger_ndh and p one by one i_ndh The similarity between them is calculated and a target web page list is determined based on the similarity.

[0097] The web page identification method based on the deep data packet inspection library provided by the present application is based on at least one known device accessing the target link multiple times, parsing the target link, determining the target data object generated in the process of loading the target web page corresponding to the target link, avoiding the situation where the target data object is not fixed due to the network environment fluctuation that may exist when directly parsing the target web page; generating multiple target data object lists based on the target data object, filtering and sorting the target data objects in each target data object list based on preset rules, generating candidate target fingerprints, generating corresponding multiple candidate target fingerprint sets based on the multiple target data objects, and selecting the target link from the multiple candidate target fingerprint sets. The corresponding target fingerprint is connected, further reducing the impact of network environment uncertainty on web page recognition; based on the target data object ranked first in the target fingerprint, multiple deep data packet detection data within a preset range in the deep data packet detection library are obtained, and the candidate data objects in each data segment of the deep data packet detection data are filtered and sorted to generate corresponding multiple candidate fingerprints; based on the target fingerprint and multiple candidate fingerprints, a set of fingerprints to be judged is generated, the set of fingerprints to be judged is normalized, the similarities between the target fingerprint and multiple candidate fingerprints in the normalized set of fingerprints to be judged are calculated one by one, and a list of target web pages is determined based on the similarities, thereby achieving a technical effect of improving the accuracy of web page recognition.

[0098] Figure 3 A schematic diagram of the structure of a web page recognition device based on a deep data packet inspection library provided in an embodiment of the present application. Figure 3 As shown, a web page identification device 300 based on a deep data packet inspection library provided in this embodiment includes: a first processing module 301, a second processing module 302, and a determination module 303.

[0099] The first processing module 301 is used to generate multiple target data object lists based on at least one known device accessing the target link multiple times, and determine the target fingerprint corresponding to the target link based on the multiple target data object lists;

[0100] The second processing module 302 is used to obtain a plurality of deep data packet detection data within a preset range in the deep data packet detection library, and determine a plurality of candidate fingerprints based on the deep data packet detection data;

[0101] The determination module 303 is used to determine a target web page list based on the first similarity between the target fingerprint and the plurality of candidate fingerprints.

[0102] In a possible implementation, the first processing module 301 is used to:

[0103] Parsing the target link, determining the target data object generated in the process of loading the target webpage corresponding to the target link, and generating a corresponding plurality of target data object lists; wherein the target data object list corresponding to each known device is determined based on the device information of each known device and the access time period for accessing the target link;

[0104] Processing the target data objects in each target data object list respectively based on preset rules, and determining each target data object list after processing as a candidate target fingerprint;

[0105] Generate corresponding multiple candidate target fingerprint sets based on multiple target data object lists, calculate the second similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets based on a preset algorithm, and determine the target fingerprint corresponding to the target link based on the second similarity.

[0106] In a possible implementation, the first processing module 301 is further configured to:

[0107] Determine a first lower quartile point corresponding to the target data object list based on a first downlink traffic value of the target data object in the target data object list;

[0108] Filter the target data object corresponding to the first downstream traffic value less than the first lower quartile point, and determine the first start access time corresponding to the filtered target data object;

[0109] Based on the first start access time, the target data objects are sorted in time order to generate candidate target fingerprints.

[0110] In a possible implementation, the second processing module 302 is used to:

[0111] Determine the target data object that ranks first in the target fingerprint as an anchor point;

[0112] Based on the deep data packet inspection library, multiple deep data packet inspection data within a preset range after the anchor point appears are obtained; wherein the deep data packet inspection data is used to characterize a set of data segments generated by multiple terminal devices within the preset range accessing multiple https links, and each data segment in the deep data packet inspection data includes multiple candidate data objects.

[0113] In a possible implementation, the second processing module 302 is further configured to:

[0114] Traversing each data segment of the deep packet inspection data, and determining a second lower quartile point corresponding to the data segment based on a second downstream flow value of a candidate data object in the data segment; wherein the candidate data object is used to represent a data object generated during the process of loading an https webpage corresponding to an https link;

[0115] Filter the candidate data objects corresponding to the second downstream traffic value less than the second lower quartile point, and determine the second start access time corresponding to the filtered candidate data objects;

[0116] Based on the second starting access time, the candidate data objects are sorted in time order to generate a corresponding plurality of candidate fingerprints.

[0117] In a possible implementation, the determination module 303 is used to:

[0118] Calculating a first similarity between the target fingerprint and a plurality of candidate fingerprints based on a preset algorithm;

[0119] A target web page list is determined based on the https links corresponding to the candidate fingerprints whose first similarity is less than a preset threshold.

[0120] In a possible implementation, the device 300 is used to:

[0121] Generate a fingerprint set to be identified based on the target fingerprint and multiple candidate fingerprints, and perform normalization on the fingerprint set to be identified;

[0122] Determine a target fingerprint and multiple candidate fingerprints in the normalized fingerprint set to be identified.

[0123] The web page recognition device based on the deep data packet detection library provided by the present application includes a first processing module, a second processing module, and a determination module. The first processing module accesses the target link multiple times based on at least one known device, parses the target link, and determines the target data object generated in the process of loading the target web page corresponding to the target link, thereby avoiding the situation that the target data object is not fixed due to the network environment fluctuation that may exist when directly parsing the target web page; generates multiple target data object lists based on the target data object, and processes the multiple target data object lists based on preset rules to generate corresponding multiple candidate target fingerprint sets, calculates the second similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets based on a preset algorithm, and determines the target fingerprint corresponding to the target link based on the second similarity, thereby further reducing the impact of network environment uncertainty on web page recognition; obtains multiple deep data packet detection data within a preset range in the deep data packet detection library based on the target data object ranked first in the target fingerprint, and traverses each data segment of the deep data packet detection data to generate multiple candidate fingerprints based on the data segment; calculates the first similarity between the target fingerprint and the multiple candidate fingerprints based on the preset algorithm by the determination module, and determines the target web page list based on the candidate fingerprints whose first similarity is less than a preset threshold, thereby achieving the technical effect of improving the accuracy of web page recognition.

[0124] Figure 4The hardware structure diagram of the web page recognition device based on the deep data packet inspection library provided in the embodiment of the present application. Figure 4 As shown, the web page identification device 400 based on the deep data packet inspection library includes:

[0125] Processor 401 and memory 402;

[0126] Memory stores computer-executable instructions;

[0127] The processor executes the computer-executable instructions stored in the memory 402, so that the web page recognition device based on the deep data packet inspection library executes the web page recognition method based on the deep data packet inspection library as described above.

[0128] It should be understood that the processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASIC). A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The memory 402 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a disk, or an optical disk.

[0129] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the web page recognition method based on the deep data packet inspection library as described above.

[0130] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0131] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0132] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0133] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0134] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0135] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0136] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0138] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A web page recognition method based on a deep data packet inspection library, characterized in that: include: Based on at least one known device accessing the target link multiple times, generating multiple target data object lists, and determining a target fingerprint corresponding to the target link based on the multiple target data object lists; Acquire a plurality of deep data packet detection data within a preset range in the deep data packet detection library, and determine a plurality of candidate fingerprints based on the deep data packet detection data; Based on the first similarity between the target fingerprint and the plurality of candidate fingerprints, a target web page list is determined.

2. The web page recognition method according to claim 1, characterized in that: The generating a plurality of target data object lists, and determining a target fingerprint corresponding to the target link based on the plurality of target data object lists, comprises: Parsing the target link, determining the target data object generated in the process of loading the target webpage corresponding to the target link, and generating the corresponding target data object lists; wherein the target data object list corresponding to each known device is determined based on the device information of each known device and the access time period for accessing the target link; Processing the target data objects in each of the target data object lists respectively based on a preset rule, and determining each of the processed target data object lists as a candidate target fingerprint; Generate corresponding multiple candidate target fingerprint sets based on the multiple target data object lists, calculate the second similarity between any two candidate target fingerprints in the multiple candidate target fingerprint sets based on a preset algorithm, and determine the target fingerprint corresponding to the target link based on the second similarity.

3. The web page recognition method according to claim 2, characterized in that: The processing of the target data objects in each of the target data object lists based on a preset rule and determining each of the processed target data object lists as a candidate target fingerprint comprises: Determine, based on a first downlink traffic value of the target data object in the target data object list, a first lower quartile point corresponding to the target data object list; Filtering target data objects corresponding to a first downstream traffic value less than the first lower quartile point, and determining a first start access time corresponding to the filtered target data objects; Based on the first start access time, the target data objects are sorted in time order to generate the candidate target fingerprints.

4. The web page recognition method according to claim 3, characterized in that: The obtaining of a plurality of deep data packet inspection data within a preset range in the deep data packet inspection library comprises: Determine the target data object ranked first in the target fingerprint as an anchor point; Based on the deep data packet inspection library, the multiple deep data packet inspection data within the preset range after the anchor point appears are obtained; wherein the deep data packet inspection data is used to characterize a set of data segments generated by multiple terminal devices within the preset range accessing multiple https links, and each data segment in the deep data packet inspection data includes multiple candidate data objects.

5. The web page recognition method according to claim 4, characterized in that: The determining a plurality of candidate fingerprints based on the deep packet inspection data comprises: Traversing each data segment of the deep data packet inspection data, and determining a second lower quartile point corresponding to the data segment based on a second downstream traffic value of the candidate data object in the data segment; wherein the candidate data object is used to represent a data object generated during the process of loading the https webpage corresponding to the https link; Filter candidate data objects corresponding to the second downstream traffic value less than the second lower quartile point, and determine a second start access time corresponding to the filtered candidate data objects; Based on the second start access time, the candidate data objects are sorted in time order to generate the corresponding multiple candidate fingerprints.

6. The web page recognition method according to claim 2, characterized in that: The step of determining a target web page list based on the first similarity between the target fingerprint and the plurality of candidate fingerprints includes: Calculating the first similarity between the target fingerprint and the plurality of candidate fingerprints based on the preset algorithm; The target web page list is determined based on the https links corresponding to the candidate fingerprints whose first similarity is less than a preset threshold.

7. The web page recognition method according to claim 1, characterized in that: Before the first similarity based on the target fingerprint and the plurality of candidate fingerprints, the method further comprises: Generate a fingerprint set to be identified based on the target fingerprint and the multiple candidate fingerprints, and perform normalization processing on the fingerprint set to be identified; The target fingerprint and the plurality of candidate fingerprints in the fingerprint set to be identified after normalization are determined.

8. A web page recognition device based on a deep data packet inspection library, characterized in that: include: A first processing module, configured to generate a plurality of target data object lists based on at least one known device accessing the target link multiple times, and determine a target fingerprint corresponding to the target link based on the plurality of target data object lists; A second processing module, configured to obtain a plurality of deep data packet detection data within a preset range in the deep data packet detection library, and determine a plurality of candidate fingerprints based on the deep data packet detection data; The determination module is used to determine a target web page list based on a first similarity between the target fingerprint and the plurality of candidate fingerprints.

9. A web page recognition device based on a deep data packet inspection library, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the web page identification method based on the deep data packet inspection library as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the web page identification method based on a deep data packet inspection library as described in any one of claims 1 to 7.

Images