Web component identification method and device based on static fingerprint

Abstract

The invention discloses a web component identification method and device based on static fingerprints. The method comprises the following steps: extracting multi-dimensional data information of a web component from original webpage data; collecting the data information of each dimension to respectively form a data set of each dimension; determining a static fingerprint of the corresponding component from the data information according to the occurrence frequency; static fingerprints corresponding to all the dimensions of all the assemblies are obtained, and a static fingerprint database is formed; initiating a request to a to-be-identified component, matching the corresponding return interface information with the static fingerprints corresponding to all dimensions in the static fingerprint database, and adding 1 to the credibility value of the component corresponding to the successfully matched static fingerprints; and obtaining the target component with the highest credibility value, and determining the to-be-identified component as the target web component. According to the scheme, a fully-covered fingerprint feature library is constructed, a multi-dimensional matching strategy is adopted, misinformation and missing report conditions can be more effectively eliminated, the operation is simple, and the workload is small.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a web component identification method and device based on static fingerprint. Background Art

[0002] With the rapid development of information technology and the widespread application of Internet technology, Web components have become an important platform for enterprises and individuals to exchange information, process business, and manage data. However, the rapid expansion of Web applications is also accompanied by an increase in security threats, such as information leakage, data tampering, and network attacks. Therefore, accurate identification and management of Web assets is crucial to improving network security and protecting data privacy. Accurate identification of Web components helps to timely discover security vulnerabilities and manage potential risks.

[0003] There are two existing web asset identification technologies for web assets. The first method is to analyze HTTP response headers, HTML codes, JavaScript variables, etc. through stack analysis tools to identify the technical frameworks and tools used by web components. This method can only identify specific fingerprint features, cannot cover all aspects, and has low recognition accuracy. The second method is to identify web component fingerprint information through command line tools, and supports plug-in extensions to improve recognition capabilities. Although this method supports plug-in extensions, users are required to write and maintain plug-ins by themselves, which increases the difficulty and workload of use. Summary of the invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of the present invention proposes a web component identification method based on static fingerprints, the method comprising:

[0005] Obtaining original web page data of each web component, and extracting data information of multiple dimensions of the web component from the original web page data, wherein the multiple dimensions at least include a referenced resource file path dimension, a website title dimension, a website icon dimension, and a return interface dimension corresponding to different status codes;

[0006] Collecting data information of each dimension to form a data set of each dimension, each data element in the data set includes the name of the web component and data information of the web component in the current dimension;

[0007] Determine the number of occurrences of each piece of data information in the data set of the corresponding dimension, and determine the static fingerprint of the corresponding web component from the data information according to the number of occurrences;

[0008] Obtaining static fingerprints corresponding to each of the web components in each dimension to form a static fingerprint library;

[0009] Initiate an HTTP request to the web component to be identified, obtain a returned response packet, and extract referenced resource file path information, website title information, and website icon information from the response packet;

[0010] Sending http requests to multiple error address pages of the web component to be identified to obtain return interface information corresponding to different status codes;

[0011] Respectively matching the resource file path information, website title information, website icon information, and return interface information corresponding to different status codes with the static fingerprints corresponding to each dimension in the static fingerprint library, and adding 1 to the credibility value of the web component corresponding to the successfully matched static fingerprint;

[0012] A target web component with the highest credibility value is obtained, and the web component to be identified is determined as the target web component.

[0013] Optionally, determining the number of occurrences of each piece of data information in a data set of a corresponding dimension, and determining a static fingerprint of a corresponding web component from the data information according to the number of occurrences includes:

[0014] In the referenced resource file path dimension, determining the number of occurrences of the referenced resource file path information of each data element in the data set of the corresponding dimension, and determining the resource file path information with a number of occurrences of 1 as the static fingerprint of the corresponding web component;

[0015] In the dimension of the return interface corresponding to the different status codes, the return interface information corresponding to the same target status code of each of the data elements is obtained, the field content of the http header in the return interface information is extracted, the number of occurrences of the field content of each of the data elements in the data set of the target status code is determined, and the field content with the number of occurrences of 1 is determined as the static fingerprint of the corresponding web component in the target status code;

[0016] In the website title dimension, obtaining website title information of each data element, and determining the number of occurrences of the website title information in the data set of the corresponding dimension, and the inclusion relationship between the website title information and other website title information, and determining the static fingerprint of the corresponding web component according to the number of occurrences and the inclusion relationship;

[0017] In the website icon dimension, the number of occurrences of the website icon information of each of the data elements in the data set of the corresponding dimension is determined, and the website icon information with an occurrence number of 1 is determined as the static fingerprint of the corresponding web component.

[0018] Optionally, determining the static fingerprint of the corresponding web component according to the number of occurrences and the inclusion relationship includes:

[0019] If the website title information appears once in the data set of the corresponding dimension, and there is no inclusion relationship between the website title information and the other website title information, the website title information is determined as the static fingerprint of the corresponding web component;

[0020] If the website title information appears once in the data set of the corresponding dimension, and there is an inclusion relationship between the website title information and the other website title information, then obtaining each target website title information having the inclusion relationship;

[0021] The cosine similarities of the website title information and each of the target website title information are calculated respectively to obtain multiple cosine similarities, and the average similarity value of the multiple cosine similarities is calculated; when the average similarity value is less than a preset first threshold, the website title information is determined as a static fingerprint of the corresponding web component; the first threshold is less than 0.5.

[0022] Optionally, respectively calculating the cosine similarity between the website title information and each target website title information includes:

[0023] Splitting the website title information and the target website title information into a plurality of word units respectively, and forming a vocabulary set with the plurality of word units;

[0024] For the word-grams included in the website title information and the target website title information, the word-grams existing in the vocabulary set are represented by 1, and the word-grams not existing in the vocabulary set are represented by 0, respectively, to obtain a first vector corresponding to the website title information and a second vector corresponding to the target website title information;

[0025] The similarity between the first vector and the second vector is calculated using cosine similarity to obtain the cosine similarity between the website title information and the target website title information.

[0026] Optionally, after obtaining the static fingerprints corresponding to each of the web components in each dimension to form a static fingerprint library, the method further includes:

[0027] Obtain resource file paths referenced by different versions of the web component from historical data;

[0028] Acquire the content of the resource file according to the resource file path, extract the content summary of the resource file, and obtain the content summary of the resource file of different versions of the web component;

[0029] The resource file content summary is stored in the static fingerprint library as the static fingerprint of the corresponding web component in the version dimension.

[0030] Optionally, after determining the web component to be identified as the target web component, the method further includes:

[0031] Obtain resource file path information from the return package, obtain a resource file according to the resource file path information, extract a content summary of the resource file, and obtain a content summary of the resource file to be identified;

[0032] Obtaining the static fingerprint of the target web component in the version dimension in the static fingerprint library, and matching the content summary of the resource file to be identified with the static fingerprint of the version dimension;

[0033] If there is a target static fingerprint that successfully matches the content summary of the resource file to be identified, the target version information of the target static fingerprint is obtained, and the target version information is determined as the version information of the target web component.

[0034] Optionally, determining the number of occurrences of the website icon information of each data element in the data set of the corresponding dimension includes:

[0035] Calculate the hash value of the website icon information of each data element, and form a data set with the hash values ​​of all the website icon information;

[0036] The number of occurrences of the hash value of the website icon information of each of the data elements in the data set is determined.

[0037] A second aspect of the present invention provides a web component identification device based on static fingerprints, the device comprising:

[0038] A data acquisition module, used to acquire original web page data of each web component, and extract data information of multiple dimensions of the web component from the original web page data, wherein the multiple dimensions at least include a referenced resource file path dimension, a website title dimension, a website icon dimension, and a return interface dimension corresponding to different status codes;

[0039] A data set forming module, used to collect data information of each dimension to form a data set of each dimension, each data element in the data set includes the name of the web component and the data information of the web component in the current dimension;

[0040] A static fingerprint determination module, used to determine the number of occurrences of each of the data information in the data set of the corresponding dimension, and determine the static fingerprint of the corresponding web component from the data information according to the number of occurrences;

[0041] A static fingerprint library building module is used to obtain the static fingerprints corresponding to each of the web components in each dimension to form a static fingerprint library;

[0042] An extraction module is used to initiate an HTTP request to the web component to be identified, obtain a returned response packet, and extract referenced resource file path information, website title information, and website icon information from the response packet;

[0043] A request module, used to send http requests to multiple error address pages of the web component to be identified, and obtain return interface information corresponding to different status codes;

[0044] A matching module, used to match the resource file path information, website title information, website icon information, and return interface information corresponding to different status codes with the static fingerprints corresponding to each dimension in the static fingerprint library, and add 1 to the credibility value of the web component corresponding to the successfully matched static fingerprint;

[0045] The web component determination module is used to obtain a target web component with the highest credibility value, and determine the web component to be identified as the target web component.

[0046] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the web component identification method based on static fingerprint as described in the first aspect.

[0047] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the web component identification method based on static fingerprint as described in the first aspect.

[0048] The embodiments of the present invention have the following beneficial effects:

[0049] In the embodiment of the present invention, by directly extracting multi-dimensional features such as resource file path information, website title information, website icon information, and return interface information corresponding to different status codes from web components, various web components and applications can be fully covered, and a comprehensive fingerprint feature library can be constructed. In addition, the use of a multi-dimensional matching strategy can more effectively eliminate false positives and false negatives, thereby ensuring high accuracy of the recognition results. After the static fingerprint library is constructed, this solution directly matches the relevant information of the web component to be identified with the static fingerprint in the static fingerprint library, which is simple to operate, has a small workload, and has high recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of a method for identifying a web component based on static fingerprints provided by an embodiment of the present invention;

[0051] Figure 2 It is a schematic diagram of a data set of resource file path dimension provided by an embodiment of the present invention;

[0052] Figure 3 It is a structural block diagram of a web component identification device based on static fingerprint provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​may be based on additional conditions or values ​​beyond the described values ​​in practice.

[0055] Figure 1 FIG. 1 is a flowchart of a method for identifying a web component based on static fingerprints provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0056] Step 101: obtain original web page data of each web component, and extract data information of multiple dimensions of the web component from the original web page data, wherein the multiple dimensions at least include a referenced resource file path dimension, a website title dimension, a website icon dimension, and a return interface dimension corresponding to different status codes.

[0057] Web components are a set of technologies that allow developers to create reusable custom elements. These custom elements encapsulate specific functionality, making them reusable in web applications.

[0058] The web component in the embodiment of the present invention can also be extended to a web framework, that is, the method of the present invention is also applicable to a web framework. A web framework is a widespread development tool that provides a set of predefined codes and structures to help developers build web applications.

[0059] The original web page data of the web component, including the description of the web component, the website data corresponding to the web component, and the relevant code address.

[0060] Encapsulates an algorithm that uses a general abstract syntax tree (AST) analysis framework to parse raw web page data and extract routing and HTTP request configuration information from it. The algorithm defines the grammatical rules of various Web languages, generates corresponding parsers, traverses code files and generates ASTs, extracts and formats routing and HTTP request configurations from them. In addition, by simulating HTTP requests, the algorithm can parse the web page content of the specified URL (uniform resource locator), extract resource files and title information referenced by the page, and implement standardized analysis and extraction of web page elements. At the same time, the algorithm obtains the data information returned by the Web component for different status code interfaces and the content summary data of the website icon.

[0061] In this way, the above algorithm can extract data information of multiple dimensions of web components from the original web page data, including at least the referenced resource file path dimension, the website title dimension, the website icon dimension and the return interface dimension corresponding to different status codes.

[0062] Specifically, the referenced resource file path information can be extracted from the source code page corresponding to the original web page. For example, the referenced resource file path information extracted by web component A is as follows:

[0063] / K3Cloud / HTML5 / content / themes / kd.ui.login.css

[0064] / K3Cloud / HTML5 / content / themes / kdcss.kdvgrid.min.css

[0065] / K3Cloud / HTML5 / content / themes / kdcss.standard.min.css

[0066] / K3Cloud / HTML5 / Script / kd / global / KDRes_2052.js

[0067] The website title information extracted by web component A is: AAA

[0068] The website icon information is an image. For the convenience of subsequent calculation, the image can be converted into content summary data through the MD5 algorithm. For example, the content summary data of the website icon of web component A is: 825af13371930eeb2f85cf075fa25b68.

[0069] Different status codes can specifically include: 404 status code for page not found, 301 status code for page redirection, 500 status code for server attack, etc. By requesting the website links corresponding to different status codes one by one, you can get the return interface information of different status codes.

[0070] For example, the response information of the 301 status code corresponding to web component A is:

[0071] HTTP / 2 301 Moved Permanently

[0072] Content-Type: text / html; charset=UTF-8

[0073] Location:https: / / kd.vdegs.vde.com / K3Cloud

[0074] Server:Microsoft-IIS / 10.0

[0075] X-Powered-By:ASP.NET

[0076] Date:Fri,19Jul 2024 03:29:52GMT

[0077] Content-Length:155

[0078] <title> Document Moved< / title>

[0079] <h1> Object Moved< / h1>This document may be found <a href＝"https: kd.vdegs.vde.com k3cloud"> here

[0080] Step 102: Collect the data information of each dimension to form a data set of each dimension, each data element in the data set includes the name of the web component and the data information of the web component in the current dimension.

[0081] The data information of multiple dimensions of each web component is saved according to the category of the dimension to form a data set corresponding to each dimension, wherein the data elements in each data set include the name of the web component and the data information of the web component in the current dimension.

[0082] Step 103: Determine the number of occurrences of each piece of data information in the data set of the corresponding dimension, and determine the static fingerprint of the corresponding web component from the data information according to the number of occurrences.

[0083] If the data information of a web component in a certain dimension appears less frequently in the corresponding data set, it means that the data information is unique to the web component and can be used to identify the web component as a static fingerprint for identifying the web component.

[0084] Step 104: Obtain the static fingerprint corresponding to each of the web components in each dimension to form a static fingerprint library.

[0085] Obtain the static fingerprints corresponding to each web component in each dimension to form a static fingerprint library. The static fingerprint library includes static fingerprint libraries corresponding to each dimension, and the data elements in each static fingerprint library are the identifier of the web component and the static fingerprint of the web component in the dimension.

[0086] Step 105: Initiate an HTTP request to the web component to be identified, obtain a returned response packet, and extract referenced resource file path information, website title information, and website icon information from the response packet.

[0087] After the static fingerprint library is successfully built, it can be used to identify web assets.

[0088] Specifically, an http request may be initiated to the address corresponding to the web component to be identified, data information of various dimensions may be extracted from the returned response packet, and the data information may be matched with the static fingerprint in the static fingerprint library.

[0089] Step 106: Send an http request to the error address page of the web component to be identified, and obtain return interface information corresponding to different status codes.

[0090] The return interface information corresponding to different status codes cannot be obtained by directly initiating an http request to the address corresponding to the web component, but can be obtained by sending an http request to the relevant error address page of the web component.

[0091] Specifically, a sixty-digit random letter combination can be hard-coded as the error address, and an http request is sent to the error address to obtain the return interface information corresponding to the 404 status code.

[0092] In addition, other status code interfaces also have corresponding web page addresses. By sending requests to these web page addresses, you can get the return interface information corresponding to different status codes.

[0093] Step 107: respectively match the resource file path information, website title information, website icon information, and return interface information corresponding to different status codes with the static fingerprints corresponding to each dimension in the static fingerprint library, and add 1 to the credibility value of the web component corresponding to the successfully matched static fingerprint.

[0094] For example, if the resource file path information of the web component to be identified matches the static fingerprint of the resource file path dimension of web component A, and also matches the static fingerprint of the website title dimension of web component A, then the credibility value of web component A is 2. If the website icon information of the web component to be identified matches the static fingerprint of the website icon dimension of web component B, then the credibility value of web component B is 1.

[0095] Step 108: Obtain a target web component with the highest credibility value, and determine the web component to be identified as the target web component.

[0096] The target web component with the highest credibility value has the most static fingerprints that successfully match the web component to be identified, which means that the web component to be identified is most likely to be the target web component, so the web component to be identified is determined to be the target web component.

[0097] In summary, in the embodiment of the present invention, by directly extracting multi-dimensional features such as resource file path information, website title information, website icon information, and return interface information corresponding to different status codes from web components, various web components and applications can be fully covered, and a comprehensive fingerprint feature library can be constructed. In addition, the use of a multi-dimensional matching strategy can more effectively eliminate false positives and false negatives, thereby ensuring high accuracy of the recognition results. After the static fingerprint library is constructed, this solution directly matches the relevant information of the web component to be identified with the static fingerprint in the static fingerprint library, which is simple to operate, has a small workload, and has high recognition efficiency.

[0098] In a possible implementation, step 103 includes:

[0099] Step 1031: In the referenced resource file path dimension, determine the number of occurrences of the referenced resource file path information of each data element in the data set of the corresponding dimension, and determine the resource file path information with an occurrence number of 1 as the static fingerprint of the corresponding web component.

[0100] Figure 2 It is a schematic diagram of a data set of a resource file path dimension provided by an embodiment of the present invention.

[0101] like Figure 2 As shown in the figure, the left side shows the referenced resource file path information of web component A, web component B, web component C, and web component D, and each web component has multiple referenced resource file paths. The middle is the sum of the resource file path information of the above web components, that is, the formed data set. The rightmost side counts the number of occurrences of each resource file path information in the data set.

[0102] For the resource file paths introduced by web component A, it can be found that these file paths appear once in the data set, so these are unique file paths. Therefore, under the dimension of referenced resource file paths, the resource file paths in the table are all static fingerprints of web component A.

[0103] Looking at web component B again, the three resource file paths appear twice each, which is not unique. Therefore, no static fingerprint of web component B in the dimension of referenced resource file paths is found.

[0104] Let's look at web component C and web component D. The resource file paths they introduce appear twice and once. We take the resource file paths that appear once as the static fingerprints of web component C and web component D in the dimension of referenced resource file paths.

[0105] Step 1032: In the dimension of the return interface corresponding to the different status codes, obtain the return interface information corresponding to the same target status code of each of the data elements, extract the field content of the http header in the return interface information, determine the number of occurrences of the field content of each of the data elements in the data set of the target status code, and determine the field content with an occurrence number of 1 as the static fingerprint of the corresponding web component in the target status code.

[0106] For example, the returned interface information of web component A, web component B, web component C, and web component D under different status codes is obtained. The returned interface information corresponding to the same target status code of web component A, web component B, web component C, and web component D constitutes a data set corresponding to the target status code. Here, the 302 status code is used as an example.

[0107] 302 is a redirection type status code in HTTP status code. 302 indicates temporary redirection, which means that the location of the resource has temporarily changed. For the 302 status code, pay attention to the Location field of the http header.

[0108] In the 302 status code return interface corresponding to web component A, the location field content is: / index.php?m=user&f=login&referer=Lw==, which appears once in the 302 status code data set, so the Location field content can be used as the static fingerprint of the 302 status code of web component A. In the 302 status code return interface corresponding to web component B, the location field content is: Location:. / K3Cloud / Error.html, which appears twice in the 302 status code data set, so the static fingerprint of the 302 status code of web component B does not exist.

[0109] Step 1033: In the website title dimension, obtain the website title information of each of the data elements, and determine the number of occurrences of the website title information in the data set of the corresponding dimension, as well as the inclusion relationship between the website title information and other website title information, and determine the static fingerprint of the corresponding web component based on the number of occurrences and the inclusion relationship.

[0110] For example, the following Table 1 is a data set consisting of website title information of various web components.

[0111] Table 1

[0112] Component Name Website Title xxxx-1 Backstage management system xxxx-2 Backstage login system xxxx-3 Backstage management system login xxxx-4 Login System xxxx-5 System Login xxxx-6 Information Management System xxxx-7 User login xxxx-8 User login platform xxxx-9 Log in to the SCM management system xxxx-10 Zhuzhi User Login System

[0113] Since "Zhuzhi User Login System" appears once in the data set, "Zhuzhi User Login System" can be used as the static fingerprint of web component xxxx-10 in the website title dimension.

[0114] The website title "Login System" of the component "xxxx-4" has an inclusion relationship with other website titles. At this time, it is necessary to determine the static fingerprint of the corresponding web component based on the number of occurrences and the inclusion relationship.

[0115] In a possible implementation, determining the static fingerprint of the corresponding web component according to the number of occurrences and the inclusion relationship in step 1033 includes:

[0116] Step 10331: If the website title information appears once in the data set of the corresponding dimension, and there is no inclusion relationship between the website title information and the other website title information, the website title information is determined as the static fingerprint of the corresponding web component;

[0117] Step 10332: If the website title information appears once in the data set of the corresponding dimension, and there is an inclusion relationship between the website title information and the other website title information, then obtain the target website title information with the inclusion relationship;

[0118] Step 10333, respectively calculate the cosine similarity of the website title information and each of the target website title information to obtain multiple cosine similarities, and calculate the average similarity value of the multiple cosine similarities; when the average similarity value is less than a preset first threshold, determine the website title information as a static fingerprint of the corresponding web component; the first threshold is less than 0.5.

[0119] In steps 10331 to 10333, for example, the website title "Login System" of component "xxxx-4" has an inclusion relationship with the website titles "Backstage Login System" and "Zhuzhi User Login System". At this time, it is necessary to calculate the cosine similarity between "Login System" and "Backstage Login System", and between "Login System" and "Zhuzhi User Login System" in turn.

[0120] The two calculated cosine similarity values ​​are averaged to obtain an average similarity value. If the average similarity value is less than the preset first threshold, "login system" is used as the static fingerprint of component "xxxx-4" in the website title information dimension.

[0121] If the average similarity value is greater than or equal to the first threshold, the component "xxxx-4" has no static fingerprint in the website title information dimension.

[0122] In a possible implementation, calculating the cosine similarity of the website title information and each target website title information in step 10333 includes:

[0123] Step 103331: split the website title information and the target website title information into a plurality of word units respectively, and form a vocabulary set with the plurality of word units;

[0124] Step 103332: for the word-grams included in the website title information and the target website title information, the word-grams existing in the vocabulary set are represented by 1, and the word-grams not existing in the vocabulary set are represented by 0, respectively, to obtain a first vector corresponding to the website title information and a second vector corresponding to the target website title information;

[0125] Step 103333: Calculate the similarity between the first vector and the second vector using cosine similarity to obtain the cosine similarity between the website title information and the target website title information.

[0126] In step 103331 to step 103333, taking the calculation of "login system" and "back-end login system" as an example, the steps of calculating the cosine similarity of the website title information and the title information of each target website are explained.

[0127] First, "login system" and "background login system" are vectorized. Specifically, "login system" and "background login system" are split into multiple tokens, and multiple tokens form a vocabulary set, for example, the vocabulary set is ["login", "system", "background"].

[0128] "Login" and "system" both exist in the vocabulary, but "background" does not exist. The existing ones are set to 1, and the non-existent ones are set to 0, so "login system" is the first vector [1,1,0] after vectorization, and "background login system" is the second vector [1,1,1] after vectorization.

[0129] The cosine similarity is used to calculate the similarity between the first vector and the second vector. The formula for calculating the cosine similarity is:

[0130]

[0131] Wherein, A and B represent the first vector and the second vector respectively.

[0132] In this way, the cosine similarity calculation process of the first vector [1,1,0] and the second vector [1,1,1] is:

[0133]

[0134] Therefore, the cosine similarity between "login system" and "background login system" is: 0.8165.

[0135] Similarly, we can calculate that the cospin similarity between “login system” and “Zhuzhi user login system” is 0.7071.

[0136] If the first threshold is 0.4, since both cosine similarities are greater than 0.4, "login system" cannot be used as the static fingerprint of component "xxxx-4".

[0137] Step 1034: In the website icon dimension, determine the number of occurrences of the website icon information of each of the data elements in the data set of the corresponding dimension, and determine the website icon information with an occurrence number of 1 as the static fingerprint of the corresponding web component.

[0138] The website icon information is an image, which can be converted into content summary data through the MD5 algorithm, and the number of occurrences of the website icon in the data set can be determined by comparing the content summary data.

[0139] In a possible implementation, determining the number of occurrences of the website icon information of each data element in the data set of the corresponding dimension in step 1034 includes:

[0140] Step 10341: Calculate the hash value of the website icon information of each data element, and form a data set consisting of the hash values ​​of all the website icon information.

[0141] Step 10342: Determine the number of occurrences of the hash value of the website icon information of each data element in the data set.

[0142] In step 10341-step 10342, the hash algorithm is an algorithm that converts data of any length into a fixed length value. The main idea is to generate a unique binary string, i.e., a hash value, through a series of calculations and transformations of the input data. To generate a hash value of input data, it is only necessary to pass the input data as a parameter to a suitable hash function and obtain its return value. For example, the MD5 algorithm is used as a hash function.

[0143] The website icon information is an image. For the convenience of subsequent calculation, the image can be converted into content summary data through a hash algorithm. In this way, by determining the number of occurrences of the hash value of each website icon information in the data set, the number of occurrences of the website icon information in the data set of the corresponding dimension can be determined.

[0144] In a possible implementation manner, after step 104, the method further includes:

[0145] Step 201: Obtain resource file paths referenced by different versions of the web components from historical data;

[0146] Step 202: acquiring the content of the resource file according to the resource file path, extracting the content summary of the resource file, and obtaining the content summary of the resource file of different versions of the web component;

[0147] Step 203: Store the resource file content summary as the static fingerprint of the corresponding web component in the version dimension in the static fingerprint library.

[0148] In step 201-step 203, in the prior art, only feature files or feature information are used for matching to determine Web fingerprint information. This purely static content matching method has limitations in identifying dynamic content. For example, after the developer makes targeted modifications to the web component, the purely static content matching method may not be effective and is difficult to cope with the complex and dynamic changes in modern Web applications.

[0149] This solution can provide a version identification function by storing the version and the corresponding resource file content summary in the static fingerprint library. The version identification capability is of great significance for security vulnerability detection and system maintenance.

[0150] Specifically, since different versions of web components may reference different resource file information, the resource file path information referenced by different versions of web components is obtained from historical data, and the resource file is obtained by accessing the resource file path, extracting the content summary of the resource file, and obtaining the resource file content summary of different versions of web components.

[0151] The content summaries of different resource files are stored in the fingerprint library as static fingerprints of different versions of web components for subsequent use in identifying web component versions.

[0152] In a possible implementation manner, after step 108, the method further includes:

[0153] Step 206: Obtain resource file path information from the return package, obtain a resource file according to the resource file path information, extract a content summary of the resource file, and obtain a content summary of the resource file to be identified;

[0154] Step 207: Obtain the static fingerprint of the target web component in the version dimension from the static fingerprint library, and match the content summary of the resource file to be identified with the static fingerprint of the version dimension;

[0155] Step 208: If there is a target static fingerprint that successfully matches the content digest of the resource file to be identified, then obtain target version information of the target static fingerprint, and determine the target version information as the version information of the target web component.

[0156] In step 206-step 207, in order to determine the version information of the target web component, the resource file corresponding to the resource file path information in the return package is obtained, and the content summary of the resource file is extracted to obtain the content summary of the resource file to be identified, and the content summary of the resource file to be identified is matched with the static fingerprint of the target web component in the version dimension in the static fingerprint library to obtain the version information of the content summary of the resource file to be identified, thereby obtaining the version information of the target web component.

[0157] This solution determines the version information of the web component based on the content summary of the resource file, making the web component identification capability more comprehensive. After determining the version information of the web component, security vulnerability detection and system maintenance can be made more targeted.

[0158] Figure 3 is a structural block diagram of a web component identification device based on static fingerprint provided by an embodiment of the present invention. Figure 3 As shown, the device 300 includes:

[0159] The data acquisition module 301 is used to acquire the original web page data of each web component, and extract data information of multiple dimensions of the web component from the original web page data, wherein the multiple dimensions at least include a referenced resource file path dimension, a website title dimension, a website icon dimension, and a return interface dimension corresponding to different status codes;

[0160] A data set forming module 302 is used to collect data information of each dimension to form a data set of each dimension, each data element in the data set includes the name of the web component and the data information of the web component in the current dimension;

[0161] The static fingerprint determination module 303 is used to determine the number of occurrences of each of the data information in the data set of the corresponding dimension, and determine the static fingerprint of the corresponding web component from the data information according to the number of occurrences;

[0162] The static fingerprint library building module 304 is used to obtain the static fingerprints corresponding to each of the web components in each dimension to form a static fingerprint library;

[0163] The extraction module 305 is used to initiate an HTTP request to the web component to be identified, obtain a returned response packet, and extract referenced resource file path information, website title information, and website icon information from the response packet;

[0164] The request module 306 is used to send HTTP requests to multiple error address pages of the web component to be identified, and obtain return interface information corresponding to different status codes;

[0165] The matching module 307 is used to match the resource file path information, website title information, website icon information, and return interface information corresponding to different status codes with the static fingerprints corresponding to each dimension in the static fingerprint library, and add 1 to the credibility value of the web component corresponding to the successfully matched static fingerprint;

[0166] The web component determination module 308 is used to obtain a target web component with the highest credibility value, and determine the web component to be identified as the target web component.

[0167] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0169] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that comply with the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0170] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A web component identification method based on static fingerprint, characterized in that: The method comprises: Obtaining original web page data of each web component, and extracting data information of multiple dimensions of the web component from the original web page data, wherein the multiple dimensions at least include a referenced resource file path dimension, a website title dimension, a website icon dimension, and a return interface dimension corresponding to different status codes; Collecting data information of each dimension to form a data set of each dimension, each data element in the data set includes the name of the web component and data information of the web component in the current dimension; Determine the number of occurrences of each piece of data information in the data set of the corresponding dimension, and determine the static fingerprint of the corresponding web component from the data information according to the number of occurrences; Obtaining static fingerprints corresponding to each of the web components in each dimension to form a static fingerprint library; Initiate an HTTP request to the web component to be identified, obtain a returned response packet, and extract referenced resource file path information, website title information, and website icon information from the response packet; Sending http requests to multiple error address pages of the web component to be identified to obtain return interface information corresponding to different status codes; Respectively matching the resource file path information, website title information, website icon information, and return interface information corresponding to different status codes with the static fingerprints corresponding to each dimension in the static fingerprint library, and adding 1 to the credibility value of the web component corresponding to the successfully matched static fingerprint; A target web component with the highest credibility value is obtained, and the web component to be identified is determined as the target web component.

2. The method according to claim 1, characterized in that The determining the number of occurrences of each piece of data information in a data set of a corresponding dimension, and determining a static fingerprint of a corresponding web component from the data information according to the number of occurrences, includes: In the referenced resource file path dimension, determining the number of occurrences of the referenced resource file path information of each data element in the data set of the corresponding dimension, and determining the resource file path information with a number of occurrences of 1 as the static fingerprint of the corresponding web component; In the dimension of the return interface corresponding to the different status codes, the return interface information corresponding to the same target status code of each of the data elements is obtained, the field content of the http header in the return interface information is extracted, the number of occurrences of the field content of each of the data elements in the data set of the target status code is determined, and the field content with the number of occurrences of 1 is determined as the static fingerprint of the corresponding web component in the target status code; In the website title dimension, obtaining website title information of each data element, and determining the number of occurrences of the website title information in the data set of the corresponding dimension, and the inclusion relationship between the website title information and other website title information, and determining the static fingerprint of the corresponding web component according to the number of occurrences and the inclusion relationship; In the website icon dimension, the number of occurrences of the website icon information of each of the data elements in the data set of the corresponding dimension is determined, and the website icon information with an occurrence number of 1 is determined as the static fingerprint of the corresponding web component.

3. The method according to claim 2, characterized in that Determining the static fingerprint of the corresponding web component according to the number of occurrences and the inclusion relationship includes: If the website title information appears once in the data set of the corresponding dimension, and there is no inclusion relationship between the website title information and the other website title information, the website title information is determined as the static fingerprint of the corresponding web component; If the website title information appears once in the data set of the corresponding dimension, and there is an inclusion relationship between the website title information and the other website title information, then obtaining each target website title information having the inclusion relationship; The cosine similarities of the website title information and each of the target website title information are calculated respectively to obtain multiple cosine similarities, and the average similarity value of the multiple cosine similarities is calculated; when the average similarity value is less than a preset first threshold, the website title information is determined as a static fingerprint of the corresponding web component; the first threshold is less than 0.

5.

4. The method according to claim 3, characterized in that: The respectively calculating the cosine similarity between the website title information and each of the target website title information comprises: Splitting the website title information and the target website title information into a plurality of word units respectively, and forming a vocabulary set with the plurality of word units; For the word-grams included in the website title information and the target website title information, the word-grams existing in the vocabulary set are represented by 1, and the word-grams not existing in the vocabulary set are represented by 0, respectively, to obtain a first vector corresponding to the website title information and a second vector corresponding to the target website title information; The similarity between the first vector and the second vector is calculated using cosine similarity to obtain the cosine similarity between the website title information and the target website title information.

5. The method according to claim 1, characterized in that After obtaining the static fingerprints corresponding to each of the web components in each dimension and forming a static fingerprint library, the following further includes: Obtain resource file paths referenced by different versions of the web component from historical data; Acquire the content of the resource file according to the resource file path, extract the content summary of the resource file, and obtain the content summary of the resource file of different versions of the web component; The resource file content summary is stored in the static fingerprint library as the static fingerprint of the corresponding web component in the version dimension.

6. The method according to claim 5, characterized in that After determining the web component to be identified as the target web component, the method further includes: Obtain resource file path information from the return package, obtain a resource file according to the resource file path information, extract a content summary of the resource file, and obtain a content summary of the resource file to be identified; Obtaining the static fingerprint of the target web component in the version dimension in the static fingerprint library, and matching the content summary of the resource file to be identified with the static fingerprint of the version dimension; If there is a target static fingerprint that successfully matches the content summary of the resource file to be identified, the target version information of the target static fingerprint is obtained, and the target version information is determined as the version information of the target web component.

7. The method according to claim 2, characterized in that Determining the number of occurrences of the website icon information of each data element in the data set of the corresponding dimension includes: Calculate the hash value of the website icon information of each data element, and form a data set with the hash values ​​of all the website icon information; The number of occurrences of the hash value of the website icon information of each of the data elements in the data set is determined.

8. A web component identification device based on static fingerprint, characterized in that: The device comprises: A data acquisition module, used to acquire original web page data of each web component, and extract data information of multiple dimensions of the web component from the original web page data, wherein the multiple dimensions at least include a referenced resource file path dimension, a website title dimension, a website icon dimension, and a return interface dimension corresponding to different status codes; A data set forming module, used to collect data information of each dimension to form a data set of each dimension, each data element in the data set includes the name of the web component and the data information of the web component in the current dimension; A static fingerprint determination module, used to determine the number of occurrences of each of the data information in the data set of the corresponding dimension, and determine the static fingerprint of the corresponding web component from the data information according to the number of occurrences; A static fingerprint library building module is used to obtain the static fingerprints corresponding to each of the web components in each dimension to form a static fingerprint library; An extraction module is used to initiate an HTTP request to the web component to be identified, obtain a returned response packet, and extract referenced resource file path information, website title information, and website icon information from the response packet; A request module, used to send http requests to multiple error address pages of the web component to be identified, and obtain return interface information corresponding to different status codes; A matching module, used to match the resource file path information, website title information, website icon information, and return interface information corresponding to different status codes with the static fingerprints corresponding to each dimension in the static fingerprint library, and add 1 to the credibility value of the web component corresponding to the successfully matched static fingerprint; The web component determination module is used to obtain a target web component with the highest credibility value, and determine the web component to be identified as the target web component.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the web component identification method based on static fingerprint as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the web component identification method based on static fingerprint according to any one of claims 1 to 7.

Images