A vulnerability detection method and apparatus

Abstract

The application relates to a vulnerability detection method and device, and relates to the technical field of vulnerability scanning and vulnerability management. The method comprises the following steps: obtaining a current URL to be detected in a web application; initiating a request to a server through the current URL, and determining a target URL according to the page jump condition of the web application in the case that response data returned by the server is received; performing validity evaluation on the target URL according to the response data to obtain a validity evaluation result; obtaining a target vulnerability verification script matched with the target URL in the case that the validity evaluation result meets preset evaluation requirements; and performing vulnerability detection on the target URL by using the target vulnerability verification script. According to the page jump condition of the web application, the target URL actually processing the request can be determined in the redirection scene driven by the front end, then the validity evaluation is performed on the target URL according to the response data, the vulnerability detection on the real and valid target URL is realized, and the validity of the vulnerability detection is improved.

Description

Technical Field

[0001] This application relates to the field of vulnerability scanning and vulnerability management technology, and in particular to a vulnerability detection method and apparatus. Background Technology

[0002] A web application is a program accessed through a browser and running on a remote server. Users interact with the web application over the network to achieve specific functions. Vulnerabilities in web applications, such as Structured Query Language (SQL) injection, unauthorized access, and file reading vulnerabilities, can lead to the leakage of users' personal information, contract information, and other private data. Therefore, vulnerability detection for web applications is crucial for ensuring data security and business continuity.

[0003] In related technologies, vulnerability scanning tools typically only detect the initial Uniform Resource Locator (URL), which can easily lead to the omission of valid URLs in front-end driven redirection scenarios.

[0004] Therefore, how to achieve vulnerability detection on valid URLs and improve the effectiveness of vulnerability detection is an issue worthy of attention. Summary of the Invention

[0005] Therefore, it is necessary to provide a vulnerability detection method and apparatus that can perform vulnerability detection on valid URLs and improve the effectiveness of vulnerability detection, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a vulnerability detection method, including:

[0007] Get the current URL to be detected in the web application;

[0008] A request is sent to the server using the current URL, and upon receiving the response data from the server, the target URL is determined based on the page redirection behavior of the web application.

[0009] Based on the response data, the validity of the target URL is evaluated, and the validity evaluation result is obtained;

[0010] If the effectiveness evaluation results meet the preset evaluation requirements, obtain the target vulnerability verification script that matches the target URL;

[0011] Use a target vulnerability verification script to perform vulnerability detection on the target URL.

[0012] In one embodiment, determining the target URL based on the page redirection status of the web application includes: determining the target URL based on response data when the web application redirects; and using the current URL as the target URL when the web application does not redirect.

[0013] In one embodiment, determining the target URL based on the response data includes: determining the target redirection rule for the page redirection in the web application; determining the redirection URL corresponding to the redirected page based on the target redirection rule and the response data; and determining the target URL based on the redirection URL.

[0014] In one embodiment, determining the redirect URL corresponding to the redirect page based on the target redirect rule and the response data includes: if the target redirect rule is a static string concatenation redirect, restoring the static string contained in the response data to obtain the redirect path; and determining the redirect URL corresponding to the redirect page based on the redirect path.

[0015] In one embodiment, determining the redirect URL corresponding to the redirect page based on the target redirect rule and the response data includes: extracting the reference URL included in the response data when the target redirect rule is the first redirect rule; using the reference URL as the redirect URL corresponding to the redirect page; wherein the first redirect rule is a redirect rule other than static string concatenation redirection.

[0016] In one embodiment, determining the target URL based on the redirect URL includes: normalizing the redirect URL to obtain a normalized URL; and determining the target URL based on the normalized URL.

[0017] In one embodiment, determining the target URL based on the redirect URL includes: taking the redirect URL as the new current URL, returning to execute the steps of initiating a request to the server through the current URL, and, upon receiving the response data returned by the server, determining the target URL based on the page redirection situation of the web application, until the target URL is obtained.

[0018] In one embodiment, the validity of the target URL is evaluated based on the response data to obtain a validity evaluation result, including: determining a first evaluation result corresponding to the target URL based on the status code and preset status code in the response data; determining a second evaluation result corresponding to the target URL based on the existence of business-oriented links within the response data; determining a third evaluation result corresponding to the target URL based on the existence of static resources in the response data; determining a fourth evaluation result corresponding to the target URL based on the existence of forms in the response data; determining a fifth evaluation result corresponding to the target URL based on the consistency of the page title and structure summary in the response data after multiple consecutive requests; determining a sixth evaluation result corresponding to the target URL based on the structure of the navigation area in the page corresponding to the target URL; and determining a validity evaluation result based on at least one of the first, second, third, fourth, fifth, and sixth evaluation results.

[0019] In one embodiment, after a request is initiated to the server via the current URL, the method includes: switching the first protocol used in the current request to a second protocol if the request process meets preset protocol switching conditions; wherein the preset protocol switching conditions include at least one of the following: the server rejects the request multiple times, the Transport Layer Security (TLS) handshake between the server fails, the server returns an explicit protocol mismatch message, and the server returns a status code pointing to another protocol address under the current URL; and based on the second protocol, re-initiating a request to the server via the current URL.

[0020] Secondly, this application provides a vulnerability detection device, comprising:

[0021] The first acquisition module is used to acquire the current URL to be detected in the web application;

[0022] The determination module is used to send a request to the server using the current URL, and upon receiving the response data returned by the server, to determine the target URL based on the page redirection behavior of the web application.

[0023] The evaluation module is used to evaluate the validity of the target URL based on the response data and obtain the validity evaluation result;

[0024] The second acquisition module is used to acquire the target vulnerability verification script that matches the target URL, provided that the validity evaluation result meets the preset evaluation requirements.

[0025] The detection module is used to perform vulnerability detection on the target URL using the target vulnerability verification script.

[0026] The aforementioned vulnerability detection method and apparatus obtain the current URL to be detected in a web application; initiate a request to the server using the current URL, and upon receiving response data from the server, determine the target URL based on the page redirection behavior of the web application; evaluate the validity of the target URL based on the response data, and obtain a validity evaluation result; if the validity evaluation result meets preset evaluation requirements, obtain a target vulnerability verification script that matches the target URL; and use the target vulnerability verification script to perform vulnerability detection on the target URL. In the above process, based on the page redirection behavior of the web application, in front-end driven redirection scenarios, the actual URL processing the request, i.e., the target URL, can be determined. Therefore, based on the response data, the target URL is evaluated for validity, thereby enabling vulnerability detection on truly valid target URLs and improving the effectiveness of vulnerability detection. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a vulnerability detection method in one embodiment;

[0029] Figure 2 This is a flowchart illustrating the steps for determining the target URL in one embodiment;

[0030] Figure 3 This is a flowchart illustrating the steps for evaluating the validity of a target URL in one embodiment;

[0031] Figure 4 This is a flowchart illustrating a vulnerability detection method in another embodiment;

[0032] Figure 5 This is a flowchart illustrating a vulnerability detection method in yet another embodiment;

[0033] Figure 6 This is a structural block diagram of a vulnerability detection system in one embodiment;

[0034] Figure 7 This is a structural block diagram of a vulnerability detection device in one embodiment;

[0035] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0037] The vulnerability detection method provided in this application can be applied to computer devices, which can be terminals or servers. The terminals can be, but are not limited to, various personal computers, laptops, smartphones, and tablets. The servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide cloud computing services.

[0038] In related technologies, in front-end driven redirection scenarios, after a request is initiated based on the initial URL, the server can return a success status code, but will not return a redirection-related status code; the redirect address is located in the page source code. Traditional vulnerability scanning tools cannot identify redirection instructions included in the page source code, i.e., they cannot identify valid URLs, but instead use the initial URL as the final business address for vulnerability detection, thus easily leading to the missed detection of valid URLs.

[0039] In one alternative embodiment, such as Figure 1 As shown, a vulnerability detection method is provided that can be applied to front-end driver redirection scenarios. The following explanation uses the application of this method to a server as an example, and the method includes the following steps:

[0040] S110, Get the current URL to be detected in the web application.

[0041] Among these, web applications are web applications. Web applications can be understood as web applications that are to be tested for vulnerabilities.

[0042] Optionally, the web application may include at least one URL to be detected.

[0043] In some embodiments, during the initialization phase, at least one URL to be detected may be written to a URL queue.

[0044] In some embodiments, during the initialization phase, the set of visited and deduplicated URLs U_seen, the number of jump depths, and the number of protocol switches can be initialized.

[0045] The set of visited and deduplicated URLs can be understood as the set of URLs for which requests have been sent to the server and which have undergone deduplication.

[0046] The redirect depth can be understood as the number of times the page redirects from the starting URL.

[0047] The starting URL can be understood as the unified access address exposed to the outside world, i.e., the entry URL. Vulnerability scanning tools start scanning from the starting URL.

[0048] The protocol switching count can be understood as the number of times the protocol used to send a request to the server is switched.

[0049] During vulnerability detection, the current URL can be selected sequentially from the URL queue according to the first-in-first-out principle of the queue.

[0050] Optionally, the current URL can be the starting URL or the redirect URL corresponding to the page after a page redirect occurs.

[0051] It's important to understand that the redirect URL might not be the actual URL that handles the request (the target URL). That is, after a request is sent to the server via the redirect URL, the web application might redirect again. Therefore, the redirect URL corresponding to the redirected page can be written back into the URL queue, waiting to be selected as the new current URL and requested from the server, until the actual URL is determined. Thus, the obtained current URL could be either the starting URL or the redirect URL corresponding to the redirected page.

[0052] S120: Send a request to the server using the current URL, and upon receiving the response data returned by the server, determine the target URL based on the page redirection behavior of the web application.

[0053] The response data can be understood as the data that the server responds to in response to a request initiated through the current URL.

[0054] Optionally, response data may include at least one of status codes, response headers, response bodies, page titles, forms, internal links, and static resources.

[0055] In this context, "page redirection" can be understood as whether a page redirection occurs in the web application.

[0056] The target URL can be understood as the service address that actually processes the request, i.e., the real URL.

[0057] In some embodiments, a Hypertext Transfer Protocol (HTTP) GET request can be sent to the server via the current URL to request a specified resource.

[0058] In some embodiments, the waiting time for the server to return response data can be set according to a single request waiting time.

[0059] If no response data is received from the server after the single request waiting time has elapsed, the request will be marked as an abnormal request and the current URL will be abandoned, meaning no further processing will be performed on the current URL.

[0060] If the server's response data is received before the single request waiting time has expired, the target URL is determined based on the page redirection behavior of the web application.

[0061] S130, Based on the response data, evaluate the validity of the target URL and obtain the validity evaluation result.

[0062] Among them, validity evaluation can be understood as the process of determining whether a URL conforms to standard specifications and whether the resource can be accessed normally at present.

[0063] The validity evaluation result can be understood as the result of evaluating the validity of the target URL.

[0064] In some embodiments, the validity of the target URL can be evaluated based on at least one piece of data in the response data.

[0065] In some embodiments, the validity of the target URL can be evaluated for each type of data in the response data to obtain a sub-evaluation result under that data, and then the validity evaluation result can be obtained based on the sub-evaluation result under each type of data.

[0066] In some embodiments, the validity of the target URL can be evaluated for each type of data in the response data to obtain a sub-score for that data, and then a validity score can be obtained based on the sub-score for each type of data.

[0067] S140, if the validity evaluation result meets the preset evaluation requirements, obtain the target vulnerability verification script that matches the target URL.

[0068] In some embodiments, when the effectiveness evaluation result is an effectiveness score, the preset evaluation requirement can be a preset scoring threshold. The preset scoring threshold can be set or adjusted by technical personnel based on needs or experience, or determined repeatedly through numerous experiments; this application does not impose any limitations on this.

[0069] Among them, the Proof of Concept (POC) script can be understood as a vulnerability verification script used to perform vulnerability detection on the target URL.

[0070] In some embodiments, the target CMS characteristics of the content management system (CMS) corresponding to the target URL are obtained; and the target vulnerability verification script that matches the target CMS characteristics is selected from the candidate vulnerability verification scripts corresponding to each preset CMS characteristic.

[0071] Among them, CMS features can be understood as CMS fingerprint information.

[0072] The target CMS characteristics include at least one of the target CMS type and the target version.

[0073] In some embodiments, each candidate vulnerability verification script is labeled with the applicable CMS type and applicable version.

[0074] In some embodiments, a first vulnerability verification script with the same CMS type as the target CMS type can be selected from each candidate vulnerability verification script, and a second vulnerability verification script with the same applicable version as the target version can be selected from each candidate vulnerability verification script. Then, the vulnerability verification script obtained from the intersection of the first vulnerability verification script and the second vulnerability verification script can be used as the target vulnerability verification script that matches the characteristics of the target CMS.

[0075] In some embodiments, where the target CMS features include a target version prefix, a second vulnerability verification script with the same version prefix as the target version can be selected from among the candidate vulnerability verification scripts.

[0076] In some embodiments, if no target vulnerability verification script matching the characteristics of the target CMS is selected, a vulnerability verification script marked as highly generalizable can be used as the target vulnerability verification script instead of using the full set of vulnerability verification scripts for vulnerability detection.

[0077] S150 uses a target vulnerability verification script to perform vulnerability detection on the target URL.

[0078] In some embodiments, a target vulnerability verification script can be used to access the real business page corresponding to the target URL based on a pre-provided authentication cookie, and to perform vulnerability detection on the target URL.

[0079] In some embodiments, the target vulnerability verification script can be concurrently scanned according to the number of concurrent threads to perform vulnerability detection on the target URL and obtain vulnerability detection results.

[0080] The aforementioned vulnerability detection method and apparatus, based on the page redirection situation of web applications, can determine the real URL of the actual request being processed, i.e., the target URL, in front-end driven redirection scenarios. Then, based on the response data, the validity of the target URL is evaluated, thereby realizing vulnerability detection on real and valid target URLs and improving the effectiveness of vulnerability detection.

[0081] Based on the technical solutions of the above embodiments, this application also provides another optional embodiment. In this optional embodiment, the step of determining the target URL in S120 is refined, specifically including:

[0082] 1) When a web application redirects, determine the target URL based on the response data.

[0083] It is understandable that when a web application redirects, the response data includes the address data of the redirected page, so the target URL can be determined based on the response data.

[0084] 2) If the web application does not redirect, use the current URL as the target URL.

[0085] It is understandable that, in the absence of page redirection in a web application, the current URL is the actual service address that handles the request, and therefore, the current URL can be used as the target URL.

[0086] Based on the technical solutions of the above embodiments, this application also provides another optional embodiment, in which the step of determining the target URL in 1) is refined.

[0087] See Figure 2 The steps for determining the target URL shown include:

[0088] S210, Determine the target redirection rule for the web application to redirect the page.

[0089] The redirection rules can be understood as the rules that cause a page to redirect.

[0090] In some embodiments, the target redirection rule for a web application to redirect can be determined based on the response body included in the response data.

[0091] In practice, the response body included in the response data can be matched with each preset jump rule using regular expressions, and the matched preset jump rule can be used as the target jump rule.

[0092] The preset redirection rules may include at least one of the following: meta refresh, top.location, self.location, window.location.href, window.open, and static string concatenation redirection.

[0093] Among them, static string concatenation redirection refers to the redirection rule in front-end hard-coding that forms a complete URL by concatenating static strings with dynamic variables.

[0094] S220: Determine the redirect URL corresponding to the redirect page based on the target redirect rules and response data.

[0095] In some embodiments, the process of determining the redirect URL differs depending on the target redirection rule.

[0096] In some embodiments, when the target redirection rule is a static string concatenation-based redirection, the static string contained in the response data is restored to obtain the redirection path. Based on the redirection path, the redirection URL corresponding to the redirected page is determined.

[0097] When the target redirection rule is a static string concatenation redirection, the response body includes a static string. By restoring the static string, the redirection path can be obtained.

[0098] In some embodiments, the redirect path can be used as the redirect URL corresponding to the redirect page.

[0099] In some embodiments, when the target redirection rule is the first redirection rule, the reference URL included in the response data is extracted; the reference URL is used as the redirection URL corresponding to the redirection page.

[0100] The first jump rule is any jump rule other than static string concatenation jump.

[0101] When the target redirection rule is the first redirection rule, the target address literal can be extracted from the response body included in the response data to obtain the reference URL.

[0102] For example, if the first redirect rule is `meta refresh`, the "url=" field in the response body can be extracted using regular expressions to obtain the target URL literal. If the first redirect rule is `location`, `window.open`, or similar rules, the string parameters in the response body can be extracted to obtain the target URL literal, and thus the reference URL.

[0103] S230, Determine the target URL based on the redirect URL.

[0104] As mentioned earlier, the redirect URL may not be the target URL for actually processing the request. Therefore, in some embodiments, the redirect URL can be used as the new current URL, and the process can be repeated to send a request to the server through the current URL, and upon receiving the response data returned by the server, determine the target URL based on the page redirection situation of the web application, until the target URL is obtained.

[0105] In some embodiments, the redirect URL corresponding to the redirected page can be written back to the URL queue, waiting to be selected as the new current URL. If the redirect URL is selected as the new current URL, a request is sent to the server using the new current URL until the target URL is determined.

[0106] In some embodiments, after obtaining the redirect URL, the redirect URL can be normalized to obtain a normalized URL; the target URL is then determined based on the normalized URL.

[0107] The normalization process may include at least one of the following: performing relative path resolution on the redirect URL, representing the protocol name and / or domain name in the redirect URL with lowercase letters, removing fragment content from the redirect URL, removing the default port from the redirect URL, and ordering the query parameters in the redirect URL.

[0108] In some embodiments, if the normalized URL meets the preset deduplication conditions, the normalized URL is discarded; if the normalized URL does not meet the preset deduplication conditions, the target URL is determined based on the normalized URL.

[0109] The preset deduplication conditions may include at least one of the following: existing in the set of visited and deduplicated URLs, the number of redirects exceeding the preset number of redirects, and the domain name exceeding the range of the same host or the same registered main domain.

[0110] In some embodiments, the obtained canonicalized URL can be written back to the URL queue, waiting to be selected as the new current URL. If the canonicalized URL is selected as the new current URL, a request is sent to the server using the new current URL until the target URL is determined.

[0111] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment, in which the step of evaluating the validity of the target URL in S130 is described in detail.

[0112] See Figure 3 The steps shown for evaluating the validity of a target URL include:

[0113] S310, determine the first evaluation result corresponding to the target URL based on the status code in the response data and the preset status code.

[0114] The preset status codes may include at least one of the following: 200, 401, and 403.

[0115] The default status code 200 indicates a successful request. The default status code 401 indicates that the user is not logged in or the login failed. The default status code 403 indicates that the user is logged in but does not have permission to access the resource.

[0116] In some embodiments, the first evaluation result corresponding to the target URL can be determined based on whether the status code in the response data is a preset status code.

[0117] Specifically, the first rating corresponding to the target URL can be determined based on whether the status code is a preset status code.

[0118] For example, if the status code in the response data is a preset status code, then the first score corresponding to the target URL is determined to be 1; if the status code in the response data is not a preset status code, then the first score corresponding to the target URL is determined to be 0.

[0119] S320. Based on the presence of business-oriented links within the response data, determine the second evaluation result corresponding to the target URL.

[0120] The business-specific nature of internal links can be understood as internal links pointing to specific business pages.

[0121] In some embodiments, a second evaluation result corresponding to the target URL can be determined based on whether the internal links in the response data have business relevance.

[0122] Specifically, the second score corresponding to the target URL can be determined based on whether the internal links in the response data have business relevance.

[0123] For example, if the internal links in the response data have business relevance, the second score corresponding to the target URL is determined to be 1; if the internal links in the response data do not have business relevance, the second score corresponding to the target URL is determined to be 0.

[0124] S330, Based on the existence of static resources in the response data, determine the third evaluation result corresponding to the target URL.

[0125] In some embodiments, a third evaluation result corresponding to the target URL can be determined based on whether static resources exist in the response data.

[0126] Specifically, the third rating corresponding to the target URL can be determined based on whether static resources are present in the response data.

[0127] For example, if static resources exist in the response data, the third score corresponding to the target URL is determined to be 1; if static resources do not exist in the response data, the third score corresponding to the target URL is determined to be 0.

[0128] S340, Based on the presence of the form in the response data, determine the fourth evaluation result corresponding to the target URL.

[0129] In some embodiments, the fourth evaluation result corresponding to the target URL can be determined based on whether a form exists in the response data.

[0130] Specifically, the fourth rating corresponding to the target URL can be determined based on whether a form exists in the response data.

[0131] For example, if a form is present in the response data, the fourth rating corresponding to the target URL is determined to be 1; if no form is present in the response data, the fourth rating corresponding to the target URL is determined to be 0.

[0132] S350. Based on the consistency of the page title and structure summary in the response data after multiple consecutive requests, determine the fifth evaluation result corresponding to the target URL.

[0133] In some embodiments, a fifth evaluation result corresponding to the target URL can be determined based on whether the page title and structural summary in the response data are consistent after multiple consecutive requests.

[0134] Optionally, the fifth score corresponding to the target URL can be determined based on whether the page title and structure summary in the response data are consistent after multiple consecutive requests.

[0135] For example, if the page title and structure summary in the response data are consistent after multiple consecutive requests, the fifth score corresponding to the target URL is determined to be 1; if the page title and structure summary in the response data are inconsistent after multiple consecutive requests, the fifth score corresponding to the target URL is determined to be 0.

[0136] Alternatively, a fifth score can be determined based on the similarity of the structural summary in the response data after multiple consecutive requests.

[0137] For example, the higher the similarity of the structural summary after multiple consecutive requests, the higher the fifth score corresponding to the target URL can be. The value of the fifth score ranges from [0,1].

[0138] S360 determines the sixth evaluation result corresponding to the target URL based on the structure of the navigation area in the page corresponding to the target URL.

[0139] In some embodiments, the sixth evaluation result corresponding to the target URL can be determined based on at least one of the following: whether there is a navigation node in the page corresponding to the target URL, the number of valid links on the same site, and the diversity of link paths.

[0140] Specifically, the sixth score for a target URL can be determined based on at least one of the following: whether there is a navigation node on the page corresponding to the target URL, the number of valid links on the same site, and the diversity of link paths.

[0141] For example, if the page corresponding to the target URL does not contain a navigation node, the number of valid links on the same site is less than the preset number, and the diversity of link paths does not meet the preset diversity requirement, the sixth score corresponding to the target URL is determined to be 0; if the page corresponding to the target URL contains a navigation node, the number of valid links on the same site is not less than the preset number, and the diversity of link paths meets the preset diversity requirement, the sixth score corresponding to the target URL is determined to be 1; otherwise, the sixth score corresponding to the target URL can be determined to be 0.5.

[0142] S370, determine the effectiveness evaluation result based on at least one of the first evaluation result, the second evaluation result, the third evaluation result, the fourth evaluation result, the fifth evaluation result, and the sixth evaluation result.

[0143] In some embodiments, the effectiveness evaluation result can be determined based on the first evaluation result, the second evaluation result, the third evaluation result, the fourth evaluation result, the fifth evaluation result, and the sixth evaluation result.

[0144] In some embodiments, the first evaluation result, the second evaluation result, the third evaluation result, the fourth evaluation result, the fifth evaluation result, and the sixth evaluation result can be weighted and summed to obtain the effectiveness evaluation result.

[0145] Specifically, the first, second, third, fourth, fifth, and sixth scores can be weighted and summed to obtain the validity score.

[0146] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment in which the vulnerability detection method is described in detail.

[0147] See Figure 4 The vulnerability detection method shown includes, after sending a request to the server via the current URL, the following:

[0148] S410, if the request process meets the preset protocol switching conditions, switch the first protocol used in the current request to the second protocol.

[0149] The preset protocol switching conditions include at least one of the following: the server rejects the request multiple times, the Transport Layer Security (TLS) handshake between the server fails, the server returns an explicit protocol mismatch message, and the server returns a status code pointing to another protocol address under the current URL.

[0150] The current request can be understood as a request sent to the server via the current URL.

[0151] In this context, "the server rejects the request multiple times" can be understood as the server rejecting the request multiple times in a row.

[0152] Where the first protocol used in the current request is HTTP, the second protocol can be Hypertext Transfer Protocol Secure (HTTPS); where the first protocol used in the current request is HTTPS, the second protocol can be HTTP.

[0153] S420, based on the second protocol, re-initiates a request to the server using the current URL.

[0154] In some embodiments, if the number of protocol switching attempts does not exceed a preset number of switching attempts, a new request is sent to the server using the current URL based on the second protocol. If the number of protocol switching attempts exceeds the preset number of switching attempts, the current URL is discarded.

[0155] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment in which the vulnerability detection method is described in detail.

[0156] See Figure 5 The vulnerability detection methods shown include:

[0157] S501, Select the current URL to be detected from the URL queue corresponding to the web application.

[0158] S502, based on the first protocol, initiates a request to the server using the current URL.

[0159] S503, if the request process meets the preset protocol switching conditions and the number of switching does not exceed the preset number of switching, the first protocol is switched to the second protocol.

[0160] The preset protocol switching conditions include at least one of the following: the server rejects the request multiple times, the TLS handshake between the server and the server fails, the server returns an explicit protocol mismatch message, and the server returns a status code pointing to another protocol address under the current URL.

[0161] S504, based on the second protocol, resends a request to the server using the current URL.

[0162] S505, upon receiving response data from the server and a page redirection occurs in the web application, determine the target redirection rule for the page redirection.

[0163] S506, when the target redirection rule is static string concatenation redirection, restore the static string contained in the response data to obtain the redirection path.

[0164] S507, determine the redirect URL corresponding to the redirect page based on the redirect path.

[0165] S508, when the target redirection rule is the first redirection rule, extract the reference URL included in the response data.

[0166] The first jump rule is any jump rule other than static string concatenation jump.

[0167] S509, use the reference URL as the redirect URL for the redirected page.

[0168] S510 performs normalization processing on the redirect URL to obtain a normalized URL.

[0169] S511, if the normalized URL does not meet the preset deduplication conditions, write the normalized URL back to the URL queue and return to step S501.

[0170] S512: If the response data returned by the server is received and the web application does not redirect, the current URL is used as the target URL.

[0171] S513. Based on the response data, evaluate the validity of the target URL and obtain the validity evaluation result.

[0172] In practice, the following evaluation results are determined: First, based on the status codes and preset status codes in the response data; second, based on the presence of business-oriented links within the response data; third, based on the presence of static resources in the response data; fourth, based on the presence of forms in the response data; fifth, based on the consistency of the page title and structure summary in the response data across multiple consecutive requests; sixth, based on the structure of the navigation area on the page corresponding to the target URL; and finally, based on the first, second, third, fourth, fifth, and sixth evaluation results, a validity evaluation result is determined.

[0173] S514, if the validity evaluation results meet the preset evaluation requirements, obtain the target CMS characteristics of the content management system CMS corresponding to the target URL.

[0174] The target CMS characteristics include the target CMS type and the target version.

[0175] S515: Select the target vulnerability verification script that matches the target CMS feature from the candidate vulnerability verification scripts corresponding to each preset CMS feature.

[0176] S516 utilizes a target vulnerability verification script to perform vulnerability detection on the target URL.

[0177] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0178] This application also provides an optional embodiment in which the vulnerability detection method is described in detail.

[0179] Obtain the starting URL to be tested in the web application, denoted as http: / / portal.example.com. The vulnerability scanning tool sends a request to the server based on http: / / portal.example.com. After receiving the response data returned by the server, the response body is parsed to determine that the redirection rule corresponding to the starting URL is window.location.href=" / oa / ". The redirection address in the response body is extracted to obtain the canonicalized URL, denoted as http: / / portal.example.com / oa / . A new request is sent to the server via http: / / portal.example.com / oa / . After receiving the server's explicit protocol mismatch message "plain HTTP request was sent to HTTPS port", the protocol is switched from HTTP to HTTPS. Based on HTTPS, a new request is sent to the server via http: / / portal.example.com / oa / . After successfully returning to the login page, the canonicalized URL is evaluated for validity based on the received response data to obtain a validity score. If the validity score corresponding to the normalized URL is greater than the preset score threshold, the target CMS fingerprint corresponding to the normalized URL is determined. Based on the candidate vulnerability detection scripts corresponding to each preset CMS fingerprint, the target vulnerability detection script corresponding to the target CMS fingerprint is determined. Based on the target vulnerability detection script, vulnerability detection is performed on the normalized URL.

[0180] Based on the same inventive concept, this application also provides a vulnerability detection system for implementing the vulnerability detection method described above.

[0181] like Figure 6 The vulnerability detection system shown includes: a response data parsing module, a redirection pattern recognition module, an entry point normalization and deduplication module, a protocol adaptive switching module, an entry point validity scoring module, a valid entry point confirmation module, a feature extraction module, a POC targeted pruning module, a concurrent scanning engine, and a result aggregation and output module.

[0182] The response data parsing module is used to send a request to the server through the current URL, obtain the response data returned by the server, and provide raw data for subsequent URL extraction, validity scoring, and matching vulnerability detection scripts.

[0183] The redirection pattern recognition module is used to match redirection rules in the response body of the response data. When any redirection rule is matched, it outputs the redirection address literal, the redirection rule identifier, and the source page address.

[0184] Entry point normalization and deduplication module: It parses the jump target extracted by the module redirection pattern recognition module and the current page address, and uniformly processes relative paths, protocol relative addresses and absolute addresses; at the same time, it performs URL normalization and writes the normalized URL into the URL queue and the visited set to avoid the same URL being detected repeatedly due to different forms.

[0185] The protocol adaptive switching module monitors request anomalies and response characteristics. A protocol switch is triggered when the current URL experiences a connection rejection, TLS handshake failure, a returned explicit protocol mismatch message, or a server-returned status code pointing to another protocol address under the current URL. The number of switches for the same URL cannot exceed a preset number. If the number of switches for the same URL exceeds the preset number, processing for that URL is terminated to avoid an infinite loop in protocol mismatch scenarios.

[0186] The entry validity scoring module is used to calculate a validity score for a given target URL.

[0187] The valid entry point confirmation module compares the validity score of the entry point validity scoring module with a preset scoring threshold. When the target URL's validity score is greater than the preset scoring threshold, the number of redirects is not greater than the preset number of redirects, the number of protocol switches does not exceed the preset number of switches, and the domain name range belongs to the same host or the same registered main domain, the target URL is used as the actual business entry point for vulnerability detection. If multiple target URLs exist on the same site, the target URL with the highest score and the highest page stability is prioritized as the actual business entry point for subsequent scheduling.

[0188] The feature extraction module is used to extract features such as web server type, development language, middleware version, and CMS fingerprint from target URLs confirmed through valid entry points, providing context for vulnerability detection script matching.

[0189] The POC targeted tailoring module is used to filter target vulnerability detection scripts from candidate vulnerability detection scripts corresponding to various preset features based on the characteristics of the target URL, selecting those that match the target CMS signature and target version. For target URLs where no target vulnerability detection script is successfully selected, a highly generalized vulnerability detection script is used to perform vulnerability detection.

[0190] The concurrent scanning engine is used to schedule and execute target vulnerability detection scripts in a multi-threaded / multi-process manner, and supports custom concurrency levels, timeout periods, and abnormal circuit breaker policies.

[0191] The results aggregation output module is used to sort and deduplicate vulnerability detection reports based on the execution results of each target vulnerability detection script, according to vulnerability level, score of a general vulnerability scoring system, and exploitability, and generate a structured vulnerability detection report.

[0192] Based on the same inventive concept, this application also provides a vulnerability detection device for implementing the vulnerability detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more vulnerability detection device embodiments provided below can be found in the limitations of the vulnerability detection method described above, and will not be repeated here.

[0193] In one embodiment, such as Figure 7 As shown, a vulnerability detection device is provided, including: a first acquisition module 710, a determination module 720, an evaluation module 730, a second acquisition module 740, and a detection module 750. Among them,

[0194] The first acquisition module 710 is used to acquire the current URL to be detected in the web application;

[0195] The determination module 720 is used to initiate a request to the server using the current URL, and, upon receiving the response data returned by the server, determine the target URL based on the page redirection behavior of the web application.

[0196] Evaluation module 730 is used to evaluate the validity of the target URL based on the response data and obtain the validity evaluation result;

[0197] The second acquisition module 740 is used to acquire the target vulnerability verification script that matches the target URL when the validity evaluation result meets the preset evaluation requirements.

[0198] The detection module 750 is used to perform vulnerability detection on the target URL using the target vulnerability verification script.

[0199] In one embodiment, the determining module 720 is specifically used to: determine the target URL based on the response data when a page redirect occurs in the web application; and use the current URL as the target URL when no page redirect occurs in the web application.

[0200] In one embodiment, the determining module 720 is specifically used to: determine the target redirection rule for the page redirection in the web application; determine the redirection URL corresponding to the redirection page based on the target redirection rule and response data; and determine the target URL based on the redirection URL.

[0201] In one embodiment, the determining module 720 is specifically used to: restore the static string contained in the response data to obtain the jump path when the target jump rule is a static string concatenation jump; and determine the jump URL corresponding to the jump page based on the jump path.

[0202] In one embodiment, the determining module 720 is specifically used to: extract the reference URL included in the response data when the target redirection rule is the first redirection rule; and use the reference URL as the redirection URL corresponding to the redirection page; wherein the first redirection rule is a redirection rule other than static string concatenation redirection.

[0203] In one embodiment, the determining module 720 is specifically used to: normalize the redirect URL to obtain a normalized URL; and determine the target URL based on the normalized URL.

[0204] In one embodiment, the determining module 720 is specifically used to: take the redirect URL as the new current URL, return to execute the steps of initiating a request to the server through the current URL, and, upon receiving the response data returned by the server, determine the target URL based on the page redirection situation of the web application, until the target URL is obtained.

[0205] In one embodiment, the evaluation module 730 is specifically configured to: determine a first evaluation result corresponding to the target URL based on the status code and a preset status code in the response data; determine a second evaluation result corresponding to the target URL based on the existence of business orientation of internal links in the response data; determine a third evaluation result corresponding to the target URL based on the existence of static resources in the response data; determine a fourth evaluation result corresponding to the target URL based on the existence of forms in the response data; determine a fifth evaluation result corresponding to the target URL based on the consistency of the page title and structure summary in the response data after multiple consecutive requests; determine a sixth evaluation result corresponding to the target URL based on the structure of the navigation area in the page corresponding to the target URL; and determine a validity evaluation result based on at least one of the first, second, third, fourth, fifth, and sixth evaluation results.

[0206] In one embodiment, after a request is initiated to the server via the current URL, the vulnerability detection device further includes: a switching module, configured to switch the first protocol used in the current request to a second protocol if the request process meets preset protocol switching conditions; wherein the preset protocol switching conditions include at least one of the following: the server rejects the request multiple times, the Transport Layer Security (TLS) handshake between the server fails, the server returns an explicit protocol mismatch message, and the server returns a status code pointing to another protocol address under the current URL; and a sending module, configured to re-initiate a request to the server via the current URL based on the second protocol.

[0207] Each module in the aforementioned vulnerability detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0208] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores URL queues, server response data, target URLs, validity evaluation results, target vulnerability verification scripts, and other related data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a vulnerability detection method.

[0209] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0210] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the processing logic in the computer program to implement the steps of the vulnerability detection method provided in any of the above embodiments.

[0211] In one embodiment, when the processor executes the processing logic in the computer program, it also implements the steps of the vulnerability detection method provided in any of the above embodiments.

[0212] In one embodiment, a computer-readable storage medium or computer program product is provided, on which a computer program is stored, wherein the processing logic in the computer program, when executed by a processor, implements the steps of the vulnerability detection method provided in any of the above embodiments.

[0213] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0214] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0215] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A vulnerability detection method, characterized in that, include: Retrieve the current Uniform Resource Locator (URL) to be detected in the web application; A request is sent to the server using the current URL, and upon receiving the response data returned by the server, the target URL is determined based on the page redirection behavior of the web application. Based on the response data, the target URL is evaluated for validity to obtain a validity evaluation result; wherein, the validity evaluation result is determined at least based on a fifth evaluation result and a sixth evaluation result, the fifth evaluation result is determined based on the consistency of the page title and structure summary in the response data after multiple consecutive requests, and the sixth evaluation result is determined based on the structure of the navigation area in the page corresponding to the target URL; If the effectiveness evaluation result meets the preset evaluation requirements, obtain the target vulnerability verification script that matches the target URL; The target vulnerability verification script is used to perform vulnerability detection on the target URL.

2. The method according to claim 1, characterized in that, Determining the target URL based on the page redirection behavior of the web application includes: When a page redirect occurs in the web application, the target URL is determined based on the response data; If the web application does not redirect, the current URL will be used as the target URL.

3. The method according to claim 2, characterized in that, Determining the target URL based on the response data includes: Determine the target redirection rule for the page redirection in the web application; Based on the target redirection rules and the response data, determine the redirection URL corresponding to the redirection page; Determine the target URL based on the redirect URL.

4. The method according to claim 3, characterized in that, The step of determining the redirect URL corresponding to the redirect page based on the target redirection rule and the response data includes: If the target redirection rule is a static string concatenation redirection, the static string contained in the response data is restored to obtain the redirection path; Based on the redirection path, determine the redirection URL corresponding to the redirection page.

5. The method according to claim 3, characterized in that, The step of determining the redirect URL corresponding to the redirect page based on the target redirection rule and the response data includes: If the target redirection rule is the first redirection rule, extract the reference URL included in the response data; The reference URL is used as the redirect URL for the redirect page; wherein, the first redirect rule is a redirect rule other than static string concatenation redirect.

6. The method according to claim 3, characterized in that, Determining the target URL based on the redirect URL includes: The redirect URL is normalized to obtain a normalized URL; The target URL is determined based on the normalized URL.

7. The method according to claim 3, characterized in that, Determining the target URL based on the redirect URL includes: The redirect URL is used as the new current URL, and the process of sending a request to the server through the current URL and determining the target URL based on the page redirection of the web application is repeated until the target URL is obtained.

8. The method according to any one of claims 1-7, characterized in that, The step of evaluating the validity of the target URL based on the response data to obtain a validity evaluation result includes: Based on the status code and preset status code in the response data, determine the first evaluation result corresponding to the target URL; Based on the presence of business-oriented links within the response data, a second evaluation result corresponding to the target URL is determined; Based on the presence of static resources in the response data, a third evaluation result corresponding to the target URL is determined; Based on the presence of forms in the response data, determine the fourth evaluation result corresponding to the target URL; Based on the consistency of the page title and structure summary in the response data after multiple consecutive requests, the fifth evaluation result corresponding to the target URL is determined; The sixth evaluation result corresponding to the target URL is determined based on the structure of the navigation area in the page corresponding to the target URL; The effectiveness evaluation result is determined based on at least one of the first evaluation result, the second evaluation result, the third evaluation result, the fourth evaluation result, the fifth evaluation result, and the sixth evaluation result.

9. The method according to any one of claims 1-7, characterized in that, After making a request to the server via the current URL, the method includes: If the request process meets the preset protocol switching conditions, the first protocol used in the current request will be switched to the second protocol; wherein, the preset protocol switching conditions include at least one of the following: the server rejects the request multiple times, the Transport Layer Security (TLS) handshake between the server fails, the server returns an explicit protocol mismatch message, and the status code returned by the server points to another protocol address under the current URL; Based on the second protocol, a new request is sent to the server using the current URL.

10. A vulnerability detection device, characterized in that, include: The first acquisition module is used to acquire the current URL to be detected in the web application; The determination module is used to initiate a request to the server through the current URL, and upon receiving the response data returned by the server, determine the target URL based on the page redirection situation of the web application; An evaluation module is used to evaluate the validity of the target URL based on the response data and obtain a validity evaluation result; wherein the validity evaluation result is determined at least based on a fifth evaluation result and a sixth evaluation result, the fifth evaluation result is determined based on the consistency of the page title and structure summary in the response data after multiple consecutive requests, and the sixth evaluation result is determined based on the structure of the navigation area in the page corresponding to the target URL; The second acquisition module is used to acquire a target vulnerability verification script that matches the target URL when the validity evaluation result meets the preset evaluation requirements. The detection module is used to perform vulnerability detection on the target URL using the target vulnerability verification script.

Images