Webpage information extraction and vulnerability detection method and system based on JavaScript intelligent analysis

Abstract

The invention relates to the technical field of network security, and discloses a webpage information extraction and vulnerability detection method and system based on JavaScript intelligent analysis. The method comprises the following steps: in a webpage loading stage, collecting static resources and dynamic request data, and generating enhanced interface entries; constructing a global evidence graph based on the enhanced interface entries, and performing vulnerability detection based on the global evidence graph; and acquiring node detection confidence according to a detection result, dynamically updating node and edge weights of the global evidence graph to form a confidence data set, and realizing adaptive iteration of a detection process. According to the method, a dynamic feedback and confidence coefficient updating mechanism is introduced, so that the accuracy and robustness of a vulnerability detection result are improved, the problem that a detection rule is static and is difficult to reflect a global dependency relationship between interfaces in a traditional method is solved, and continuous optimization and intelligent evolution of a vulnerability detection process are realized.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and more specifically, to a method and system for extracting web page information and detecting vulnerabilities based on JavaScript intelligent analysis. Background Technology

[0002] In today's digital age, web applications and cloud services are developing rapidly, leading to an explosive growth in the number of API interfaces under front-end / back-end separation architectures. In modern web application security testing scenarios, how to efficiently detect and analyze vulnerabilities in these interfaces has become a critical issue that urgently needs to be addressed.

[0003] In the prior art, Chinese patent application CN104618178A discloses an online website vulnerability assessment device. This device receives website vulnerability information from vulnerability detection webpages, determines the vulnerability risk assessment level based on vulnerability classification and website traffic, and sends alerts to the website, improving the accuracy of vulnerability risk assessment and the timeliness of alerts. However, this prior art does not address the problem of deep business logic vulnerabilities being difficult to trigger due to the lack of interface call context in dynamic API interfaces under a front-end / back-end separation architecture. In practical applications, it is necessary to receive discovered vulnerability information in advance as input, making it difficult to achieve automated discovery and accurate detection of deep business logic vulnerabilities.

[0004] Chinese patent application CN102663296B discloses an intelligent detection method for malicious JavaScript code on web pages. It primarily focuses on classifying JavaScript scripts using N-gram and KNN machine learning algorithms to identify malicious code and ensure the security of script execution on web application clients. However, this existing technology does not consider the security of business logic at the API call level. In complex web applications, many high-risk vulnerabilities are not caused by the malice of the script itself, but rather by legitimate APIs being called in unexpected ways, such as unauthorized calls, parameter tampering, or bypassing the call order. These logical-level security flaws cannot be detected by this method.

[0005] However, the current process of web application vulnerability detection faces severe challenges when dealing with complex interface calls, including a lack of detection priority, broken interface call context, and a technical disconnect between static rules and isolated detection results. Summary of the Invention

[0006] This invention is applicable to vulnerability detection scenarios with dense interfaces in web application security testing, such as API security auditing of large e-commerce platforms, penetration testing of financial system interfaces, and security assessment of SaaS applications. In these scenarios, the front-end and back-end separation architecture generates massive dynamic API requests in real time, and there are complex call dependencies and data flows between interfaces. Intelligent vulnerability detection is needed to identify unauthorized access, discover business logic vulnerabilities, and provide system-level security protection.

[0007] To overcome the aforementioned shortcomings of existing technologies, this invention provides a method and system for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis. It constructs enhanced interface entries by fusing dynamic request capture with static code parsing (headless browser monitoring and AST parsing), integrating static interface features, dynamic call behavior, and contextual semantic relationships to form a three-dimensional detection network covering the interface layer, call layer, and evidence layer. This solution achieves efficient collaboration between priority detection based on a global evidence graph and dynamic optimization based on confidence feedback, solving the problems of false positives and false negatives caused by blind traversal and lack of context in traditional scanners. It significantly improves the accuracy and efficiency of vulnerability discovery in complex business scenarios, providing intelligent detection support for web application security protection, API security governance, and penetration testing automation, breaking through the technical bottlenecks of static rules and isolated detection results in traditional security tools.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis, comprising: Load the target webpage and obtain all static resources and dynamic request data during the loading process. Based on all static resources and dynamic request data during the webpage loading process, generate enhanced interface entries; Based on the generated enhanced interface entries, vulnerability detection and result storage are implemented.

[0009] Furthermore, the method for loading the target webpage includes: Access the target webpage using a headless browser, obtain all static resources of the target webpage, and listen for dynamic requests initiated by the target webpage. Mark the request parameters and response data in all dynamic requests initiated by the target web pages being monitored as dynamic request data and cache the data; The obtained static resources and dynamic request data are stored in a unified manner to generate a webpage loading dataset.

[0010] Furthermore, the method for generating enhanced interface entries includes: Perform abstract syntax tree parsing on static JavaScript files, extract the hard-coded API paths, request methods, and parameter structures, and mark them as static parsing results; Based on the obtained static resources, establish a mapping relationship between script execution logic and actual requests; Analyze dynamic request data to identify concatenated or variable-structured APIs. Based on context analysis, filter out non-business interfaces for the identified concatenated or variable-structured APIs. Based on the filtered concatenated or variable-structured APIs, the path and parameter structure of the concatenated or variable-structured APIs are restored along the established mapping relationship between the script execution logic and the actual request. The restored path and parameter structure of the concatenated or variable-structured APIs are then marked as dynamic request characteristics. The static parsing results and dynamic request characteristics are integrated to generate enhanced interface entries.

[0011] Furthermore, the method for extracting the sensitive field information includes: By summarizing the features of dynamic request data, a rule base containing field patterns and matching expressions is constructed. Based on the rule base, dynamic request data is matched and analyzed to obtain the corresponding sensitive field information.

[0012] Furthermore, the method for implementing vulnerability detection and result storage includes: Construct a global evidence graph based on enhanced interface entries; Vulnerability detection is performed based on a global evidence graph to obtain vulnerability detection results; The global evidence graph is updated based on the vulnerability detection results to generate a confidence dataset. The vulnerability detection results, updated global evidence graph, confidence dataset, and sensitive field information are stored in a structured manner and provided for secondary retrieval.

[0013] Furthermore, the method for constructing a global evidence graph based on enhanced interface entries includes: Based on the enhanced interface entries, extract the interface, interface path, and corresponding request parameters and response fields; Taking interface i and interface j as examples, we construct a global evidence graph by taking the enhanced interface entries interface i and interface j, their request parameters, and response fields as nodes, and taking the interface from interface i to interface j as edges. Based on the mapping relationship between script execution logic and actual requests, dynamic request data is analyzed to obtain the corresponding interface call sequence logs and calculate the call frequency components and data dependency strength from interface i to interface j. The interface response fields are matched based on a custom rule base to obtain a custom rule score; Calculate the edge weight from interface i to interface j based on call frequency component, data dependency strength, and custom rule score; Based on the edge weights of the global evidence graph, calculate the comprehensive node score of the interface nodes; and generate a vulnerability detection sequence by sorting the comprehensive node scores from largest to smallest.

[0014] Furthermore, the method for vulnerability detection based on a global evidence graph includes: Based on the frequency of API calls and the strength of data dependencies, a contextual semantic rule base is automatically generated; the basic detection rule base, the contextual semantic rule base, and the custom rule base are integrated to generate a comprehensive rule base. Obtain the interface information corresponding to the node in the vulnerability detection sequence, make a real request to the interface and monitor it, and obtain the monitored interface response fields; Based on the comprehensive rule base, data matching analysis is performed on the monitored interface response fields to obtain the number of matches between the interface response fields and the rule base, the number of abnormal responses to the interface response fields, and whether the interface call order matches the global evidence. Figure 1 To; Vulnerability detection is performed on each node in the global evidence graph according to the vulnerability detection sequence to obtain vulnerability detection results.

[0015] Furthermore, the method for updating the global evidence graph based on vulnerability detection results includes: Calculate the node detection confidence level based on the vulnerability detection results; A second vulnerability detection is performed based on the global evidence graph. The results of the second vulnerability detection are obtained, and the detection confidence of the current node is calculated based on the results of the second vulnerability detection. The node detection confidence is updated based on the node detection confidence and the current node detection confidence. Based on the updated node detection confidence, the edge weights in the global evidence graph are updated; Based on the updated edge weights, calculate the new node comprehensive score and generate a new vulnerability detection sequence; perform a new round of vulnerability detection based on the updated global evidence graph; record the node detection confidence and the updated node detection confidence to generate a confidence dataset.

[0016] Furthermore, the method for structuring and storing vulnerability detection results and providing secondary retrieval includes: The vulnerability detection results, updated global evidence graph, confidence dataset, and sensitive field information are stored in the Elasticsearch database in the form of structured documents. They are generated into five types of document collections according to their functions, including: interface node index, interface edge index, detection result index, confidence dataset index, and sensitive field index. Provide secondary retrieval based on the generated document collection.

[0017] A webpage information extraction and vulnerability detection system based on JavaScript intelligent analysis is provided to implement the aforementioned webpage information extraction and vulnerability detection method based on JavaScript intelligent analysis. The system includes: Webpage Loading and Dynamic Parsing Module: Used to load the target webpage, capture all static resources and dynamic request data during the webpage loading process, and obtain a webpage loading dataset with complete request and response characteristics; API Intelligent Extraction Module: Based on a webpage loading dataset carrying complete request and response characteristics, it obtains static parsing results and dynamic request characteristics to construct enhanced interface entries; Sensitive information extraction module: Based on dynamic request data, it constructs a sensitive field rule base through feature summarization to identify sensitive information in interface responses; Global Evidence Graph Generation Module: Based on enhanced interface entries, constructs a global evidence graph of interface call relationships and data dependencies; Vulnerability detection module: Based on the global evidence graph, it implements priority vulnerability detection and confidence feedback updates based on node comprehensive scoring; The results storage and retrieval module stores vulnerability detection results, updated global evidence graphs, confidence datasets, and sensitive field information in a structured manner and supports secondary retrieval.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention effectively solves the problems of incomplete interface coverage, vulnerability triggering failures due to missing context, and difficulty in discovering deep business logic vulnerabilities in traditional vulnerability scanners by capturing dynamic requests during webpage loading in real time and integrating them with static code parsing to construct enhanced interface entries. This method integrates multi-dimensional information such as static interface features, dynamic call behavior, and contextual semantic relationships to construct a three-dimensional detection network covering the interface layer, call layer, and evidence layer, breaking through the technical separation between static rule detection and dynamic behavior analysis in traditional security tools. Based on the constructed global evidence graph, this invention achieves efficient collaboration between priority vulnerability detection based on node comprehensive scoring and dynamic optimization based on confidence feedback. While ensuring the priority discovery of high-risk interfaces, it also guarantees the adaptive evolution of detection strategies and the accuracy of vulnerability identification. Through multi-round iterative detection and evidence graph update mechanisms, this solution significantly improves the efficiency of vulnerability discovery in complex business scenarios, providing intelligent technical support for network security protection and API security governance. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a method for extracting web page information and detecting vulnerabilities based on JavaScript intelligent analysis in this invention; Figure 2 This is a schematic diagram of the global evidence graph in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the principle of generating vulnerability detection sequences based on a global evidence graph in an embodiment of the present invention; Figure 4 This is a flowchart of a method for updating confidence based on vulnerability detection results in a webpage information extraction and vulnerability detection method based on JavaScript intelligent analysis according to the present invention. Figure 5 This is a functional module diagram of a web page information extraction and vulnerability detection system based on JavaScript intelligent analysis, as described in this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1 Please see Figure 1 As shown, this embodiment provides a method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis, including: Step S1000: Load the target webpage and obtain all static resources and dynamic request data during the loading process of the target webpage; Further, step S1000 includes: Step S1100: Simulate real user access behavior based on headless browser, load target webpage and trigger webpage initialization process; Step S1200: Parse all static resources of the target webpage and listen for dynamic requests initiated by the target webpage; the static resources include the webpage's HTML, CSS, and static JavaScript files; Step S1300: Mark the request parameters and response data in all dynamic requests initiated by the target web pages as dynamic request data and cache the data. Specifically, the purpose of step S1000 is to achieve complete capture and unified modeling of static and dynamic resources throughout the entire webpage loading process through simulation execution in a real browser environment and dynamic network event listening, thereby solving the problems in existing webpage monitoring systems where static analysis cannot identify dynamic requests and network proxy methods cannot trace request context.

[0023] During loading, a browser instance is launched and a virtual user session is created, automatically executing a series of native browser behaviors such as page opening, DOM construction, script parsing and execution, style rendering, and event listening. When the target webpage contains complex behaviors such as lazy loading, redirection, or asynchronous data requests, data extraction is performed only after the page has been fully rendered to ensure that the obtained webpage information is consistent with the actual user's access state.

[0024] Step S1100 simulates real-world access behavior to achieve "behavioral-level webpage collection," effectively solving the problems of traditional crawling methods being unable to load dynamic content or reproduce the execution state of front-end logic. By reproducing browser kernel-level behavior, the integrity and authenticity of page loading are guaranteed, providing a reliable data source for subsequent dynamic request capture, interface dependency identification, and global evidence graph construction.

[0025] After the webpage loads, step S1200 parses all static resources of the target webpage and listens for dynamic requests initiated by the webpage during its execution. These static resources include the webpage's HTML document, CSS, static JavaScript files, images, fonts, etc. By analyzing the DOM tree structure and resource reference paths, a webpage resource dependency index is established, thus forming a static structure description of the page. Simultaneously, to capture dynamic behavior, request interceptors and response listeners are mounted in the browser context to monitor various dynamic requests initiated by the webpage during its execution in real time. For each detected dynamic request, its request method, URL, request headers, request parameters, and corresponding response data are extracted, and its triggering source is marked, such as click events, page initialization, asynchronous loading, etc. For example, when a user accesses a product details page, the webpage initially returns only the HTML frame, while product details, prices, recommendation lists, and other content are obtained through an asynchronous API.

[0026] In practical implementation, to differentiate requests from different stages or modules, timestamps and page context identifiers need to be introduced into the cache structure, enabling rapid identification of request behavior at specific stages during subsequent analysis. The tagging of dynamic request data includes not only semantic recognition of the request itself but also annotation of its response characteristics, such as response data structure, field types, and return code features, providing data information for vulnerability detection. This annotation mechanism allows for rapid acquisition of interface response field information during subsequent rule matching and vulnerability detection, improving detection efficiency and accuracy.

[0027] Step S2000: Based on all static resources and dynamic request data during the webpage loading process, generate enhanced interface entries; Further, step S2000 includes: Step S2100: Perform abstract syntax tree parsing on the static JavaScript file, extract the hard-coded API paths, request methods, and parameter structures, and mark them as static parsing results; During webpage loading, static JavaScript files typically play a crucial role in request construction and business logic control. Traditional webpage API analysis tools can only capture network layer requests and cannot understand API call logic from the script semantic layer. Step S2100 performs abstract syntax tree parsing on the static JavaScript file, extracting potential API definitions and request logic at the source code level, thus establishing a mapping relationship from code semantics to request structure.

[0028] Specifically, the source code of a static JavaScript file is converted into an abstract syntax tree (AST) structure. This AST represents the semantic components of the code in a hierarchical node manner, such as function declarations, variable definitions, object property accesses, string concatenation, and function calls. By traversing the AST nodes, function call statements containing network request behavior are identified, such as `fetch()`, `XMLHttpRequest()`, `axios()`, and `$.ajax()`. For each type of request call, its parameter list is extracted, and further analysis is performed to determine if there are hard-coded URL strings or statically defined request paths. When such hard-coded requests are detected, the request method, such as GET, POST, PUT, and DELETE, and the request parameter structure, including fixed parameters and default values, are parsed simultaneously. For example, if the source code of a static JavaScript file contains the statement: `fetch(" / api / user / info", { method: "GET"});`, the system will automatically identify the API path as ` / api / user / info`, the request method as GET, and mark it as a static parsing result.

[0029] Step S2200: Analyze the dynamic request data to obtain dynamic request characteristics; Further, step S2200 includes: Step S2210: Based on the obtained static resources, establish a mapping relationship between script execution logic and actual requests; Step S2220: Analyze the dynamic request data to identify concatenated or variable-structured APIs. Based on context analysis, filter the identified concatenated or variable-structured APIs for non-business interfaces. Step S2230: Based on the identified concatenated or variable-structured APIs after filtering, restore the concatenated or variable-structured API path and parameter structure along the established mapping relationship between script execution logic and actual requests; mark the restored concatenated or variable-structured API path and parameter structure as dynamic request features; Specifically, the core of this step lies in achieving traceable reconstruction of dynamic APIs through logical mapping and context analysis.

[0030] During webpage loading, numerous API paths in static JavaScript files are dynamically generated through variable concatenation, template strings, or runtime logic. For example: `const base = getBaseURL(); const api = base +' / v1 / ' + user.id + ' / detail';` Static scanning often only extracts string fragments of this concatenation logic, failing to reconstruct the complete API path, leading to incomplete API analysis. Step S2200 establishes a mapping relationship between script execution logic and actual requests, corresponding the runtime real requests to their generation logic within the script. Specifically, during webpage loading, the script execution order, variable values, and request trigger stack information are recorded, and this information is mapped to the abstract syntax tree nodes of the static JavaScript file, thus constructing a complete logical chain from "variable definition—path concatenation—request initiation." This mapping relationship identifies which requests originate from API calls in the static JavaScript file and further performs semantic-level parsing on these calls. When concatenated or variable-constructed APIs are detected, they are filtered based on code context information to eliminate non-business requests such as monitoring, advertising, and heartbeat detection. After filtering, the actual path and parameter structure of the API in the dynamic request data are reconstructed along the aforementioned mapping relationship. By tracing back the variable dependency chain, the complete form of the original API can be automatically pieced together, and the corresponding parameter names, types, and default values ​​can be restored. For example, for a request / user / ${id} / profile constructed from a template string, the system can replace ${id} with the value rules defined at the time of execution based on the execution context information, resulting in the normalized API path / user / :id / profile. Simultaneously, this step also parses the parameter structure in the request body, generating dynamic request features that can be used for subsequent vulnerability detection and interface analysis.

[0031] In traditional technologies, front-end interface analysis is typically divided into two methods: static scanning and network listening. Static scanning cannot identify dynamically constructed logic, while network listening, although capable of capturing all requests, cannot trace their source code. The lack of a connecting bridge between the two leads to fragmented and semantically incomplete analysis results. Step S2200, through a combination of logical mapping and context restoration, constructs a unified framework for extracting dynamic request features, resolving issues such as incomplete dynamic API identification, severe interference from non-business requests, and untraceable interface semantics.

[0032] Step S2300: Integrate the static parsing results and dynamic request features to generate enhanced interface entries; Specifically, after extracting static parsing results (including hard-coded API paths, request methods, and parameter structures) and dynamic request features (including the reconstructed concatenated or variable-structured API paths and parameter structures), these two types of information need to be integrated to form enhanced interface entries. An enhanced interface entry is a structured data representation used to fully describe the relevant information of each interface on the target webpage, including the interface path, request method, parameter structure, response fields, and the call relationships between interfaces. Taking the login interface on a target webpage as an example, the static parsing results provide a fixed path ` / api / login` and fixed parameters `username` and `password`, while dynamic request features may reconstruct dynamically concatenated `token` or `sessionId` parameters. By integrating the information from both, an enhanced interface entry is generated, fully covering the static and dynamic features of the interface. The generation of enhanced interface entries solves the problem that static analysis cannot cover dynamically constructed APIs, and also compensates for the deficiency that dynamic request analysis may miss static fixed parameters, achieving full capture of interface features. In the enhanced interface entry, each interface record can be accompanied by contextual information to mark sensitive fields or rule base matching, thereby providing basic data for subsequent vulnerability detection. The enhanced interface entries serve as the core nodes for constructing the global evidence graph, providing node attributes such as interface paths, parameter structures, and response fields. Meanwhile, the call relationships, parameter dependencies, and response triggering relationships between interfaces provide the basic data for the edges in the graph.

[0033] Step S2300: By summarizing the features of dynamic request data, a rule base containing field patterns and matching expressions is constructed; based on the rule base, the dynamic request data is matched and analyzed to obtain the corresponding sensitive field information; Specifically, after collecting and reconstructing dynamic request data, pattern analysis needs to be performed on the interface request parameters and response fields. Frequently occurring field names, data types, and value characteristics are compiled into rule base entries. Each rule includes a field identifier, a matching expression, and contextual constraints. The main purpose of this is vulnerability detection. The rule base allows for the rapid identification of fields that may involve sensitive information, such as passwords, tokens, or key parameters. This step ensures that vulnerability detection can accurately focus on data that may pose security risks, improving detection efficiency and accuracy.

[0034] Step S3000: Based on the generated enhanced interface entries, vulnerability detection and result storage are implemented.

[0035] Further, step S3000 includes: Step S3100: Construct a global evidence graph based on the enhanced interface entries; Furthermore, such as Figure 2As shown, step S3100 includes: Step S3110: Based on the enhanced interface entries, extract the interfaces, interface paths, and corresponding request parameters and response fields; Step S3120: Taking interface i and interface j as examples, interface i and interface j of the enhanced interface entry, as well as their request parameters and response fields, are used as nodes, and the connection from interface i to interface j is used as an edge to construct a global evidence graph. Specifically, the enhanced interface entries are the set of interface information generated in step S2300 by integrating static parsing results and dynamic request features. Each enhanced interface entry includes an interface name, interface path, request parameter structure, and response field information. By constructing a global evidence graph, with each enhanced interface entry as a node in the graph, the interface call relationship and data dependency structure can be formally represented, enabling visual analysis of the overall interaction structure of interfaces within a webpage or system. Based on the node construction, it is necessary to analyze the call relationship between interfaces, representing the process of interface i calling interface j as an edge in the graph. Each edge not only represents the call relationship but also carries data dependency information between interfaces, such as the response field of interface i mapping to the request parameters of interface j. By constructing this global evidence graph, the call chain and data dependency structure between interfaces can be clearly represented, providing basic data support for subsequent interface call frequency statistics, data dependency analysis, and vulnerability detection.

[0036] Specifically, each node in the graph contains complete interface information, including interface identifier, request parameter type and fields, response field type and fields, etc. Through this standardized modeling method of nodes and edges, the logical structure of the entire interface system is visualized on the graph, and the calling relationship and data dependency between interfaces are clearly mapped.

[0037] Step S3130: Analyze the dynamic request data based on the mapping relationship between the script execution logic and the actual request to obtain the interface call sequence log; Step S3140: Record data in the interface call sequence log; Furthermore, step S3140 includes: Step S3141: Set a time window. If interface i is connected to interface j in the interface call sequence log within the time window, record it as a call event from i to j. Step S3142: Based on the recorded data, obtain the number of call events from i to j within the time window.

[0038] Step S3143: Compare the number of times each event is called within the time window, and take the maximum number of times the event is called within the time window as the maximum call frequency. Specifically, an API call sequence log is a data structure that records API call behavior in a time-sequential manner within a target system, recording the start and end times, request parameters, and response results of each API call. Through script execution logic, the call order and dependencies between APIs can be accurately identified, thus forming a complete call chain. For example, when API A calls API B, the log records the time API A initiates the request, the time API B receives the request, and related parameters.

[0039] The purpose of setting a time window is to segment and statistically analyze API call events, similar to the time window analysis method in industrial examples. In practice, continuous API call behavior is divided into fixed time windows, and the number of call events from API i to API j is counted. In this way, the frequency of API calls is quantified, and the maximum frequency of calls between APIs is obtained.

[0040] Step S3150: Based on the obtained data recording results, obtain the edge weights of the global evidence graph; Furthermore, step S3150 includes: Step S3151: Based on the number of call events from i to j within the time window and the maximum call frequency, obtain the call frequency components from interface i to interface j. Step S3152: Obtain the response fields and request parameters corresponding to the call events from i to j, perform similarity matching on the response fields of interface i and the request parameters of interface j, and obtain the similarity of field names and field values ​​between the response fields of interface i and the request parameters of interface j. Step S3153: Based on the similarity of field names and field values ​​of the response fields of interface i and the request parameters of interface j, obtain the comprehensive similarity of field pairs of the response fields of interface i and the request parameters of interface j. Based on the obtained comprehensive similarity of field pairs of the response fields of interface i and the request parameters of interface j, obtain the data dependency strength of the response fields of interface i and the request parameters of interface j. Step S3154: Set up a custom rule base and obtain the custom rule score; Furthermore, step S3154 includes: Step S3154.1: Assume a custom rule in the custom rule base is: If the following fields are present in the interface response: token cannot be empty, token expired, token cannot be empty, token expired, Unauthorized, Access Denied, authentication failed, '"code":(4[0-9]{2})', '"code":"(4[0-9]{2})"', "Not logged in", "Not logged in", "token verify fail", "Authentication failed", "404 NOT_FOUND", "accessToken error", "Please log in", "Login status abnormal", "Login abnormal", "No token", "Login information expired", "Login information expired", then the interface is judged to have unauthorized access, and the custom score is 0. If the interface response does not contain the above fields, the custom score is 1. Step S3154.2: Match the response fields of interface i with the above custom rules to obtain the custom rule score; Step S3155: Calculate the weight values ​​corresponding to interfaces i to j in the global evidence graph based on the obtained call frequency components, data dependency strength, and custom rule scores from interface i to interface j. The formula for calculating the weight values ​​of interfaces i to interface j is as follows: ; in, This represents the weight values ​​from interface i to interface j. This is represented as the call frequency component from interface i to interface j. This represents the strength of the data dependency between the response fields of interface i and the request parameters of interface j. This is represented as a score based on custom rules. , Let be the weight value, and ; Specifically, edge weights are a comprehensive indicator used to quantify the closeness of the relationship between API calls. Their calculation integrates information from three dimensions: call frequency, data dependency, and security rules. The calculation of edge weights breaks through the limitations of single-dimensional evaluation, and unifies the modeling of the dynamic behavior of API calls, data flow relationships, and security risk characteristics.

[0041] When calculating the call frequency component, the number of call events from interface i to interface j within the time window is first counted and then normalized to the maximum call frequency. For example, if the login interface to the user information interface is called 15 times within the time window, and the maximum call frequency among all call events is 20 times, then the call frequency component is 15 / 20 = 0.75. This metric reflects the activity level of calls between interfaces; a higher call frequency indicates a closer business relationship between the two interfaces.

[0042] In calculating data dependency strength, similarity matching is performed by extracting the response fields of interface i and the request parameters of interface j. For example, if the login interface response field contains "user_id" and the user information interface request parameters also contain "user_id", and the field names match exactly and the value types are consistent, then the overall similarity is close to 1, indicating a strong data dependency relationship between the two interfaces. This calculation method aligns with the actual situation where data flows between interfaces through field passing during webpage loading, and can accurately identify the data dependency path between interfaces.

[0043] Step S3200: Perform vulnerability detection based on the global evidence graph and obtain the vulnerability detection results; Furthermore, such as Figure 3 As shown, step S3200 includes: Step S3210: Based on the edge weights of the global evidence graph, calculate the node comprehensive score of interface i. The calculation formula for the node comprehensive score is as follows: ; in, This is represented by the overall score of node i. Represented as outbound weight coefficient, Represented as the incoming weight coefficient, This represents the total number of interfaces. This is represented as the outgoing part, which is the set of edge weights for all call events originating from interface i. This is represented as the inbound part, which is the set of edge weights for all call events pointing to interface i; Step S3220: Obtain the comprehensive node score of each node in the global evidence graph and compare them. Take the node with the largest comprehensive node score as node 1 and sort them in descending order of comprehensive node score to generate a vulnerability detection sequence. Specifically, the node comprehensive score is a core indicator used to quantify the risk status of an interface during webpage loading, integrating weight information from two dimensions: the outbound call relationship and the inbound callee relationship of the interface.

[0044] When calculating the overall node score, all outgoing and incoming edges of interface i are extracted from the global evidence graph. The outgoing edge represents the set of edge weights from calls originating from interface i to other interfaces, reflecting the interface's influence on subsequent interfaces. The incoming edge represents the set of edge weights from calls from other interfaces to interface i, reflecting the interface's dependence on preceding interfaces. For example, the incoming edge of the order query interface contains an edge weight of 0.92 from the user information interface, and the outgoing edge contains an edge weight of 0.88 pointing to the payment interface. By weighting and summing the outgoing edge weights (0.4) and incoming edge weights (0.6), the overall node score is 0.4 × 0.88 + 0.6 × 0.92 = 0.904.

[0045] After obtaining the comprehensive score of all nodes, the node with the highest score is selected as node number 1 by comparison, and then the nodes are sorted in descending order of score to generate a vulnerability detection sequence. This sorting method prioritizes limited detection resources to the interfaces with the highest risk scores, solving the problem of delayed discovery of high-risk interfaces caused by the indiscriminate traversal of traditional scanners. Prioritization based on the evidence graph ensures that critical business interfaces and highly dependent interfaces are covered in the first round of detection, significantly improving the efficiency of vulnerability discovery. Simultaneously, it facilitates subsequent confidence level updates; once the detection results are fed back to the global evidence graph, the vulnerability detection sequence dynamically adjusts to adapt to the actual risk distribution, achieving an evolution from static scoring to dynamic optimization.

[0046] Step S3230: Set up a comprehensive rule base for vulnerability detection; Furthermore, step S3230 includes: Step S3231: Based on the call frequency components and data dependency strength of the interface, automatically generate interface context features, and based on the interface context features and the call frequency components of the interface, automatically generate a context semantic rule base. Step S3232: Organize the system's built-in basic detection rule library, context semantic rule library, and custom rule library to generate a comprehensive rule library for vulnerability detection. This rule library can be directly applied to the matching and verification of interface responses and request parameters during the detection process. Specifically, the comprehensive rule base is a multi-layered detection knowledge base that integrates static detection rules, dynamic contextual semantics, and custom security policies. It is used to intelligently match and verify interface responses and request parameters during vulnerability detection. The three rule bases are managed hierarchically according to priority and scope of application. The basic detection rule base contains regular expressions and keyword matching rules for common vulnerability characteristics such as SQL injection and XSS attacks, serving as the underlying detection foundation. The contextual semantic rule base contains business logic rules automatically generated based on interface call relationships, serving as the intermediate enhancement layer. The custom rule base contains specialized rules configured by security experts for specific applications, serving as the top-level customization layer. During detection, rules are applied sequentially from top to bottom, ensuring that specialized rules are matched first, with general rules providing fallback coverage. This layered integration approach resolves the problems of rule conflicts and priority confusion, enabling vulnerability detection to discover both general security issues and business logic vulnerabilities, significantly improving the accuracy and adaptability of detection.

[0047] Step S3240: Obtain the interface information corresponding to node 1 in the vulnerability detection sequence. The interface information includes the interface and its parameter structure and response fields. Make a real request to the interface and monitor it to obtain the monitored interface response fields. Step S3250: Perform data matching analysis on the monitored interface response fields based on the comprehensive rule base to obtain the number of matches between the interface response fields and the rule base, the number of abnormal responses in the interface response fields, and whether the interface call order matches the global evidence. Figure 1 To; Step S3260: The number of abnormal responses in the interface response field is used to determine the potential vulnerability type of the interface when a situation occurs in the interface response field that matches certain fields in the rule base. For example: When the API response data contains fields such as "not logged in", "Unauthorized", or "login information has expired", it is determined that the API may pose a risk of unauthorized access. When SQL keywords or exception stack information appear in the interface response fields, it is determined that the interface may have an SQL injection vulnerability; the SQL keywords and exception stack information are general detection features of the basic detection rule base; When an API response field returns sensitive information, it is determined that the API may pose a risk of information leakage. Step S3270: Record the number of times the interface has an abnormal response, and obtain the number of abnormal responses in the interface response field; Step S3280: Compare the number of matches between the obtained interface response fields and the rule base, the number of abnormal responses in the interface response fields, and the interface call order with the global evidence. Figure 1The vulnerability detection results are organized and marked as node 1 in the vulnerability detection sequence. Based on the vulnerability detection sequence, vulnerability detection is performed on each node in the global evidence graph in sequence to obtain the vulnerability detection results. Specifically, to quantify the risks posed by genuine requests, it is necessary to systematically compare captured interface responses with a comprehensive rule base and transform the matching results into structured detection conclusions. During data matching analysis, all rules in the comprehensive rule base are applied to perform pattern matching on each captured response body. For example, if a response contains both the word "unauthorized" and a string resembling an ID number, it matches once with both the "unauthorized access rule" and the "sensitive information disclosure rule," resulting in a total of 2 matches. Since both of these rules belong to the abnormal response category, the number of abnormal responses is also 2. Simultaneously, the call chain of the request is traced back and compared with the path defined in the global evidence graph. If they are completely consistent, the "call order consistency" is marked as 1.

[0048] To more effectively manage and utilize the detection results, all the analyzed quantitative indicators are compiled into a standardized vulnerability detection result record and associated with the corresponding interface nodes. Subsequent nodes are then processed sequentially according to the vulnerability detection sequence. For example, for node 1, its detection result is formatted as an object containing the node ID, total number of matches, number of abnormal responses, call order consistency, a list of potential vulnerability types, and key evidence fragments. After completing the recording for node 1, the detection engine moves to node 2 in the sequence and repeats the entire process of "acquiring information - constructing a request - initiating a request - capturing a response - matching analysis - organizing results" until all nodes requiring detection have been traversed.

[0049] Step S3300: Update the global evidence graph based on the vulnerability detection results to generate a confidence dataset; Furthermore, such as Figure 4 As shown, step S3300 includes: Step S3310: Calculate the node detection confidence score based on the vulnerability detection results, denoted as... ; in, ; in, This represents the number of matches between the interface response fields and the rule base. This is represented by the number of abnormal responses in the API response field. This indicates whether the order of API calls is consistent with global evidence. Figure 1 To, , , Represented as weighting coefficients; Step S3320: Perform a second vulnerability detection based on the global evidence graph, obtain the second vulnerability detection result, and calculate the detection confidence of the current node based on the second vulnerability detection result; Step S3330: Update the node detection confidence based on the obtained node detection confidence and the current node detection confidence. The update formula for the node confidence is expressed as follows: ; in, This represents the updated node detection confidence level. This is represented as the node detection confidence score. This represents the detection confidence level of the current node. Represented as the time decay smoothing coefficient; Step S3340: Based on the obtained updated node detection confidence, update the edge weights in the global evidence graph. The update formula for the edge weights in the global evidence graph is expressed as: ; in, This represents the updated weight values ​​from interface i to interface j. This represents the weight values ​​from interface i to interface j. This represents the node detection confidence of the updated interface i. This represents the node detection confidence score of the updated interface j. , Represented as the weighting balance coefficient, Represented as the historical weight retention coefficient; Specifically, node detection confidence is a comprehensive assessment metric used to quantify the risk level of an interface vulnerability. Its calculation is based on multi-dimensional vulnerability detection results obtained from the first round of vulnerability testing, including the number of matches between interface response fields and the rule base, the number of abnormal responses in the interface response fields, and whether the interface call order aligns with global evidence. Figure 1The node detection confidence score overcomes the limitations of single-rule matching by weightedly integrating three dimensions: rule coverage, anomaly severity, and context consistency. For example, if a user query interface triggers five rule matches in the first round of detection, including unauthorized access and SQL injection rules, and detects three abnormal responses (such as returning sensitive user data or displaying database error messages), and the actual call order of this interface perfectly matches the predicted call chain in the global evidence graph, then according to the confidence score calculation formula, the values ​​of these three dimensions are assigned weight coefficients, summed, and normalized to obtain a detection confidence score of 0.82 for this node. Step S3330 establishes a unified confidence metric model, integrating the scattered multi-dimensional evidence during vulnerability detection into a single numerical indicator, enabling subsequent risk ranking of all interface nodes on a standardized scale. In vulnerability detection environments, there are numerous interfaces and diverse detection evidence. This confidence quantification structure can effectively integrate and manage heterogeneous detection evidence, providing a foundation for the subsequent construction of dynamic update mechanisms and evidence graph optimization. It solves the problems of difficulty in quantifying and comparing detection results and strong subjectivity in risk assessment, and provides an objective node risk scoring framework, making it convenient to grasp the overall vulnerability distribution of the system.

[0050] After calculating the confidence level of the first round of node detection, a second round of detection is needed to verify the stability and accuracy of the confidence level. For example, in the first round of detection, the confidence level of the order interface was 0.85. In the second round of detection, it was found that this interface could not only reliably trigger unauthorized access vulnerabilities, but also additionally discovered the risk of unauthorized operation. At this time, the number of abnormal responses increased from 3 to 5, and the number of rule matches increased from 5 to 7. Based on the results of the second round of detection, the current node detection confidence level of this node was recalculated, resulting in a value of 0.91. In actual security testing, a single detection often misses some vulnerabilities due to incomplete context or insufficient parameter construction. This multi-round verification method can significantly improve the recall and accuracy of vulnerability discovery, solve the problems of unstable results in single-round detection and difficulty in triggering deep vulnerabilities, reveal the risk change pattern of the interface under different detection conditions, and provide key evidence for analyzing the true severity of vulnerabilities.

[0051] To fully utilize multi-round detection information and avoid complete overwriting of historical data, the node detection confidence level needs to be dynamically updated. This involves calculating the updated node detection confidence level based on the current node detection confidence level and the original node detection confidence level. The update mechanism introduces a time decay smoothing coefficient to achieve a weighted fusion of historical and current confidence levels. This means that while preserving the risk awareness accumulated from historical detections, it can also respond promptly to newly detected anomaly patterns.

[0052] To propagate node-level confidence updates throughout the entire evidence graph structure, the edge weights in the global evidence graph need to be updated based on the updated node detection confidence. The edge weight update integrates historical edge weights and the latest confidence of connected nodes, achieving multi-factor coordinated control through historical weight retention coefficients and weight balancing coefficients. For example, the original edge weight from interface A to interface B is 0.72. After two rounds of detection, the updated confidence of interface A is 0.85, and the updated confidence of interface B is 0.42. Setting the historical weight retention coefficient to 0.5 and the weight balancing coefficients to 0.6 and 0.4 respectively, the updated edge weight is 0.5 × 0.72 + (1 - 0.5) × (0.6 × 0.85 + 0.4 × 0.42) = 0.360 + 0.5 × 0.678 = 0.699. This updated weight value is then written back to the weight attribute field of the corresponding edge in the global evidence graph. This update strategy can recalibrate the importance of edges based on the actual risk level of nodes while preserving historical call patterns. It solves the problem that static weights in graph structures cannot reflect dynamic risk changes. While maintaining the topological stability of the evidence graph, it achieves adaptive optimization of weights and improves the accuracy of the evidence graph in depicting the actual vulnerability propagation path.

[0053] In traditional vulnerability detection architectures, the utilization of vulnerability detection results often remains at the level of simple vulnerability report generation, lacking in-depth mining of detection evidence and feedback optimization mechanisms. On the one hand, the calculation of detection confidence usually relies only on the result of rule matching in a single dimension, failing to comprehensively consider multi-dimensional factors such as anomaly severity and contextual consistency, resulting in inaccurate risk assessment. On the other hand, there is a lack of effective feedback channels between detection results and detection strategies. Vulnerability features discovered in the first round of detection cannot guide subsequent detection optimization, causing multiple rounds of detection to become simple repetitive scanning. This results in a lack of correlation between vulnerability detection results, making it difficult to form a systematic risk understanding, thus affecting the overall grasp of application security posture. The S3300, by constructing a confidence feedback update mechanism, forms a dynamic closed loop of detection-evaluation-optimization, successfully breaking through the static limitations of traditional detection. The confidence dataset makes vulnerability risk assessment more accurate and traceable. High-risk interfaces can be quickly screened through the first round of confidence, the stability of the risk can be verified through the current confidence, and the true threat level of the interface can be comprehensively assessed through the updated confidence. This multi-round iterative evaluation method helps reduce false positives and false negatives, improving the accuracy of detection.

[0054] Step S3350: Based on the edge weights of the updated global evidence graph, calculate the new node comprehensive score, generate a new vulnerability detection sequence based on the new node comprehensive score, and complete the update of the global evidence graph. Specifically, after the first round of vulnerability detection is completed and the detection confidence scores of each node are obtained, these actual detection feedbacks need to be integrated into the structure of the global evidence graph to achieve adaptive optimization of the graph structure. In the global evidence graph, each node represents an interface, and each edge represents the call relationship between interfaces and its weight. Taking the example that interface A has a confidence score of 0.85 in the first round of detection, and its successor interface B has a confidence score of 0.42, when updating the edge weights, the edge weights originally calculated based on call frequency and data dependency will be recalibrated. In specific implementation, by substituting the updated node detection confidence scores of interfaces A and B into the edge weight update formula, combined with the historical weight retention coefficient and weight balance coefficient, the edge weights from A to B are recalculated. At the same time, a confidence attribute field is maintained for each node in the graph data structure, and the value of this field is dynamically updated; a version marker is added to each edge to record the historical evolution trajectory of the edge weights. After all edge weights are updated, the graph structure is traversed again, and the node comprehensive scoring formula is applied to calculate the weighted sum of the outgoing and incoming weights of each node to obtain a new node comprehensive score. The vulnerability detection sequence is reordered from highest to lowest based on the new scores, generating an optimized sequence with more accurate ranking of high-risk nodes. This feedback-based dynamic graph structure update mechanism solves the problem that static graph scores cannot reflect the true distribution of vulnerabilities, providing more accurate priority guidance for subsequent iterations. This allows the system to concentrate detection resources on truly high-risk interface nodes, significantly improving the efficiency and accuracy of vulnerability discovery.

[0055] Step S3360: Perform a new round of vulnerability detection based on the updated global evidence graph; Step S3370: Record the node detection confidence and the updated node detection confidence to generate a confidence dataset; Specifically, after completing the first and second rounds of testing, each interface node will generate at least two confidence scores. These scores reflect the changes in risk assessment at different testing stages, and this time-series data needs to be systematically stored to support subsequent analysis. During data recording, a confidence time-series data structure is created for each interface node. This structure includes fields such as node identifier, timestamp, testing round, confidence score, and trigger rule type. Taking the user login interface as an example, its confidence score is 0.65 in the first round of testing. After graph updates, the confidence score rises to 0.78 in the second round. The system will create two records in the data structure corresponding to this interface: the first record is labeled with testing round 1, the timestamp is the first round testing time, the confidence score is 0.65, and the trigger rule is unauthorized access rule; the second record is labeled with testing round 2, the timestamp is the second round testing time, the confidence score is 0.78, and the trigger rules are unauthorized access rule and SQL injection rule.

[0056] Step S3400: The vulnerability detection results, the updated global evidence graph, the confidence dataset, and the sensitive field information are stored in a structured manner and provided for secondary retrieval. Furthermore, step S3400 includes: Step S3410: Store the vulnerability detection results, updated global evidence graph, confidence dataset, and sensitive field information in the Elasticsearch database in the form of structured documents, and generate five types of document collections according to function, including: interface node index, interface edge index, detection result index, confidence dataset index, and sensitive field index. Specifically, after multiple rounds of vulnerability detection are completed and a large amount of heterogeneous data is accumulated, these scattered detection results need to be transformed into persistent and queryable structured documents to support subsequent in-depth analysis and evidence tracing. During the data transformation process, based on the semantic features and usage scenarios of the data, the raw data is mapped to five different document index types. Taking vulnerability detection in a financial system as an example, the interface node index stores the static attributes and dynamic scores of each API interface, including fields such as interface path, request method, parameter structure, node comprehensive score, and resource type; the interface edge index records the call relationships between interfaces, including fields such as source interface ID, target interface ID, edge weight, call frequency component, and data dependency strength. The detection result index stores the vulnerability discovery records generated by each detection, including fields such as vulnerability type, triggering rule, abnormal response content, detection time, and interface. The confidence dataset index maintains the confidence evolution data of each node in different detection rounds, including fields such as interface ID, detection round, confidence value, timestamp, and confidence change. Sensitive field indexes store sensitive information discovered during the detection process separately, including fields such as field type, frequency of occurrence, associated interfaces, and discovery time. Each index uses a predefined mapping structure. For example, the interface node index is defined as containing an interface path field of type keyword, a node score field of type float, and a parameter structure field of type nested. The interface edge index uses a parent-child relationship to map the source interface and the target interface to support graph traversal queries.

[0057] Step S3420: Based on the generated document set, provide secondary retrieval of nodes, edges, and vulnerability detection results of the global evidence graph by interface, call event, vulnerability type, node detection confidence, and time window.

[0058] Example 2 This embodiment, based on Embodiment 1, provides a webpage information extraction and vulnerability detection system based on JavaScript intelligent analysis, such as... Figure 5 As shown, it includes: Webpage Loading and Dynamic Parsing Module: Used to load the target webpage, capture all static resources and dynamic request data during the webpage loading process, and obtain a webpage loading dataset with complete request and response characteristics; API Intelligent Extraction Module: Based on a webpage loading dataset carrying complete request and response characteristics, it obtains static parsing results and dynamic request characteristics to construct enhanced interface entries; Sensitive information extraction module: Based on dynamic request data, it constructs a sensitive field rule base through feature summarization to identify sensitive information in interface responses; Global Evidence Graph Generation Module: Based on enhanced interface entries, constructs a global evidence graph of interface call relationships and data dependencies; Vulnerability detection module: Based on the global evidence graph, it implements priority vulnerability detection and confidence feedback updates based on node comprehensive scoring; The results storage and retrieval module stores vulnerability detection results, updated global evidence graphs, confidence datasets, and sensitive field information in a structured manner and supports secondary retrieval.

[0059] In the webpage loading and dynamic parsing module, the method for loading the target webpage using a headless browser includes: Step S1100: Access the target webpage using a headless browser, obtain all static resources of the target webpage, and listen for dynamic requests initiated by the target webpage; the static resources include the webpage's HTML, CSS, and static JavaScript files; Step S1200: Mark the request parameters and response data in all dynamic requests initiated by the target web pages as dynamic request data and cache the data. Step S1300: The obtained static resources and dynamic request data are stored in a unified manner to generate a webpage loading dataset.

[0060] In the API intelligent extraction module, the method for constructing enhanced interface entries includes: Step S2100: Perform abstract syntax tree parsing on the static JavaScript file, extract the hard-coded API path, request method and parameter structure and mark them as static parsing results; Step S2200: Based on the obtained static resources, establish a mapping relationship between script execution logic and actual requests; analyze dynamic request data to identify concatenated or variable-structured APIs, and filter non-business interfaces based on context analysis; according to the filtered APIs, restore the API path and parameter structure along the mapping relationship and mark them as dynamic request features. Step S2300: Integrate the static parsing results and dynamic request features to generate enhanced interface entries.

[0061] In the sensitive information extraction module, the method for constructing a sensitive field rule base through feature induction includes: Step S2400: By summarizing the features of dynamic request data, a rule base containing field patterns and matching expressions is constructed; based on the rule base, the dynamic request data is matched and analyzed to obtain the corresponding sensitive field information.

[0062] In the global evidence graph generation module, the method for constructing the global evidence graph includes: Step S3100: Based on the enhanced interface entries, extract the interfaces, interface paths, and corresponding request parameters and response fields; construct a global evidence graph using the interfaces, their parameters, and response fields as nodes and the call relationships between interfaces as edges. Step S3120: Based on the mapping relationship between script execution logic and actual requests, obtain the interface call sequence log; set a time window, record call events between interfaces, and calculate the call frequency component and data dependency strength; Step S3130: Match the interface response fields based on the custom rule base to obtain the custom rule score; calculate the edge weight based on the call frequency component, data dependency strength, and custom rule score; Step S3140: Calculate the node comprehensive score based on the edge weights, wherein the node comprehensive score combines the outgoing weights and the incoming weights; sort the nodes from largest to smallest based on the comprehensive scores to generate a vulnerability detection sequence.

[0063] In the vulnerability detection module, the method for prioritizing vulnerability detection based on a global evidence graph includes: Step S3200: Based on the frequency components of interface calls and the strength of data dependencies, automatically generate a contextual semantic rule base; integrate the system's built-in basic detection rule base, contextual semantic rule base, and custom rule base to generate a comprehensive rule base; Step S3210: Obtain the interface information corresponding to the nodes in the vulnerability detection sequence, actually initiate a request and monitor it, and obtain the interface response fields; perform matching analysis on the response fields according to the comprehensive rule base to obtain the number of matches, the number of abnormal responses, and the consistency of the call order; perform detection on each node according to the vulnerability detection sequence to obtain the vulnerability detection results; Step S3300: Calculate the node detection confidence based on the vulnerability detection results; perform a second vulnerability detection and calculate the current node detection confidence; update the node detection confidence based on the node detection confidence and the current node detection confidence. Step S3310: Based on the updated node detection confidence, update the edge weights in the global evidence graph; calculate the new node comprehensive score, generate a new vulnerability detection sequence, and complete the update of the global evidence graph; perform a new round of vulnerability detection based on the updated global evidence graph; record the confidence data and generate a confidence dataset.

[0064] In the results storage and retrieval module, the method for achieving multi-dimensional structured storage and secondary retrieval capabilities includes: Step S3410: Store the vulnerability detection results, updated global evidence graph, confidence dataset, and sensitive field information in the Elasticsearch database in the form of structured documents, and generate five types of document collections according to function: interface node index, interface edge index, detection result index, confidence dataset index, and sensitive field index. Step S3420: Based on the generated document set, provide secondary retrieval of nodes, edges, and vulnerability detection results of the global evidence graph by interface, call event, vulnerability type, node detection confidence, and time window.

[0065] The methods and systems of this application may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the method is for illustrative purposes only, and the steps of the method of this application are not limited to the order specifically described above, unless otherwise specifically stated.

[0066] In addition, the parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of the corresponding technical solutions in the prior art have not been described in detail, so as to avoid excessive elaboration.

[0067] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis, characterized in that, The method includes: Load the target webpage and obtain all static resources and dynamic request data during the loading process. Based on all static resources and dynamic request data during the webpage loading process, enhanced interface entries are generated and sensitive field information is extracted. A global evidence graph is constructed based on the generated enhanced interface entries, and vulnerability detection is performed. The global evidence graph is updated based on the vulnerability detection results to generate a confidence dataset. The vulnerability detection results, updated global evidence graph, confidence dataset, and sensitive field information are stored in a structured manner and provided for secondary retrieval.

2. The method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis according to claim 1, characterized in that, The steps to obtain all static resources and dynamic request data during the loading process of the target webpage include: Access the target webpage using a headless browser, obtain all static resources of the target webpage, and listen for dynamic requests initiated by the webpage. The request parameters and response data in the detected dynamic requests are marked as dynamic request data and cached. The obtained static resources and dynamic request data are stored in a unified manner to generate a webpage loading dataset.

3. The method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis according to claim 2, characterized in that, Methods for generating enhanced interface entries include: Perform abstract syntax tree parsing on static JavaScript files to extract hard-coded API paths, request methods, and parameter structures; Based on static resources, establish a mapping relationship between script execution logic and actual requests, analyze dynamic request data, identify concatenated or variable-structured APIs, and filter non-business interfaces. Reconstruct the concatenated or variable-structured API path and parameter structure along the mapping relationship to obtain dynamic request characteristics; The static parsing results are integrated with the dynamic request characteristics to generate enhanced interface entries.

4. The method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis according to claim 3, characterized in that, Methods for extracting sensitive field information include: A rule base for field patterns and matching expressions is built based on the characteristics of dynamic request data to extract sensitive field information.

5. The method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis according to claim 4, characterized in that, Methods for implementing vulnerability detection and result storage include: Construct a global evidence graph based on enhanced interface entries; Vulnerability detection is performed based on a global evidence graph to obtain vulnerability detection results; The global evidence graph is updated based on the vulnerability detection results to generate a confidence dataset. The vulnerability detection results, updated global evidence graph, confidence dataset, and sensitive field information are stored in a structured manner and provided for secondary retrieval.

6. The method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis according to claim 5, characterized in that, Methods for constructing a global evidence graph based on enhanced interface entries include: Based on the enhanced interface entries, extract the interface, interface path, and corresponding request parameters and response fields; A global evidence graph is constructed by treating the interfaces, their request parameters, and response fields as nodes, and the call relationships between interfaces as edges. The dynamic request data is analyzed based on the mapping relationship between script execution logic and actual requests to obtain interface call sequence logs; a time window is set to calculate the call frequency components and data dependency strength between interfaces; The interface response fields are matched based on a custom rule base to obtain a custom rule score; The weights of edges in the global evidence graph are calculated based on the call frequency component, data dependency strength, and custom rule scores. Calculate the node comprehensive score based on the edge weights of the global evidence graph; generate a vulnerability detection sequence by sorting the nodes from largest to smallest based on their comprehensive scores.

7. The method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis according to claim 6, characterized in that, Methods for vulnerability detection based on global evidence graphs include: Based on the frequency of API calls and the strength of data dependencies, a contextual semantic rule base is automatically generated; the basic detection rule base, the contextual semantic rule base, and the custom rule base are integrated to generate a comprehensive rule base. Obtain the interface information corresponding to the node in the vulnerability detection sequence, make a real request to the interface and monitor it, and obtain the monitored interface response fields; Based on the comprehensive rule base, data matching analysis is performed on the monitored interface response fields to obtain the number of matches between the interface response fields and the rule base, the number of abnormal responses of the interface response fields, and whether the interface call order is consistent with the global evidence graph. Vulnerability detection is performed on each node in the global evidence graph according to the vulnerability detection sequence to obtain vulnerability detection results.

8. The method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis according to claim 7, characterized in that, The vulnerability detection results include the number of matches between the interface response fields and the rule base, the number of abnormal responses, and the consistency index between the interface call order and the global evidence graph.

9. The method for webpage information extraction and vulnerability detection based on JavaScript intelligent analysis according to claim 8, characterized in that, Methods for updating the global evidence graph based on vulnerability detection results include: Calculate the node detection confidence level based on the vulnerability detection results; A second vulnerability detection is performed based on the global evidence graph. The results of the second vulnerability detection are obtained, and the detection confidence of the current node is calculated based on the results of the second vulnerability detection. The node detection confidence is updated based on the node detection confidence and the current node detection confidence. Based on the updated node detection confidence, the edge weights in the global evidence graph are updated; Based on the updated edge weights, calculate the new node comprehensive score and generate a new vulnerability detection sequence; perform a new round of vulnerability detection based on the updated global evidence graph; record the node detection confidence and the updated node detection confidence to generate a confidence dataset.

10. A webpage information extraction and vulnerability detection system based on JavaScript intelligent analysis, used to implement the webpage information extraction and vulnerability detection method based on JavaScript intelligent analysis as described in any one of claims 1-9, characterized in that, The system includes: Webpage Loading and Dynamic Parsing Module: Used to load the target webpage, capture all static resources and dynamic request data during the webpage loading process, and obtain a webpage loading dataset with complete request and response characteristics; API Intelligent Extraction Module: Based on a webpage loading dataset carrying complete request and response characteristics, it obtains static parsing results and dynamic request characteristics to construct enhanced interface entries; Sensitive information extraction module: Based on dynamic request data, it constructs a sensitive field rule base through feature summarization to identify sensitive information in interface responses; Global Evidence Graph Generation Module: Based on enhanced interface entries, constructs a global evidence graph of interface call relationships and data dependencies; Vulnerability detection module: Based on the global evidence graph, it implements priority vulnerability detection and confidence feedback updates based on node comprehensive scoring; The results storage and retrieval module stores vulnerability detection results, updated global evidence graphs, confidence datasets, and sensitive field information in a structured manner and supports secondary retrieval.

Images