API information extraction method and device, electronic equipment and storage medium

Abstract

The invention relates to an API (Application Program Interface) information extraction method and device, electronic equipment and a storage medium. The method comprises the following steps: loading a collection Agent when a target application is started; a callback hook is injected through the collection Agent before a starting method of the target application is close to a return point, and the return point is used for indicating that starting of the target application is completed; when it is detected that the starting method is executed to the adjacent return point, triggering the callback hook, and obtaining a completely initialized application context returned by the starting method through the callback hook; and extracting all API information in the application context through the acquisition Agent. According to the method and the device, the accuracy of the acquired API information is ensured.

Description

Technical Field

[0001] This application relates to the field of computer security technology, and in particular to an API information extraction method, apparatus, electronic device, and storage medium. Background Technology

[0002] In modern software development practices, especially with the widespread application of microservice architecture and distributed systems, Application Programming Interfaces (APIs) have become the core means of inter-service communication and function exposure. The surge in the number and complexity of APIs has posed a severe challenge to API security. Interface vulnerability scanning is a crucial step in ensuring API security, and its effectiveness primarily depends on accurately and comprehensively obtaining the endpoint paths and detailed parameter definitions of all APIs in the system under test.

[0003] However, existing methods for obtaining API definition information to support vulnerability scanning have shortcomings. Manually maintaining API documentation is prone to inconsistencies between the documentation and the actual code implementation, leading to delayed updates, missing information, or errors. While traditional static source code analysis tools can automate extraction, these tools often struggle to fully simulate the dynamic configuration, dependency injection, and AOP behaviors of complex application frameworks (such as Spring Boot) at runtime. Static analysis does not execute code and cannot perceive the runtime environment, dependency injection results, or the framework's dynamic processing logic, potentially resulting in inaccurate parsing of API paths (especially those containing dynamic fragments or inherited from parent classes or interfaces) or parameters (especially complex nested objects, generics, or parameters indirectly defined through specific annotations).

[0004] Therefore, existing technologies face the problem of inaccurate parsing of API information when preparing API lists for automated interface vulnerability scanning. Summary of the Invention

[0005] This application provides an API information extraction method, apparatus, electronic device, and storage medium to solve the problem of inaccurate API information parsing.

[0006] Firstly, this application provides a method for extracting API information, the method comprising:

[0007] Load the data collection agent when the target application starts;

[0008] The collection agent injects a callback hook into the target application's startup method before the return point, where the return point is used to indicate that the target application has finished starting.

[0009] When the startup method is detected to have reached the near return point, the callback hook is triggered, and the fully initialized application context returned by the startup method is obtained through the callback hook;

[0010] The collection agent extracts all API information from the application context.

[0011] Optionally, injecting a callback hook by the collection agent before the return point of the launch method of the target application includes:

[0012] A class file converter is created using the registration class converter in the acquisition agent and registered to the virtual machine;

[0013] The target class currently loaded by the virtual machine is obtained through the class file converter.

[0014] If the class file converter determines that the target class is a preset startup class, the original bytecode of the target class is passed to the bytecode enhancer.

[0015] The bytecode enhancer locates the startup method in the target class and injects a callback hook before the nearest return point of the startup method.

[0016] Optionally, triggering the callback hook when the startup method is detected to have reached the near return point, and obtaining the fully initialized application context returned by the startup method through the callback hook, includes:

[0017] The virtual machine executes the injected callback hook when the startup method reaches the near return point;

[0018] The fully initialized application context is obtained from the return value of the startup method through the context access and triggering module;

[0019] The application context is captured through the callback hook.

[0020] Optionally, extracting all API information from the application context through the collection agent includes:

[0021] The application context is passed to the API extractor through the context access and triggering module.

[0022] The API extractor extracts the API information of each API object from the application context, wherein the API information includes the API path and HTTP method, the processor method, the API data structure, and the method parameters.

[0023] The API extractor combines the extracted API information into a structured API list and returns it to the context access and triggering module.

[0024] Optionally, extracting API information for each API object from the application context using the API extractor includes:

[0025] The API extractor obtains a mapping table from the application context, wherein the mapping table contains a mapping of all registered API request conditions to their corresponding handler methods;

[0026] Iterate through each entry in the mapping table and extract the API information for each API object;

[0027] After the traversal is complete, a list containing API information for all API objects is obtained.

[0028] Optionally, the method parameters include:

[0029] Parameter names, parameter types, parameter sources and attributes, parameter constraints, and the internal hierarchical structure of complex objects.

[0030] Optionally, after sending the extracted API information to a preset external endpoint, the method further includes:

[0031] The collection agent is deactivated from intervening in the normal API request processing flow.

[0032] Secondly, this application provides an API information extraction device, the device comprising:

[0033] The loading module is used to load the collection agent when the target application starts.

[0034] An injection module is used to inject a callback hook into the startup method of the target application near the return point through the collection agent, wherein the return point is used to indicate that the target application has finished starting.

[0035] The triggering module is used to trigger the callback hook when the startup method is detected to have reached the near return point, and to obtain the fully initialized application context returned by the startup method through the callback hook;

[0036] The extraction module is used to extract all API information in the application context through the collection agent.

[0037] Thirdly, this application provides an electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0038] Fourthly, this application also provides a computer storage medium storing computer-executable instructions for executing the API information extraction method described in any of the preceding claims of this application.

[0039] Compared with the prior art, the technical solution provided in this application has the following advantages: The collection agent is loaded at the initial stage of the target application startup, and a callback hook is injected before the return point during the execution of the core startup method of the target application. When the startup method reaches the near return point, it indicates that the target application has completed all initialization processes and the application context is in a fully ready state. At this time, the callback hook is triggered, and the fully initialized application context is accurately obtained from the return value of the startup method. Based on this application context, the collection agent can accurately extract all API registration information. Since the API information is collected after the application context is fully mature, the extracted information not only includes static definitions but also covers dynamically generated configurations and framework processing results at runtime, ensuring the accuracy of the collected API information. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0043] Figure 1 A flowchart of an API information extraction method provided in this application embodiment;

[0044] Figure 2 This is a schematic diagram illustrating the overall process of API information extraction provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the structure of an API information extraction device provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0049] The following will describe in detail an API information extraction method provided in this application embodiment, taking an application to the terminal where the target application is located as an example. Figure 1 As shown, the specific steps are as follows:

[0050] Step 101: Load the collection agent when the target application starts;

[0051] Step 102: By collecting data, the Agent injects a callback hook near the return point of the target application's startup method. The return point is used to indicate that the target application has finished starting.

[0052] Step 103: When the startup method is detected to be nearing its return point, a callback hook is triggered, and the fully initialized application context returned by the startup method is obtained through the callback hook;

[0053] Step 104: Extract all API information from the application context by collecting the Agent.

[0054] First, this application provides explanations for some terms involved, including the following:

[0055] Target application: refers to the business system that needs to extract API information, usually a microservice or web service built on a certain language (such as Java or Python).

[0056] API Information Collection Agent: This is a lightweight agent that loads when the target application starts and modifies specific methods using bytecode enhancement techniques to collect API information.

[0057] Callback hooks (Hook logic): A piece of logic inserted at a specific point in code execution, specifically referring to the logic inserted before the target application's startup method (such as Spring Application.run) returns successfully, used to obtain a fully initialized application context.

[0058] Application context: The core container in the Spring framework, it is the fully initialized runtime environment after the application starts, containing all registered beans and their configurations. For API information collection, it contains the definitions and mapping information of all API interfaces.

[0059] API information includes API endpoint paths, HTTP methods, parameter definitions (name, type, source, required, constraints, etc.), and the specific methods used to process these requests.

[0060] In step 101, when the terminal starts the application (whether it's Java, Python, or Node.js, etc.), it loads the collection agent through a specific mechanism. This agent is loaded early in the application's startup phase and parses the passed configuration parameters (such as the reporting address and application name) to complete initialization. The collection agent completes its own configuration parsing and module registration in the early stages of application initialization, preparing for subsequent operations and ensuring that the collection agent can intervene in the target application's lifecycle without affecting business logic.

[0061] In step 102, the acquisition agent uses bytecode enhancement techniques (such as ASM or Javassist in Java) to modify the target application's critical startup method (such as SpringApplication.run in SpringBoot). A custom logic (i.e., a callback hook) is inserted near the return point of the startup method. This process does not affect the original method's functionality; it only triggers the callback hook just before the startup method's return point (when the target application has fully started), allowing the callback hook to capture the startup method's return value (i.e., the fully initialized application context). This ensures that the application context is only acquired after the target application's state has stabilized.

[0062] In step 103, once the core startup method is about to reach its return point, indicating that the target application has completed the startup process, its core startup method will return a fully initialized application context (such as WebApplicationContext in Java). At this time, the pre-set callback hooks will be automatically triggered, safely obtaining this context instance. This context instance is the core container that manages all components within the framework, containing all API registration information maintained internally by the framework, and is the basis for extracting API information in the next step.

[0063] In step 104, using the acquired application context, the Agent performs in-depth analysis of all API mapping tables maintained by the framework within the context instance, extracting detailed information for each API endpoint. This information includes, but is not limited to: API path, HTTP method, controller class name, processing method name, request parameter type, and its constraints. Finally, this API information is organized into a structured data format (such as JSON) and sent over the network to a pre-defined external system (such as a vulnerability scanning platform or API documentation service).

[0064] In this application, the data collection agent is loaded at the initial stage of the target application's startup and a callback hook is injected during the execution of the core startup method of the target application, just before its return point. When the startup method reaches this near-return point, it indicates that the target application has completed all initialization processes and the application context is in a fully ready state. At this time, the callback hook is triggered, and the fully initialized application context is accurately obtained from the return value of the startup method. Based on this application context, the data collection agent can accurately extract all API information. Since the API information is collected after the application context is fully mature, the extracted information includes not only static definitions but also dynamically generated configurations and framework processing results at runtime, ensuring the accuracy of the collected API information.

[0065] As an optional implementation, after sending the extracted API information to a preset external endpoint, the method further includes: withdrawing the collection agent's intervention in the normal API request processing flow. The collection of API information is a one-time startup phase behavior; once completed, the agent no longer interferes with the normal API request processing flow, thus having almost no impact on the application's runtime performance. Unlike existing APM tools or certain API discovery technologies that collect information or monitor performance by intercepting each API request, this application exits after completing its core task, making it performance-friendly for the target application.

[0066] In step 101, taking Java as an example, the loading of the data collection agent when the target application starts includes the following: When the data collection agent starts, the JVM (Java Virtual Machine) passes the string passed through the -javaAgent parameter (e.g., "reportUrl=http: / / vul-scanner-input.example.com / APIs; appName=OrderService") to the premain method. The data collection agent is responsible for parsing this string, extracting key-value pairs, assigning http: / / vul-scanner-input.example.com / APIs to the internal variable targetReportUrl, and assigning OrderService to the internal variable applicationIdentifier. If parsing fails or necessary parameters are missing, a warning is logged and default values ​​may be used.

[0067] As an optional implementation, step 102, injecting a callback hook by collecting data from the Agent before the return point of the target application's launch method, includes the following:

[0068] Step S11: Create a class file converter by collecting the registration class converter in the Agent and register it with the virtual machine;

[0069] Step S12: Obtain the target class currently loaded by the virtual machine through the class file converter;

[0070] Step S13: If the class file converter determines that the target class is the preset startup class, the original bytecode of the target class is passed to the bytecode enhancer;

[0071] Step S14: Locate the startup method in the target class using the bytecode enhancer and inject the callback hook before the nearest return point of the startup method.

[0072] In step S11, during the target application startup process, the acquisition agent dynamically creates a class file converter through its internal registered class converter. This class file converter is used to listen for and intercept the bytecode of classes being loaded in the virtual machine. Subsequently, the acquisition agent registers this converter with the Instrumentation interface of the runtime environment (such as the JVM), enabling it to intervene in the class loading process. The core function of this step is to establish an execution entry point for subsequent bytecode enhancement operations. By registering the class file converter, the acquisition agent can intervene in the processing flow during the class loading phase, thereby achieving non-intrusive modification of the target class.

[0073] In step S12, when the virtual machine begins loading a class, it sequentially calls all registered class file converters. At this time, the class file converters registered with the collection agent receive the raw bytecode of the class being loaded. The class file converters parse the basic information of the class (such as class name and package path) and determine whether further processing of the class is needed. The purpose of this step is to identify key classes, especially those related to application startup, providing a filtering basis for the next step of injecting callback hooks.

[0074] In step S13, once the class file converter confirms that the currently loaded class belongs to a preset startup class (such as SpringApplication in Spring Boot or the core startup class in other frameworks), the raw bytecode of that class is passed to the bytecode enhancer. The bytecode enhancer is responsible for analyzing the bytecode structure and preparing to modify it. This step achieves precise filtering and processing of target classes, avoids meaningless processing of irrelevant classes, and improves the performance and efficiency of the system.

[0075] In step S14, the bytecode enhancer searches for startup methods (such as `public static void main(String[] args)` or `SpringApplication.run()`) in the target class and locates the position before the return point of this method through bytecode analysis. Subsequently, the bytecode enhancer inserts a piece of custom logic (i.e., a callback hook) at this position. This logic will be executed before the method returns, used to capture the method's return value (usually a fully initialized application context). This step is one of the key links in the entire collection process, ensuring that the collection agent can obtain complete context information promptly after the application has finished starting.

[0076] The following explanations of steps S11 to S14 are provided with examples.

[0077] 1. Converter registration.

[0078] During the Agent initialization phase, the Agent instantiates a class file converter object and then calls the addTransformer method of the Instrumentation object passed to it by the JVM to register the converter object with the JVM and specify that subsequent class retransformation is allowed.

[0079] Converter registration requirements: The JVM supports the JavaAgent specification; the AgentJAR package and its dependencies can be accessed by the target application's class loader. Parameters: Agent startup parameter string. Result: Agent configuration is loaded, the core class file converter has been registered in the JVM, and it is ready to intercept class loading.

[0080] 2. Implementation steps of the file converter.

[0081] 2.1 Class Name Matching: When the JVM loads any class, the `transform` method of the class file converter is called, which receives the name of the loaded class. The class file converter then performs an exact comparison between the received target class name and the default startup class name, `org.springframework.boot.SpringApplication`.

[0082] 2.2 Invoking the enhancer: If the class name matches successfully, the raw bytecode of the target class (as input as a byte array) is passed to the bytecode enhancer for processing.

[0083] 3. Implementation steps of the bytecode enhancer (for the SpringApplication.run method).

[0084] 3.1 Method Locator: Use a bytecode manipulation technique (such as the functionality provided by ASM or Javassist) to parse the bytecode of the input org.springframework.boot.SpringApplication class. Within the parsed structure of this class, look for a method named run that must conform to a specific signature: accepting an array of strings (String[] args) as an argument and returning an object of type org.springframework.context.ConfigurableApplicationContext.

[0085] 3.2 Injecting Hook Logic: If the `run` method is successfully located, a new sequence of instructions (Hook logic) is prepared. This Hook logic is designed to be executed before the original `run` method returns its result after all original instructions have been executed. The core operation of this Hook logic is: obtain the return value of the `run` method (i.e., the `ConfigurableApplicationContext` instance), safely convert this return value to the `WebApplicationContext` type, check whether the converted `WebApplicationContext` is valid (not null), and if valid, trigger the pre-defined procedure of the execution context access and API extraction module (usually calling a static method and passing the `WebApplicationContext` instance and the previously obtained `applicationIdentifier` as parameters).

[0086] 3.3 Application Modification: The prepared Hook logic instruction sequence is injected into the return point of the run method before the return point using bytecode manipulation techniques.

[0087] As an optional implementation, the startup process of the target application includes the following:

[0088] Step S21: During the startup process of the target application, the new bytecode after the injection of the callback hook is sent to the virtual machine through the class file converter via the bytecode enhancer;

[0089] Step S22: Obtain the preset startup class by loading the new bytecode using the virtual machine;

[0090] Step S23: Execute the startup method based on the preset startup class through the virtual machine and complete the initialization of the application context.

[0091] In step S21, the bytecode enhancer analyzes the startup method (such as the run() method) and inserts a custom Hook logic near its return point. Subsequently, the bytecode enhancer returns the new bytecode containing the Hook logic to the class file converter, which then submits it back to the virtual machine (JVM) as part of the class definition to replace the original class's bytecode.

[0092] In step S22, after receiving the enhanced new bytecode, the virtual machine uses the current class loader to create a new class instance based on the new bytecode. This class is the core startup class of the target application (such as SpringApplication.run), but its run() method already contains hook logic for collecting Agent injection. At this point, the class is in an executable state, waiting to be called to start the application.

[0093] In step S23, the virtual machine calls the enhanced startup method (such as SpringApplication.run) to begin executing the startup process of the target application. During this process, the framework sequentially completes component scanning, bean initialization, configuration loading, and other operations, ultimately building a fully initialized application context. Throughout the entire application startup process, the injected Hook logic is not executed; it is only triggered just before the startup method returns a result, thus safely capturing the application context.

[0094] The following explanations of steps S21 to S23 are provided with examples.

[0095] 1. Generate new bytecode containing Hook logic.

[0096] The bytecode enhancer receives the raw bytecode of the target class (such as org.springframework.boot.SpringApplication) and parses and modifies it. Near the return point of the target class's startup method (such as the run() method), a custom Hook instruction sequence is injected. After modification, the bytecode enhancer returns a new bytecode array containing the Hook logic for further processing by the class file converter.

[0097] The following conditions must be met to generate new bytecode.

[0098] Requirements: Input must be the raw bytecode of the org.springframework.boot.SpringApplication class; the current environment must support bytecode enhancement operations.

[0099] Input parameters: An array of the raw bytecode of the target class.

[0100] Execution result: Returns the modified bytecode of the org.springframework.boot.SpringApplication class; its run() method executes the injected Hook logic before returning.

[0101] 2. The file converter returns the processed bytecode.

[0102] The class file converter receives bytecode data returned by the bytecode enhancer and determines whether enhancement processing has been completed. If so, it executes a startup method based on a preset startup class via the virtual machine and completes the application context initialization.

[0103] If the current class is a pre-defined startup class (such as SpringApplication), the enhanced bytecode is returned to the virtual machine; otherwise, the original bytecode or null is returned directly, indicating that no modifications are made to the class.

[0104] The conditions for the bytecode returned by the file converter to meet the following requirements are as follows.

[0105] Requirements: The target Spring Boot application is starting up; the current class is being loaded by the JVM.

[0106] Input parameter: An array of the raw bytecode of the class being loaded.

[0107] Execution result: Only when the org.springframework.boot.SpringApplication class is loaded is its bytecode replaced with the enhanced version; for other classes, the original bytecode remains unchanged, and its loading and execution process is not affected.

[0108] 3. The virtual machine executes the startup method based on the preset startup class (such as SpringApplication.run) and completes the initialization of the application context.

[0109] As an optional implementation, in step 103, when the startup method is detected to be nearing a return point, a callback hook is triggered, and the fully initialized application context returned by the startup method is obtained through the callback hook, including the following:

[0110] Step S31: The injected callback hook is executed by the virtual machine when the startup method reaches a point near its return point;

[0111] Step S32: Obtain the fully initialized application context from the return value of the startup method through the context access and triggering module;

[0112] Step S33: Capture the application context via a callback hook.

[0113] In step S31, when the startup method of the target application's core startup class (such as SpringApplication.run) is executed by the virtual machine to a point close to its return point, the callback hook logic previously injected by the collection agent will be automatically triggered. This callback hook is a custom instruction sequence inserted during the bytecode enhancement phase, located precisely after all initialization operations of the startup method are completed and before the return value is generated. Therefore, it can be executed after the application context is fully initialized, but before it is officially returned to the caller.

[0114] In step S32, after the callback hook is triggered, its internal logic accesses the local variable table in the current runtime stack frame to extract the return value of the startup method—that is, a fully initialized ConfigurableApplicationContext instance (usually of type WebApplicationContext in a web scenario). Subsequently, the callback hook calls the static interface method provided by the context access and triggering module, passing the application context as a parameter, thereby completing the acquisition of the application context and triggering of subsequent processing.

[0115] In step S33, the callback hook not only acquires the application context during execution, but also securely saves this application context to the context management module of the collection agent. The purpose is to ensure that the subsequent API extraction module can access this context and extract a complete list of APIs based on the information contained therein.

[0116] As an optional implementation, in step 104, the API information extracted from the application context by the Agent includes the following:

[0117] Step S41: Pass the application context to the API extractor through the context access and triggering module;

[0118] Step S42: Obtain the mapping table from the application context through the API extractor, wherein the mapping table contains the mapping of all registered API request conditions to the corresponding handler methods;

[0119] Step S43: Traverse each entry in the mapping table and extract the API information for each API object. The API information includes the API path and HTTP method, the handler method, the API data structure, and the method parameters.

[0120] Step S44: After the traversal is complete, a list containing API information for all API objects is obtained.

[0121] Step S45: Combine the extracted API information into a structured API list using the API extractor and return it to the context access and trigger module.

[0122] In step S41, after the context access and triggering module successfully captures the fully initialized application context, it passes it as an input parameter to the API extractor. This application context contains all component registration information maintained internally by the framework and is the core source for subsequent API information extraction.

[0123] In step S42, after receiving the application context, the API extractor calls its internal interface method to obtain a key route mapping table from the application context. This mapping table records the mapping relationship between all registered API request conditions (such as paths and HTTP methods) and their corresponding handler methods, providing the foundation for subsequent API parsing.

[0124] In step S43, the API extractor parses each entry in the mapping table and extracts the detailed definition information of each API, including but not limited to the following:

[0125] API paths and HTTP methods: all associated URL path patterns (an API may correspond to multiple paths) and all supported HTTP request methods (such as GET, POST, etc.).

[0126] Processor method: Retrieves the corresponding JavaMethod object and the Class object of the Controller class to which the method belongs from the HandlerMethod.

[0127] API Data Structure: For each valid combination of path and HTTP method, a new data structure is created to represent an API definition. The basic information populated in this data structure includes: the application name currently being processed, the URL path, the HTTP method, the Controller class name, and the method name.

[0128] The method parameters include the following.

[0129] Parameter name: The parameter name is obtained through reflection (depending on the Java compiler's -parameters option to obtain the actual name; otherwise, annotations are required).

[0130] Parameter type: The fully qualified Java class name to retrieve the parameter.

[0131] Parameter source and attributes: Check the annotations on the parameters.

[0132] @PathVariable: Marks a path parameter and extracts the path variable name specified in the annotation.

[0133] @RequestParam: Marks a query parameter and extracts the parameter name, required attribute (whether it is required), and defaultValue attribute specified in the annotation.

[0134] @RequestHeader: Marks a request header parameter and extracts the header name and required attribute specified in the annotation.

[0135] @RequestBody: Marks a request body parameter and records the required attribute.

[0136] Parameter constraints: Inspect parameters or their internal fields. For complex JSR303 / 380BeanValidation annotations on @RequestBody (such as @NotNull, @Size, @Pattern, @Min, @Max, @Valid, etc.), record these constraint information in the parameter definition.

[0137] Internal hierarchical structure of complex objects: If the Java type of this parameter is a complex object (POJO or DTO), then all public fields (or properties defined by getter / setter) of the object class must be recursively checked, and the name, type, and any validation annotations on each field are used as the internal structure definition of this @RequestBody parameter. This is crucial for vulnerability scanners to construct valid JSON / XML request bodies, etc.

[0138] In step S44, after all entries in the mapping table have been parsed, the API extractor combines all extracted API definitions into a structured list. This list is organized in a uniform format (such as JSON or a custom JavaBean list) for easy subsequent processing and transmission.

[0139] In step S45, the API extractor returns the final generated structured API list to the context access and triggering module. This module can further call the data sending module to report the extraction results to external systems (such as vulnerability scanning platforms, interface documentation centers, or service governance platforms).

[0140] The following explanations of steps S41 to S45 are provided with examples.

[0141] 1. Context access and triggering module to pass application context.

[0142] 1.1 Obtain the application context.

[0143] The logic for the context access and triggering module is triggered after the core startup method `SpringApplication.run` of the target application has finished executing. It directly receives the return value of the core startup method, which is a fully initialized `WebApplicationContext` instance. This context contains all component and API registration information maintained internally by the framework.

[0144] 1.2 Call the API extractor.

[0145] The context access and triggering module calls the public interface of the API extractor, such as extractAPIDefinitionsFromContext(), and passes the obtained WebApplicationContext instance and the application identifier parsed during the initialization of the collection agent as parameters to the API extractor.

[0146] 2. The API extractor extracts API information.

[0147] 2.1 Receive the extraction results and prepare to report them.

[0148] When the API extractor initializes, it creates an empty list of API definitions to store each API information object built during the subsequent extraction process.

[0149] 2.2 Obtain the request mapping handler from the application context.

[0150] The API extractor retrieves a Bean instance of type RequestMappingHandlerMapping from the passed-in WebApplicationContext using the getBean method. This instance is a core component in the Spring MVC framework used to maintain API mappings.

[0151] The API extractor calls the getHandlerMethods() method of the RequestMappingHandlerMapping instance to obtain a mapping table. Then, it iterates through each entry in the table, where each entry represents the binding relationship between an API request condition (RequestMappingInfo) and the corresponding handler method (HandlerMethod).

[0152] 2.3 Parsing a single API object.

[0153] For each mapping entry, the API extractor performs the following operations: extracts the API path and HTTP method, processor method, API data structure, and method parameters.

[0154] 2.4 Returns a list of structured APIs.

[0155] Once all API objects have been parsed and added to the list, the API extractor returns this structured list to the upper-level context access and triggering module, completing the entire extraction process.

[0156] The list of structured APIs returned is shown below.

[0157] Conditions: The WebApplicationContext is available and contains a RequestMappingHandlerMappingBean.

[0158] Parameters: WebApplicationContext instance, applicationIdentifier.

[0159] Result: A structured list in which each element details the path of an API endpoint, the HTTP method and the name, type, origin, constraints and complex structure of all its parameters.

[0160] As an optional implementation, after the acquisition agent extracts all API information, it sends the API information to a preset external endpoint. The process includes the following:

[0161] Step S51: The context access and triggering module receives the return result from the API extractor call. This result is a structured list of APIs containing the paths and parameter definitions of all discovered APIs. The context access and triggering module calls the public interface of the data sending module (e.g., a static method named sendAPIData), passing the list of API definitions and the targetReportUrl (reporting address) obtained during Agent initialization as parameters.

[0162] The parameters passed include the following.

[0163] Conditions: The Spring Boot application has started successfully and the WebApplicationContext has been fully initialized.

[0164] Parameters: WebApplicationContext instance, applicationIdentifier, targetReportUrl.

[0165] Result: The API definition was extracted and scheduled for delivery.

[0166] Step S52: Implementation steps of the data sending module.

[0167] Step S521: Data Inspection. Receive the API list and target reporting URL. Check if the list is empty; if empty, do not send.

[0168] Step S522: Serialization. Use a JSON serialization library (such as Jackson or Gson, but the logic is described here) to convert the API list into a JSON-formatted string.

[0169] Step S523: HTTP POST Request. Construct an HTTP POST request. Set the target URL of the request to the passed-in targetReportUrl, set the Content-Type request header to application / json, and use the generated JSON string as the request body.

[0170] Step S524: Execution and Response Processing. Execute this request using an HTTP client, wait for and receive the server's response, check the response status code, and record the success or failure status and related information (such as error details).

[0171] The execution and response processing process includes the following:

[0172] Conditions: Network connection is available, and the target reported URL is accessible.

[0173] Parameters: API list, targetReportUrl.

[0174] Result: The API list was sent to the specified URL, and the result was recorded.

[0175] S53: Receiving and utilizing remote API receivers.

[0176] Step S531: Listening and Receiving. An HTTP service listens for POST requests on the targetReportUrl configured in the Agent. When a request is received, it reads the JSON data from the request body.

[0177] Step S532: Deserialization and Validation. Deserialize the received JSON data back into a list structure of the API definition object, and optionally perform data validation.

[0178] Step S533: Store or Use Directly. Store the parsed API information in a database or file, or load it directly into the vulnerability scanning engine as input for the scanning task.

[0179] Storage or direct use includes the following.

[0180] Condition: The receiving service is running and accessible.

[0181] Parameter: The API definition JSON data sent by the Agent.

[0182] Result: The vulnerability scanning tool obtained an accurate and detailed list of APIs, and could begin security scanning.

[0183] This application provides an overall architecture for an API information extraction system, including the following:

[0184] I. API Information Collection Agent.

[0185] Deployment method: As a JavaAgent JAR package, it is loaded when the target application starts via the JVM's -javaAgent parameter.

[0186] Overall functionality: During the startup process of the target application, it uses bytecode enhancement technology to hook key points, obtain the SpringWebApplicationContext, extract API definition information, and send it out.

[0187] The Agent collection submodule includes the following components.

[0188] 1. Agent Main Control Module: Implements the `premain` method of the Agent, serving as the entry point for the Agent. It is responsible for parsing the passed Agent configuration parameters (such as the API definition reporting URL and application name), initializing the Agent's internal state, and registering the class file converter.

[0189] 2. Class File Transformer: Implements the java.lang.instrument.ClassFileTransformer interface. It is called back when the JVM loads a class file. Its core task is to determine whether the currently loaded class is the preset target startup class, i.e., org.springframework.boot.SpringApplication. If so, it passes the bytecode of that class to the bytecode enhancer for processing.

[0190] 3. Bytecode Enhancer: Receives the bytecode of the target startup class (org.springframework.boot.SpringApplication). Using bytecode manipulation libraries (such as ASM or Javassist's abstraction capabilities), it locates the core startup method (such as the run method) of this class, modifies its bytecode, and injects custom hook logic into the node where the method successfully returns (afterReturning).

[0191] 4. Context Access and API Extraction Trigger Module: The logic of this module is injected into the return point of the target startup method by the bytecode enhancer. When triggered, it is responsible for safely obtaining a fully initialized WebApplicationContext instance from the return value of the startup method and immediately invoking the API extractor to pass the obtained context and application name configuration information.

[0192] 5. API Extractor: The core data extraction engine of this application. It receives a WebApplicationContext instance as input. By accessing and parsing the API registration information managed by the Spring framework in this context (mainly the content in RequestMappingHandlerMapping), it extracts the endpoint path, HTTP method, and detailed parameter definitions for each API (including parameter name, source, Java type, required status, default value, JSR303 / 380 validation annotations, and the internal field structure and validation rules of the @RequestBody complex object). Output: A structured list of data where each element represents the complete static definition of an API.

[0193] 6. Data Sending Module: Receives the list of API definitions generated by the API extractor. Serializes this list (usually in JSON format) and sends it over the network (e.g., via an HTTP POST request) to the remote URL specified by the Agent configuration parameter.

[0194] II. API defines storage and management services.

[0195] Deployment method: It is usually a standalone backend service.

[0196] Overall functionality: Responsible for receiving, storing, and managing API definition information reported by one or more Agents, and providing a query interface for vulnerability scanning tools or other systems.

[0197] The API defines the storage and management service sub-modules, which include the following:

[0198] 1. Data receiving interface: An HTTP endpoint (or other protocol interface) used to receive serialized API information sent by the Agent.

[0199] 2. Data parsing and verification module: Responsible for deserializing the received data and verifying its format and content.

[0200] 3. Data storage module: Persistently store API definition information using a database (such as a relational, document, or graph database) or a file system.

[0201] 4. Query and Supply API Module: Provides API interfaces that allow authorized clients (such as vulnerability scanning tools) to query and retrieve stored API information.

[0202] This application provides an overall step-by-step method for extracting API information, such as... Figure 2 As shown, it includes the following content.

[0203] Step 201: Agent loading and target class identification.

[0204] The target Java application (Spring Boot application) is started via the JVM's `-javaAgent` parameter, and the API information collection agent is loaded. The Agent's main control module executes the `premain` method to complete initialization and register the class file converter. During the JVM's class loading process, when the `org.springframework.boot.SpringApplication` class is loaded, the class file converter is triggered.

[0205] Step 202: Bytecode enhancement and Hook injection.

[0206] After the class file converter confirms that the current class is the target class, it calls the bytecode enhancer. The bytecode enhancer modifies the bytecode of the `run` method of the `org.springframework.boot.SpringApplication` class, injecting Hook logic at the method's successful return point, which is the entry point for the context access and API extraction trigger module.

[0207] Step 203: Application startup and context acquisition.

[0208] The `SpringApplication.run()` method of the target Spring Boot application executes normally and completes the initialization of the Spring application context. As the `run` method is about to return, the injected Hook logic is triggered. The context access and API extraction trigger module obtains the fully initialized `WebApplicationContext` instance from the return value of the `run` method.

[0209] Step 204: Extract API information.

[0210] The context access and API extraction trigger module calls the API extractor, passing it the obtained WebApplicationContext and application name configuration information. The API extractor then begins a deep analysis of the WebApplicationContext. It obtains the RequestMappingHandlerMapping, iterates through each API mapping, and parses out the API path and HTTP method. For each API's handler method, it performs a detailed analysis of all parameter names, Java types, source annotations (such as @PathVariable, @RequestParam, @RequestBody), constraint annotations, etc., and specifically handles the internal field structure and constraints of the complex object corresponding to @RequestBody. Finally, it constructs a structured list of all extracted API definition information.

[0211] Step 205: API information reporting.

[0212] The API extractor returns a list containing all API definitions to the caller (i.e., part of the Hook logic). Subsequently, the data sending module is invoked, which receives this list of API definitions. The data sending module serializes the API definition list into a JSON (or other conventionally formatted) string. This serialized data is then sent via an HTTP POST request to a remote URL specified in the Agent configuration, such as the data receiving interface of an API definition storage and management service, or directly to the data input endpoint of a vulnerability scanning tool.

[0213] Step 206: Remote reception and processing.

[0214] The remote API definition storage and management service receives an HTTP request via its data receiving interface. The data parsing and verification module processes the received data. The data storage module persists the API definition information. Vulnerability scanning tools or other authorized systems retrieve these API definitions by querying and supplying the API module for subsequent security scanning tasks.

[0215] The core content and technical effects achieved in this application include the following.

[0216] 1. Accurate capture and utilization of the context when the application starts up.

[0217] Technical Principle: Utilizing JavaAgent and bytecode enhancement techniques, this application precisely hooks the `run` method of the `org.springframework.boot.SpringApplication` class at the moment of its successful return. At this point, Spring's `WebApplicationContext` has completed the instantiation of all beans, dependency injection, AOP weaving, and the initialization and API registration of all web-related components (such as `RequestMappingHandlerMapping`). This application obtains a reference to this golden context through this hook point.

[0218] Innovation: Unlike analysis performed during class loading or earlier, this application intervenes after the application context has fully matured, ensuring that the analysis object is the final state containing all dynamic configurations and framework processing results, thereby achieving the highest accuracy of the API definition.

[0219] 2. Deep API metadata extraction targeting SpringMVC internal mechanisms.

[0220] Technical Principle: The API static definition extractor module in this application does not simply scan annotations, but interacts directly with core Spring MVC components (such as RequestMappingHandlerMapping). It programmatically queries these components to obtain the API registry parsed and maintained by the Spring framework itself.

[0221] Innovation: This method is equivalent to directly reading the framework's official documentation, which can accurately understand complex situations such as combined annotations, path variables, wildcards, HTTP method constraints, and parameter binding logic. The extracted API information is highly consistent with the actual processing behavior of Spring runtime.

[0222] 3. Parameter structuring and constraint parsing for vulnerability scanning requirements.

[0223] Technical Principle: When extracting API parameters, we not only focus on the parameter name and basic type, but also use reflection and annotation analysis to deeply analyze the source (@PathVariable, @RequestParam, @RequestBody, etc.), name (considering explicit naming in annotations), necessity, and default value of each parameter. In particular, for complex object parameters modified by @RequestBody, we recursively analyze their class definition to extract the names, types, and JSR303 / 380BeanValidation annotations (such as @NotNull, @Size, @Pattern, etc.) of all related fields, forming a hierarchical parameter structure description.

[0224] Innovation: This deep structuring of parameters and extraction of constraint information provides unprecedented detailed information for automated vulnerability scanning tools to generate effective and targeted test payloads, significantly improving the depth of scanning and the ability to discover vulnerabilities.

[0225] 4. One-time snapshot acquisition mode with extremely low runtime intrusion.

[0226] Technical principle: The collection of API definition information is strictly limited to a one-time action during the application startup phase. Once the information is extracted and reported, the Agent's main task is completed, and it will no longer incur any performance loss on the application's regular API request processing path.

[0227] Innovation: This snapshot mode ensures that the performance impact on the target application is minimized without sacrificing the accuracy and completeness of API definition information. It is particularly suitable for production environments or performance-sensitive systems, unlike solutions that require continuous monitoring of API calls.

[0228] The core features of this application include the following.

[0229] 1. API manifest is generated instantly upon startup.

[0230] The complete static definition (path and parameters) of the API is available shortly after the application starts. Unlike runtime traffic monitoring (which requires waiting for the API to be called) or manual documentation (which is often delayed), this application provides an up-to-date list of APIs immediately for subsequent processes such as vulnerability scanning.

[0231] 2. Information dimensions optimized specifically for vulnerability scanning.

[0232] The extracted API information is highly focused on the needs of vulnerability scanning tools, with particular emphasis on parameter names, exact sources (path, query, body, header, etc.), data types, constraints (such as whether they are required, validation rules), and the internal hierarchical structure of complex objects (such as JSON request bodies). Traditional API documentation or simple monitoring may only provide API paths and basic parameter names / types, lacking the in-depth details required for effective payload construction in vulnerability scanning. This application fills this gap.

[0233] 3. Zero runtime performance loss (after information collection is completed).

[0234] The collection of API definition information is a one-time startup phase activity. Once completed, the agent no longer interferes with the normal API request processing flow, thus having almost no impact on the application's runtime performance. Unlike APM-type tools or certain API discovery technologies that collect information or monitor performance by intercepting every API request, this application remains silent after completing its core task, making it more performance-friendly for applications.

[0235] 4. Highest accuracy based on the final state of the framework.

[0236] By analyzing the state of SpringWebApplicationContext after full initialization (especially RequestMappingHandlerMapping), we obtain the final API registration information after all the framework's parsing and processing logic (including inheritance, configuration, AOP, etc.). Static source code analysis cannot fully simulate the framework's runtime behavior and final configuration state, which may lead to inaccuracies. This application extracts directly from the framework's runtime reality, resulting in higher accuracy.

[0237] 5. Deep native integration with the Spring ecosystem.

[0238] The extraction logic is specifically designed for the internal workings and core components of Spring MVC and Spring Boot, enabling an accurate understanding of their annotation-driven and convention-over-configuration API definition approach. General-purpose bytecode scanning or annotation analysis tools may not be able to leverage Spring's own metadata management mechanism as deeply as this application, potentially leading to shortcomings in handling complex API definition scenarios unique to Spring.

[0239] This application can obtain complete API information of the target application immediately after it starts up. This method has the following significant advantages:

[0240] Immediacy: The latest API list is available immediately after the application starts.

[0241] Completeness: Extracts multi-dimensional information including paths, methods, parameters, constraints, nested structures, etc.

[0242] Accuracy: Extracts API information directly from the Spring framework runtime context.

[0243] Non-intrusive: Does not modify the source code and does not affect runtime performance.

[0244] Security support: Optimized for vulnerability scanning, improving scanning accuracy and efficiency.

[0245] Scalability: The extracted results can be used for various purposes such as interface documentation generation, service governance, and compliance auditing.

[0246] High compatibility: Deeply integrated with the Spring ecosystem, adaptable to various complex API definition scenarios.

[0247] Based on the same technical concept, this application provides an API information extraction device, such as... Figure 3 As shown, the device includes:

[0248] Loading module 301 is used to load the collection agent when the target application starts;

[0249] The injection module 302 is used to inject a callback hook near the return point of the startup method of the target application by collecting the Agent, wherein the return point is used to indicate that the startup of the target application is complete;

[0250] Trigger module 303 is used to trigger a callback hook when the startup method is detected to be nearing a return point, and to obtain the fully initialized application context returned by the startup method through the callback hook;

[0251] Extraction module 304 is used to extract all API information in the application context by collecting the Agent.

[0252] Optionally, the injection module 302 is used for:

[0253] A class file converter is created by collecting the registration class converter in the Agent and registered with the virtual machine;

[0254] Obtain the target class currently loaded by the virtual machine using a class file converter;

[0255] If the class file converter determines that the target class is the default startup class, the original bytecode of the target class is passed to the bytecode enhancer.

[0256] The bytecode enhancer locates the startup method in the target class and injects a callback hook before the startup method's nearest return point.

[0257] Optionally, the trigger module 303 is used for:

[0258] The injected callback hook is executed by the virtual machine when the startup method reaches a point close to its return point.

[0259] The fully initialized application context is obtained from the return value of the startup method through the context access and triggering module;

[0260] Capture application context through callback hooks.

[0261] Optionally, the extraction module 304 is used for:

[0262] The application context is passed to the API extractor through the context access and triggering module;

[0263] The API extractor extracts the API information of each API object from the application context. The API information includes the API path and HTTP method, the handler method, the API data structure, and the method parameters.

[0264] The API extractor combines the extracted API information into a structured list of APIs and returns it to the context access and triggering module.

[0265] Optionally, the extraction module 304 is specifically used for:

[0266] The API extractor retrieves a mapping table from the application context, which contains a mapping of all registered API request conditions to their corresponding handler methods.

[0267] Iterate through each entry in the mapping table and extract the API information for each API object;

[0268] After the traversal is complete, a list containing API information for all API objects is obtained.

[0269] Optionally, the method parameters include:

[0270] Parameter names, parameter types, parameter sources and attributes, parameter constraints, and the internal hierarchical structure of complex objects.

[0271] Optionally, the device is also used for:

[0272] Exit the data collection agent's intervention in the normal API request processing flow.

[0273] like Figure 4 As shown, this application provides an electronic device including a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0274] Memory 403 is used to store computer programs.

[0275] In one embodiment of this application, when the processor 401 executes the program stored in the memory 403, it implements the API information extraction method provided in any of the foregoing method embodiments.

[0276] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the API information extraction method provided in any of the foregoing method embodiments.

[0277] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0278] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0279] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0280] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for extracting API information, characterized in that, The method includes: Load the data collection agent when the target application starts; The collection agent injects a callback hook into the target application's startup method before the return point, where the return point is used to indicate that the target application has finished starting. When the startup method is detected to have reached the near return point, the callback hook is triggered, and the fully initialized application context returned by the startup method is obtained through the callback hook; The collection agent extracts all API information from the application context.

2. The method according to claim 1, characterized in that, The collection agent injects a callback hook into the target application's startup method near the return point, including: A class file converter is created using the registration class converter in the acquisition agent and registered to the virtual machine; The target class currently loaded by the virtual machine is obtained through the class file converter. If the class file converter determines that the target class is a preset startup class, the original bytecode of the target class is passed to the bytecode enhancer. The bytecode enhancer locates the startup method in the target class and injects a callback hook before the nearest return point of the startup method.

3. The method according to claim 2, characterized in that, When the startup method is detected to have reached the near return point, the callback hook is triggered, and the fully initialized application context returned by the startup method is obtained through the callback hook, including: The virtual machine executes the injected callback hook when the startup method reaches the near return point; The fully initialized application context is obtained from the return value of the startup method through the context access and triggering module; The application context is captured through the callback hook.

4. The method according to claim 3, characterized in that, The API information extracted from the application context by the collection agent includes: The application context is passed to the API extractor through the context access and triggering module. The API extractor extracts the API information of each API object from the application context, wherein the API information includes the API path and HTTP method, the processor method, the API data structure, and the method parameters. The API extractor combines the extracted API information into a structured API list and returns it to the context access and triggering module.

5. The method according to claim 4, characterized in that, The API information extracted from the application context by the API extractor includes: The API extractor obtains a mapping table from the application context, wherein the mapping table contains a mapping of all registered API request conditions to their corresponding handler methods; Iterate through each entry in the mapping table and extract the API information for each API object; After the traversal is complete, a list containing API information for all API objects is obtained.

6. The method according to claim 4, characterized in that, The method parameters include: Parameter names, parameter types, parameter sources and attributes, parameter constraints, and the internal hierarchical structure of complex objects.

7. The method according to claim 1, characterized in that, After sending the extracted API information to a preset external endpoint, the method further includes: The collection agent is deactivated from intervening in the normal API request processing flow.

8. An API information extraction device, characterized in that, The device includes: The loading module is used to load the collection agent when the target application starts. An injection module is used to inject a callback hook into the startup method of the target application near the return point through the collection agent, wherein the return point is used to indicate that the target application has finished starting. The triggering module is used to trigger the callback hook when the startup method is detected to have reached the near return point, and to obtain the fully initialized application context returned by the startup method through the callback hook; The extraction module is used to extract all API information in the application context through the collection agent.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images