Information extraction method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an information extraction method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining a source code of a to-be-processed file, and generating an abstract syntax tree corresponding to the source code; performing information extraction on the abstract syntax tree to obtain first extraction information; marking the target code module of which the parameter type is a preset parameter type in the first parameter information; for each marked target code module, generating a first context information packet based on a code module body source code of the marked target code module and a variable name of an object corresponding to the target parameter type; based on the first context information packet and preset prompt information, calling a preset large language model to perform information extraction on the marked target code module to obtain second extraction information; and performing data integration on the first extraction information and the second extraction information to obtain an information extraction result of the to-be-processed file so as to efficiently, accurately and comprehensively extract the API information.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an information extraction method, system, electronic device, and storage medium. Background Technology

[0002] An Application Programming Interface (API) is code provided to applications for use. It consists of predefined functions designed to allow application developers to call a set of routines without needing to understand the underlying source code or the details of their internal workings. API information has wide applications in many scenarios, such as API asset management, API security auditing, and software system refactoring and migration. Therefore, extracting API information from the source code of project files is particularly important.

[0003] Currently, one method for extracting API information from source code is a text matching method based on regular expressions. This method pre-defines a regular expression and uses it to match API information from the source code. While this method is simple and fast, it suffers from the limitation that regular expressions cannot understand the context of the code, meaning even minor changes in coding style (such as adding spaces or line breaks) can lead to matching failures. Another method is an analysis method based on Abstract Syntax Trees (ASTs). This method parses the source code into a structured syntax tree and uses code annotations to identify API definitions. While this method accurately identifies API information, it lacks semantic understanding, recognizing only explicit API information and failing to parse implicit parameters hidden within method bodies, resulting in incomplete API information extraction.

[0004] Therefore, how to accurately and comprehensively extract API information has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide an information extraction method, system, electronic device, and storage medium to efficiently, accurately, and comprehensively extract API information. The specific technical solution is as follows:

[0006] In a first aspect of this invention, an information extraction method is provided, the method comprising:

[0007] Obtain the source code of the file to be processed;

[0008] Generate the abstract syntax tree corresponding to the source code;

[0009] Information is extracted from the abstract syntax tree to obtain first extracted information; the first extracted information includes path information of at least one target code module and first parameter information of the target code module; the first parameter information includes parameter name and parameter type; the target code module is used to perform a specified task or implement a specified function, and the target code module is the code module corresponding to the application programming interface endpoint in the abstract syntax tree;

[0010] Mark the target code module whose parameter type is a preset parameter type;

[0011] For each tagged target code module, a first context information package is generated based on the source code of the tagged target code module and the variable name of the object corresponding to the target parameter type; and based on the first context information package and preset prompt information, a preset large language model is invoked to extract information from the tagged target code module to obtain second extracted information; the second extracted information contains the second parameter information of the tagged target code module; the target parameter type is the parameter type that is a preset parameter type in the first parameter information of the tagged target code module;

[0012] The first extracted information and the second extracted information are integrated to obtain the information extraction result of the file to be processed.

[0013] Optionally, the step of extracting information from the abstract syntax tree to obtain first extracted information includes:

[0014] Identify the controller class nodes contained in the abstract syntax tree;

[0015] For each controller class node, the annotations corresponding to the controller class node are parsed to extract the basic path corresponding to the controller class node, and the code module nodes within the controller class node are traversed. Based on the annotations of each code module node, the application programming interface endpoints contained in the controller class node are determined.

[0016] For each application programming interface (API) endpoint, the code module path in the corresponding code module annotation is extracted, and combined with the base path corresponding to the controller class node, the path information of the code module corresponding to the API endpoint is obtained, and the first parameter information of the code module corresponding to the API endpoint is extracted.

[0017] Optionally, if the tagged target code module contains comments, the comments of the tagged target code module are tagged; the method further includes:

[0018] For each marked target code module, a second context information package is generated based on the source code of the marked target code module, the variable name of the object corresponding to the target parameter type, and the comment content.

[0019] Optionally, based on the first context information packet and preset prompt information, the step of calling a preset large language model to extract information from the marked target code module to obtain second extracted information includes:

[0020] The first context information packet and the first preset prompt information are input into the preset large language model to analyze the code module body of the marked target code module, and the output code module body analysis information is obtained; the first preset prompt information is: prompt information or instruction used to guide the preset large language model to extract the information of request parameters, parameter types and parameter sources contained in the code module body of the marked target code module.

[0021] The second context information packet and the second preset prompt information are input into the preset large language model to extract information from the annotations of the marked target code module, and the output annotation analysis information is obtained; the second preset prompt information is a prompt or instruction used to guide the preset large language model to extract the parameter descriptions and summaries contained in the annotations of the marked target code module.

[0022] Based on the code module body analysis information and the annotation analysis information, the second extraction information is obtained.

[0023] Optionally, based on the first context information packet and preset prompt information, the step of calling a preset large language model to extract information from the marked target code module to obtain second extracted information includes:

[0024] The second context information packet and the third preset prompt information are input into the preset large language model to extract information from the marked target code module, and the second extraction information is output. The third preset prompt information is a prompt or instruction used to guide the preset large language model to extract the request parameters, parameter types, parameter sources and summaries contained in the marked target code module.

[0025] Optionally, the step of integrating the first extracted information and the second extracted information to obtain the information extraction result of the file to be processed includes:

[0026] Traverse each application programming interface endpoint. If the code module corresponding to the application programming interface endpoint is a marked target code module, merge the second extracted information and the first extracted information of the code module corresponding to the application programming interface endpoint to obtain merged information.

[0027] The merged information is formatted and fields are filled to obtain the information extraction result of the file to be processed.

[0028] Optionally, the method further includes:

[0029] For each file to be processed, determine whether the file has been modified.

[0030] If the file to be processed is modified, an abstract syntax tree corresponding to the source code of the file to be processed is generated;

[0031] If the file to be processed has not been modified, obtain the information extraction results corresponding to the stored file to be processed.

[0032] Optionally, after generating the first context information packet, the method further includes:

[0033] Query the correspondence between the context information packet and the extracted information to determine whether the first context information packet corresponds to extracted information;

[0034] If there is extraction information corresponding to the first context information packet, the extraction information corresponding to the first context information packet is obtained as the second extraction information;

[0035] If there is no corresponding extraction information in the first context information packet, the system performs information extraction on the marked target code module by calling a preset large language model based on the first context information packet and preset prompt information, and obtains the second extraction information.

[0036] In a second aspect of the invention, an information extraction system is also provided, the system comprising:

[0037] The code acquisition module is used to acquire the source code of the file to be processed.

[0038] A syntax tree generation module is used to generate an abstract syntax tree corresponding to the source code.

[0039] The first information extraction module is used to extract information from the abstract syntax tree to obtain first extracted information; the first extracted information includes path information of at least one target code module and first parameter information of the target code module; the first parameter information includes parameter name and parameter type; the target code module is used to perform a specified task or implement a specified function, and the target code module is the code module corresponding to the application programming interface endpoint in the abstract syntax tree;

[0040] The tagging module is used to tag target code modules whose parameter types are preset parameter types;

[0041] The second information extraction module is used to generate a first context information package for each marked target code module based on the source code of the marked target code module and the variable name of the object corresponding to the target parameter type; and to extract information from the marked target code module by calling a preset large language model based on the first context information package and preset prompt information to obtain second extracted information; the second extracted information includes the second parameter information of the marked target code module; the target parameter type is the parameter type that is a preset parameter type in the first parameter information of the marked target code module;

[0042] The data integration module is used to integrate the first extracted information and the second extracted information to obtain the information extraction result of the file to be processed.

[0043] In another aspect of the present invention, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0044] Memory, used to store computer programs;

[0045] A processor, when executing a program stored in memory, implements any of the methods described above.

[0046] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements any of the methods described above.

[0047] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the methods described above.

[0048] This invention provides an information extraction method, system, electronic device, and storage medium. The method acquires the source code of a file to be processed, generates an Abstract Syntax Tree (AST) corresponding to the source code, and extracts information from the AST to obtain first extraction information, including path information and first parameter information of the target code module corresponding to at least one API endpoint. This allows for rapid and accurate identification of most explicit API information. Target code modules with parameter types of a preset parameter type in the first parameter information are marked to identify complex semantic points that cannot be parsed during AST analysis. Further, based on the source code of the marked target code module and the variable names of objects corresponding to the preset parameter types, a first context information package is generated. Based on the first context information package and preset prompts, a preset large language model is invoked to extract information from the marked target code module, thereby extracting implicit API parameters of the marked target code module. The extracted first and second extraction information are integrated to obtain the information extraction result of the file to be processed. In this embodiment of the invention, AST analysis and large language model are combined. AST analysis is used to quickly and accurately identify most explicit API information, and the large language model is intelligently triggered to perform deep information extraction on target code modules in a few complex scenarios, thereby extracting API information efficiently, accurately and comprehensively. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0050] Figure 1 This is a flowchart illustrating an information extraction method in an embodiment of the present invention;

[0051] Figure 2 This is a flowchart illustrating an abstract syntax tree analysis in an embodiment of the present invention;

[0052] Figure 3 This is a flowchart illustrating an example of information extraction using a large language model in an embodiment of the present invention.

[0053] Figure 4 This is a schematic diagram of a data integration process in an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of another process for the information extraction method in an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of the structure of an information extraction system in an embodiment of the present invention;

[0056] Figure 7This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0058] To efficiently, accurately, and comprehensively extract API information, embodiments of the present invention provide an information extraction method, system, electronic device, and storage medium. The information extraction method provided by these embodiments can be executed by an information extraction system deployed in an electronic device, which can be a terminal device or a server device, etc.

[0059] In one example, the source code targeted by the information extraction method provided in this embodiment of the invention is Java-based (an object-oriented programming language). The API information extraction method of this embodiment can be widely applied in many scenarios such as API asset management, automated API documentation generation, API security auditing, and software system refactoring and migration. Specifically, API asset management can automatically scan the source code of all Java projects within an enterprise to build a complete and real-time API asset list. Automated API documentation generation can automatically generate API documentation for existing software systems that lack documentation, based on their corresponding API information. API security auditing identifies each parameter and its source in the API information to provide accurate input for automated security vulnerability scanning. Software system refactoring and migration can target microservices or software systems. Before microservice splitting or software system upgrades, the API information can be used to clearly identify the call relationships and interface contracts within the microservices or software system, facilitating software system refactoring and migration.

[0060] The following is a detailed description of an information extraction method provided by an embodiment of the present invention:

[0061] like Figure 1 As shown, Figure 1 This is a schematic flowchart of an information extraction method according to an embodiment of the present invention. The method includes:

[0062] S101, Obtain the source code of the file to be processed;

[0063] S102, Generate the abstract syntax tree corresponding to the source code;

[0064] S103, extract information from the abstract syntax tree to obtain first extracted information; the first extracted information includes path information of at least one target code module and first parameter information of the target code module; the first parameter information includes parameter name and parameter type;

[0065] The target code module is used to perform a specified task or implement a specified function. The target code module is the code module corresponding to the application programming interface endpoint in the abstract syntax tree.

[0066] S104, mark the target code module whose parameter type is a preset parameter type;

[0067] S105, for each marked target code module, generate a first context information package based on the source code of the marked target code module and the variable name of the object corresponding to the target parameter type; and based on the first context information package and the preset prompt information, call the preset large language model to extract information from the marked target code module to obtain the second extracted information;

[0068] The second extracted information includes the second parameter information of the tagged target code module; the target parameter type is the parameter type that is a preset parameter type in the first parameter information of the tagged target code module.

[0069] S106, integrate the first extracted information and the second extracted information to obtain the information extraction result of the file to be processed.

[0070] The information extraction method provided in this invention obtains the source code of the file to be processed, generates an Abstract Syntax Tree (AST) corresponding to the source code, and extracts information from the AST to obtain first extraction information, including path information and first parameter information of the target code module corresponding to at least one API endpoint. This method can quickly and accurately identify most explicit API information and marks target code modules whose parameter types in the first parameter information are preset parameter types, thereby identifying complex semantic points that cannot be parsed during AST analysis. Further, based on the source code of the marked target code module and the variable names of objects corresponding to the preset parameter types, a first context information package is generated. Based on the first context information package and preset prompt information, a preset large language model is triggered to extract information from the marked target code module, realizing the extraction of implicit API parameters of the marked target code module. The extracted first and second extraction information are integrated to obtain the information extraction result of the file to be processed. This invention combines AST analysis and a large language model, using AST analysis to quickly and accurately identify most explicit API information and intelligently triggering the large language model to perform deep information extraction on target code modules in a few complex scenarios, thereby efficiently, accurately, and comprehensively extracting API information. Furthermore, narrowing the applicability of large language models to a few complex scenarios can reduce the cost of information extraction while ensuring the quality of information extraction.

[0071] In this embodiment of the invention, Java source code is used as an example for illustration.

[0072] In this embodiment of the invention, API information extraction can be applied to scenarios such as security testing, specialized testing, and encryption auditing. For example, in a security testing scenario, developers submit their code daily. To determine which interfaces have been modified, a full scan of the source code corresponding to the submitted code is inefficient. Therefore, by extracting the API information from this embodiment, the API interfaces requiring testing can be quickly identified, improving the efficiency of security testing. Similarly, in specialized testing or encryption auditing scenarios, to determine which interfaces transmit personal privacy information (such as phone numbers) and which transmit product information, a full scan of the source code is inefficient and may fail to detect the relevant information. The API information extraction from this embodiment allows for the rapid identification of the corresponding API interfaces, improving the efficiency and accuracy of specialized testing and encryption auditing involving the transmission of personal privacy or product information.

[0073] In step S101, the path of the system or project file from which API information needs to be extracted is obtained. All files under this path are traversed, and source files ending in .java are identified as files to be processed. If multiple files exist, a list of files to be processed is generated. For each file to be processed, the complete source code of the file (i.e., the source file ending in .java) is loaded into the local machine or memory to obtain the source code corresponding to the file to be processed. Then, steps S102-S106 are executed to extract the API information.

[0074] In S102, for the file to be processed, the `javalang.parse.parse()` method is called to parse the source code corresponding to the file and generate an Abstract Syntax Tree (AST). The `javalang.parse.parse()` method is the core method in the Java code parsing library `javalang` used to parse Java source code into an AST.

[0075] In one example, you can iterate through the list of files to be processed and generate corresponding ASTs for the source code of each file in batches.

[0076] In this embodiment of the invention, a fault tolerance mechanism can also be constructed. If the source code parsing of the file to be processed fails and the AST cannot be generated, a parsing failure log is recorded for manual intervention. At the same time, the generation of the AST for the source code corresponding to the file to be processed is skipped, and the source code corresponding to the next file to be processed in the list of files to be processed is parsed, so as not to interrupt the API information extraction process, so as to realize the extraction of API information of system or project files.

[0077] In S103, with a structured AST already generated for the source code corresponding to the file to be processed, each node in the AST is traversed. Based on the annotation information of each node, the path information of the code module of each API endpoint corresponding to the code module node (i.e., method node) in the AST is extracted by keyword matching to obtain the first extracted information, so as to quickly and accurately obtain the first extracted information through AST analysis.

[0078] The keywords are set according to actual needs. The code module corresponding to each API endpoint in the AST is the target code module, which is used to execute a specified task or implement a specified function, i.e., a method. The method includes the method name and method body. A method is a named sequence of statements representing the behavioral characteristics of an object. For example, a window uses the show and hide methods to implement show and hide functions, and a shopping platform uses data transmission methods to upload product information. In other words, the target code module / code module referred to in this embodiment refers to a method. The first extracted information is: information parsed from the AST corresponding to the source code of the file to be processed, containing the path information of the code modules corresponding to each API endpoint and the first parameter information of the code modules. The first parameter information includes the parameter name and parameter type. In one example, the first parameter information may also include the code module name and code module type of the code module corresponding to each API endpoint. The code module name is also the method name, and the code module type is also the method type. The code module type can represent the type of the code module's return value or the code module's request type. For example, the parameter name can represent the variable name of the extracted parameter.

[0079] AST analysis is annotation-based analysis that can identify explicit API information in annotations such as `@GetMapping` and `@RequestParam`. However, it cannot parse implicit parameters hidden in the code logic of a code module, such as parameters obtained through `HttpServletRequest.getParameter("userId")`. Here, the code module body refers to the method body. In this embodiment of the invention, AST analysis enables high-speed extraction of basic API information. Furthermore, it marks special parameter types corresponding to implicit parameters existing in the code module body that cannot be parsed by AST analysis, thereby marking code modules that require further in-depth analysis.

[0080] In this embodiment of the invention, the preset parameter type can be the HttpServletRequest interface class, or the ServletRequest interface class, etc. Correspondingly, in S104, the parameter types in the first parameter information of the code modules corresponding to each API endpoint obtained from the AST analysis are traversed. When there is an API endpoint corresponding to a code module whose first parameter information contains a preset parameter type, the corresponding code module is marked, that is, the method (i.e., the target code module) with the preset parameter type is marked, indicating that the code module needs further in-depth analysis. Simultaneously, the variable name of the object corresponding to the preset parameter type in the first parameter information of the marked target code module can also be recorded. For example, for HttpServletRequest request, it indicates that the variable name of the object corresponding to the preset parameter type HttpServletRequest is request, and thus the variable name request is recorded.

[0081] If the parameter type in the first parameter information of the code module corresponding to the API endpoint is a preset parameter type, the first extracted information can be directly determined as the information extraction result of the file to be processed.

[0082] Furthermore, in S105, for each marked target code module, at least based on the source code of the marked target code module and the variable name of the object corresponding to the target parameter type, a first context information package is dynamically generated. The target parameter type is the parameter type that is a preset parameter type in the first parameter information of the marked target code module.

[0083] For example, based on the location information of the tagged target code module in the AST, this location information is mapped to the source code of the corresponding file to be processed, so as to accurately extract the complete code module body code (from [ to ]) of the tagged target code module, thus obtaining the source code of the code module body of the tagged target code module. Here, "[" in this article represents "..." ","】"express" ".

[0084] Furthermore, a first context information package can be dynamically generated based on the source code of the marked target code module, the variable names of the objects corresponding to the target parameter types, and the existing parameter list, thereby enriching the content of the first context information package. The existing parameter list consists of the parameter names and types, excluding the target parameter types and their object variable names, from the first parameter information of the marked target code module obtained through AST analysis.

[0085] At least based on the source code of the marked target code module and the variable names of the objects corresponding to the target parameter types, a first context information packet is generated. Using the first context information packet as input to the large language model helps reduce the understanding difficulty of the large language model, enabling the large language model to analyze more focused and accurately, and effectively suppressing model illusion.

[0086] Then, using the first context information packet and the preset prompt information as input, a preset large language model is invoked to extract information from the labeled target code module, resulting in second extracted information of the labeled target code module. This second extracted information includes second parameter information of the labeled target code module, which corresponds to the deep parameter information obtained from code module body analysis of the labeled target code module and is different from the first parameter information. The preset prompt information is a pre-set prompt for extracting information from the labeled target code module; the preset large language model can be a preset open-source large language model, such as the Tongyi Qianwen large language model, or a large language model obtained by fine-tuning and training an open-source large language model. This embodiment of the invention does not impose specific limitations on this.

[0087] Regarding the first and second extracted information, the first extracted information is the basic information of the code modules corresponding to each API endpoint obtained through AST analysis, while the second extracted information is the deep parameter information obtained by analyzing the code module body of the marked target code module through a large language model. The marked target code module belongs to the code module corresponding to the API endpoint. Therefore, the second extracted information can be supplemented into the first extracted information as the skeleton to obtain the complete API information extraction result of the file to be processed.

[0088] In one possible implementation, such as Figure 2 As shown, the implementation method for extracting information from the abstract syntax tree in step S103 above to obtain the first extracted information may include:

[0089] S201, determine the controller class nodes contained in the abstract syntax tree.

[0090] For the file to be processed, after generating the AST for the corresponding source code, the nodes in the generated AST are traversed, and all class declaration nodes in the AST are filtered out by matching the first key (such as the key class).

[0091] Furthermore, for the selected class declaration nodes, based on the annotations of each class declaration node, the class declaration nodes containing the keywords @RestController or @Controller are identified as API controller class nodes by matching the second keyword (such as @RestController or @Controller). This yields the controller class nodes contained in the AST.

[0092] S202, for each controller class node, parse the annotations corresponding to the controller class node, extract the base path corresponding to the controller class node, traverse the code module nodes within the controller class node, and determine the application programming interface endpoints contained in the controller class node based on the annotations of each code module node.

[0093] In one example, for each controller class node, the corresponding class-level annotation (such as @RequestMapping) is queried to extract the base path of that controller class node. For instance, the base path can be obtained by parsing the value or path attribute in the class-level annotation corresponding to the controller class node based on the _extract_path_value (select path value) logic.

[0094] Furthermore, for each controller class node, all code module declaration nodes within that controller class node are traversed. These code module declaration nodes are method declaration nodes. The annotations of each code module declaration node are queried, and by matching third-party keywords (such as @GetMapping, @PostMapping, etc.), code module declaration nodes containing these keywords are identified as API endpoints. Each API endpoint corresponds to a code module within a controller class node.

[0095] S203. For each application programming interface endpoint, extract the code module path from the code module annotation corresponding to the application programming interface endpoint, combine it with the base path corresponding to the controller class node to obtain the path information of the code module corresponding to the application programming interface endpoint, and extract the first parameter information of the code module corresponding to the application programming interface endpoint.

[0096] For each API endpoint, extract the path (method path) from the code module annotation of the corresponding code module. Concatenate this method path with the base path of the controller class node extracted above to obtain the path information of the code module corresponding to the API endpoint, i.e., the method path information. For example, for each API endpoint, the path of the code module corresponding to the API endpoint can be concatenated with the base path of its corresponding controller class node to obtain the path information of the code module corresponding to the API endpoint. The specific method for extracting the path from the code module annotation of the code module corresponding to the API endpoint can refer to the method for extracting the base path corresponding to the controller class node described above.

[0097] For each API endpoint, the parameter names and types of the corresponding code module are extracted to obtain the first parameter information. For example, based on the annotation name of the code module corresponding to the API endpoint, such as a direct mapping from GetMapping to GET, the Hypertext Transfer Protocol (HTTP) code module request type can be determined; this code module request type is the method request type. The annotations of the code module corresponding to the API endpoint are parsed to extract the code module name, which is the method name. The return type (return_type) of the code module is then parsed; the return type may include generics (such as List). <user>) and arrays (such as String[]).

[0098] Furthermore, the parameter list (parameters) node of the code module corresponding to the API endpoint can be traversed, and for each formal parameter (FormalParameter) node, the following information can be extracted:

[0099] The parameter source (in) is determined by parameter annotations such as @PathVariable, @RequestParam, @RequestBody, and @RequestHeader. If no parameter annotation is specified, the default parameter source is Query.

[0100] Extract the variable name of the parameter to get the parameter name (name);

[0101] Extract the type of the parameter, such as generics and arrays, to get the parameter type (type);

[0102] Additionally, it parses detailed information such as required and defaultValue from the annotation element to obtain other parameter information. Among them, required is the definition of the parameter attribute, indicating whether the parameter is a required parameter, and defaultValue indicates the default value of the specified parameter.

[0103] In this embodiment of the invention, by parsing the AST, the vast majority of API information can be extracted from the source code corresponding to the file to be processed quickly and accurately, thereby improving the extraction efficiency of API information.

[0104] After analyzing the AST and obtaining the first extracted information, the parameter types in the first parameter information of the code modules corresponding to each API endpoint obtained from the AST analysis are traversed. When there is an API endpoint corresponding to a code module whose first parameter information contains HttpServletRequest or ServletRequest, the code module corresponding to the API endpoint is marked and tagged with "requires code module body analysis". In one possible implementation, if the marked target code module contains comments, the comments of the marked target code module can also be marked. For example, if the marked target code module or the controller class node to which the marked target code module belongs contains a documentation (Javadoc comment) attribute, then the comment is marked, that is, tagged with "requires comment analysis"; furthermore, the above method may also include:

[0105] For each tagged target code module, a second context information package is generated based on the source code of the tagged target code module, the variable name of the object corresponding to the target parameter type, and the comment content.

[0106] For example, a second context information package can be generated based on the source code of the code module body of the tagged target code module, the variable name of the object corresponding to the target parameter type, the existing parameter list, and the comment content. The existing parameter list consists of the parameter names and parameter types in the first parameter information of the tagged target code module obtained through AST analysis, excluding the target parameter type and its object variable name.

[0107] Based at least on the source code of the marked target code module, the variable names of the objects corresponding to the target parameter types, and the comments, a second context information package is generated. The second context information package contains not only the code module body information but also the comments, making the content of the second context information richer, so as to facilitate more detailed and accurate deep information extraction of the marked target code module.

[0108] In one possible implementation, such as Figure 3 As shown, the implementation method of step S105 above, which involves using a preset large language model to extract information from the marked target code module based on the first context information packet and preset prompt information to obtain the second extracted information, includes:

[0109] S301, input the first context information packet and the first preset prompt information into the preset large language model to analyze the code module body of the marked target code module and obtain the output code module body analysis information.

[0110] The first preset prompt message is a prompt message or instruction used to guide the preset large language model to extract information such as request parameters, parameter types, and parameter sources contained in the code module body of the marked target code module.

[0111] For example, the first preset prompt message could be: "You are a Java code analysis expert. Analyze the following Java code module body, where the HttpServletRequest object name is [request variable name]. Identify all calls that retrieve request parameters, request headers, attributes, and cookies through this object (such as getParameter, getHeader). Return a list in JSON format containing the real name of each accessed item (such as "userId"), the inferred data type, and the source (parameter, header, attribute, cookie)." Correspondingly, the obtained code module body (i.e., method body) analysis information is a list in JSON format, which includes parameter names, parameter types, and sources.

[0112] S302, input the second context information packet and the second preset prompt information into the preset large language model to extract information from the comments of the marked target code module, and obtain the output comment analysis information.

[0113] The second preset prompt message is a prompt message or instruction used to guide the preset large language model to extract parameter descriptions and summaries contained in the comments of the marked target code module.

[0114] For example, the second preset prompt message could be: "You are an API documentation expert. From the following Javadoc content, extract a concise functional summary for this API endpoint, and extract the parameter descriptions corresponding to the @param and @return tags. Return the summary and parameter description list in JSON format." Accordingly, the obtained annotation analysis information is the summary and parameter description list in JSON format.

[0115] S303, based on the code module body analysis information and comment analysis information, the second extraction information is obtained.

[0116] The analysis of the code module body of the tagged target code module focuses on extracting parameter names, parameter types, and sources contained within the code module body. Meanwhile, the extraction of information from the comments of the tagged target code module focuses on extracting summaries and parameter description lists from the comment content. These two approaches do not conflict and each has its own emphasis, resulting in more accurate and comprehensive extracted information. Furthermore, the code module body analysis information and the comment analysis information are combined to obtain the second extracted information. This second extracted information contains information from both the code module body and the comments of the tagged target code module. Correspondingly, the second parameter information also includes information from both the code module body and the comments of the tagged target code module.

[0117] The large language model is invoked to analyze the code module body and comments of the tagged target code module, enabling the large language model to perform more focused and accurate analysis. This effectively suppresses model illusions and improves the accuracy of model analysis, resulting in more comprehensive information extraction.

[0118] In one possible implementation, the implementation of step S105 above, which involves calling a preset large language model to extract information from the marked target code module based on the first context information packet and preset prompt information to obtain the second extracted information, includes:

[0119] The second context information packet and the third preset prompt information are input into the preset large language model to extract information from the marked target code module, and the second extracted information is output.

[0120] The third preset prompt message is a prompt message or instruction used to guide the preset large language model to extract the request parameters, parameter types, parameter sources, and summaries contained in the tagged target code module.

[0121] For example, the third preset prompt message could be: "You are a Java Spring framework code analysis expert. Please analyze the following provided code module body and Javadoc. The HTTP request object is named 'request'. Your tasks are: 1. Extract a functional summary of the API from the Javadoc. 2. Identify all implicit parameters called via request.getParameter() and request.getHeader(), as well as uses of the request object via Helper-like classes. 3. Return your analysis results in strict JSON format, including the 'summary', 'inferred_parameters', and 'helper_usage' fields." Correspondingly, the obtained second extraction information includes a JSON-formatted summary, implicit parameters, etc.

[0122] By calling a large language model to extract information from the labeled target code modules, implicit parameters contained in the code modules that cannot be obtained by AST analysis can be extracted, making the extracted API information more comprehensive.

[0123] If the tagged target code module does not contain comments, the first context information package and the first preset prompt information can be directly input into the preset large language model to analyze the code module body of the tagged target code module and obtain the second extraction information.

[0124] In one possible implementation, such as Figure 4 As shown, the implementation method of step S106 above, which integrates the first extracted information and the second extracted information to obtain the information extraction result of the file to be processed, includes:

[0125] S401, traverse each application programming interface endpoint, and if the code module corresponding to the application programming interface endpoint is a marked target code module, merge the second extraction information and the first extraction information of the code module corresponding to the application programming interface endpoint to obtain merged information;

[0126] The first extraction information obtained from the above AST analysis is the basic extraction information extracted from the file to be processed, containing the path information and first parameter information of the code modules corresponding to each API endpoint. Further, by traversing each API endpoint, that is, traversing each code module in the controller class node of the file to be processed, if the code module corresponding to the API endpoint is a marked target code module (the code module that needs to be analyzed), the code module corresponding to that API endpoint has second extraction information. In this case, the second extraction information of the code module corresponding to the API endpoint is merged into the first extraction information to obtain the merged information. The merged information includes the first extraction information containing the path information and first parameter information of the code modules corresponding to each API endpoint, and the second extraction information of the marked target code modules. That is, when the code module corresponding to the API endpoint is not a marked target code module, its corresponding parameter information has already been extracted into the first extraction information, and the extraction of the second extraction information is not involved.

[0127] For example, merging the second extracted information of the code module corresponding to the API endpoint into the first extracted information can be done by using the first extracted information as a skeleton and filling the first extracted information with the second extracted information, thus obtaining the complete API information of the code module corresponding to each API endpoint.

[0128] S402, perform format conversion and field filling processing on the merged information to obtain the information extraction results of the file to be processed.

[0129] In one example, merged information can be cleaned and standardized to ensure it conforms to a standardized format. For instance, all type names in the merged information can be converted to lowercase, such as converting String to string, thus achieving data cleansing. Furthermore, for missing required fields in the merged information, reasonable default values ​​can be set, such as setting the default value to true for missing fields in required parameters, thereby achieving data standardization.

[0130] After obtaining the merged information of the file to be processed and performing format conversion and field filling, the complete API extraction information of the file to be processed is obtained, so that the obtained API information can be directly used for API management, document generation or automated testing, etc.

[0131] This invention utilizes a two-layer analysis architecture, prioritizing the use of AST analysis to extract information in terms of both breadth and speed. This allows for the rapid and accurate extraction of the vast majority of API information from the source code of the file to be processed, improving the efficiency of API information extraction. Furthermore, based on the AST analysis results, target code modules with parameter types of a preset parameter type in the first parameter information are marked. This serves as a trigger mechanism for information extraction using a large language model. The large language model extracts information in terms of both depth and precision. Specifically, it generates a first context information package based at least on the source code of the marked target code module and the variable names of objects corresponding to the preset parameter types in the first parameter information. The large language model is then invoked to perform deep information extraction on the marked target code module, resulting in more comprehensive extracted information.

[0132] During AST analysis, not only is the first extracted information extracted, but more importantly, the trigger points that require deep semantic analysis are identified. That is, the target code modules whose parameter types in the first parameter information are preset parameter types are marked to trigger the call of the large language model. This selective triggering mechanism avoids the expensive large language model analysis of all code, which greatly improves the overall efficiency and economy of API information extraction.

[0133] Compared to simply using AST analysis, introducing a large language model overcomes the limitations of AST analysis in understanding the internal logic and natural language comments of code modules, and can extract implicit API parameters in the code module body that are determined by program logic.

[0134] Through AST analysis, it can handle more than 90% of simple information extraction tasks. By using a selective triggering mechanism to call the large language model, the large language model is only called when necessary, narrowing its applicability to the few most critical complex scenarios. This ensures high-quality information extraction while minimizing the cost and time of information extraction operations.

[0135] In one possible implementation, after obtaining the information extraction result of the file to be processed, the information extraction result of the file to be processed can be stored in correspondence with the file to be processed.

[0136] Thus, the above method may also include:

[0137] When information needs to be extracted, for each file to be processed, it is determined whether the file has been modified. If the file has been modified, step S102 above is executed to generate the abstract syntax tree corresponding to the source code of the file to be processed. If the file has not been modified, the information extraction result corresponding to the stored file to be processed is directly obtained.

[0138] In one example, the modification status of a file to be processed can be determined by comparing its current hash value with its most recent historical hash value. If they are the same, the file has not been modified; otherwise, it has been modified. When the file is being processed for the first time, its most recent historical hash value is the hash value at the time of completion. Alternatively, the modification status can be determined by checking the file's last modification time. If the last modification time is the same as the stored completion time, the file has not been modified; otherwise, it has been modified.

[0139] In this way, if the file to be processed is modified, the step of extracting information from the file to be processed is executed, so as to avoid repeatedly parsing the unchanged Java file and save computing resources.

[0140] Similarly, after extracting information from the labeled target code module using the large language model to obtain the second extracted information, the first context information packet and the second extracted information can be stored accordingly. For example, the first context information packet can be used as a key, and the corresponding second extracted information can be used as a value to obtain the correspondence between the context information packet and the extracted information. Thus, in subsequent information extraction processes, after generating the first context information packet, the above method can also include:

[0141] Query the correspondence between context information packets and extracted information to determine whether the first context information packet corresponds to extracted information;

[0142] If there is extraction information corresponding to the first context information packet, the extraction information corresponding to the first context information packet is obtained as the second extraction information;

[0143] If no corresponding extraction information is found in the first context information packet, the system executes the extraction of information from the marked target code module by calling the preset large language model based on the first context information packet and the preset prompt information, and obtains the second extraction information.

[0144] By querying the correspondence between context information packets and extracted information, the first context information packet is matched with the stored context information packets. When the first context information packet matches a stored context information packet (i.e., its content is consistent / completely identical), it is determined that the first context information packet corresponds to extracted information, which is the extracted information corresponding to the matched stored context information packet. In this case, the extracted information corresponding to the first context information packet is directly obtained as the second extracted information, without needing to call the large language model again. When the first context information packet does not match a stored context information packet (i.e., its content is inconsistent / not completely identical), based on the first context information packet and preset prompt information, the preset large language model is called to extract information from the marked target code module to obtain the second extracted information.

[0145] In this way, when the code module body and context (i.e. the first context information package) of the code module to be analyzed in depth match the stored context information package, the extracted information corresponding to the stored context information package can be directly obtained as the second extracted information corresponding to the first context information package. There is no need to call the large language model again, which greatly reduces the cost and time of information extraction.

[0146] In one possible implementation, if the number of tagged target code modules is large, multiple tagged target code modules can be placed in an asynchronous task queue and processed asynchronously in parallel. For example, multiple processes or threads can be invoked to generate corresponding first context information packets for each tagged target code module in parallel, and a large language model can be invoked to extract information from the tagged target code modules. This not only improves the efficiency of information extraction but also increases the speed and throughput of information extraction.

[0147] In one possible implementation, such as Figure 5 As shown, the information extraction method of this invention includes:

[0148] S501, Obtain at least one file to be processed and its corresponding source code;

[0149] S502, for each file to be processed, determine whether the file has been modified;

[0150] S503, if the file to be processed has not been modified, obtain the information extraction results corresponding to the stored file to be processed;

[0151] S504, when the file to be processed is modified, generates an abstract syntax tree corresponding to the source code of the file to be processed;

[0152] S505, extract information from the abstract syntax tree to obtain first extracted information; the first extracted information includes path information of at least one target code module and first parameter information of the target code module; the first parameter information includes parameter name and parameter type;

[0153] The target code module is used to perform a specified task or implement a specified function. The target code module is the code module corresponding to the application programming interface endpoint in the abstract syntax tree.

[0154] S506, if the parameter type in the first parameter information of the target code module is a preset parameter type, the first extracted information is determined as the information extraction result of the file to be processed;

[0155] S507, if the parameter type in the first parameter information of the target code module is a preset parameter type, mark the target code module;

[0156] S508, For each marked target code module, generate a first context information package based on the source code of the marked target code module body and the variable name of the object corresponding to the target parameter type;

[0157] The target parameter type is the parameter type that is a preset parameter type in the first parameter information of the marked target code module;

[0158] S509, query the correspondence between the context information packet and the extracted information, and determine whether the first context information packet corresponds to extracted information;

[0159] S510, if there is extraction information corresponding to the first context information packet, the extraction information corresponding to the first context information packet is obtained as the second extraction information;

[0160] S511, if the first context information package does not contain corresponding extraction information, and the marked target code module contains comments, then the comments of the marked target code module are marked, and a second context information package is generated based on the source code of the marked target code module, the variable name of the object corresponding to the target parameter type, and the comment content.

[0161] S512, input the first context information packet and the first preset prompt information into the preset large language model to analyze the code module body of the marked target code module and obtain the output code module body analysis information;

[0162] The first preset prompt message is: a prompt message or instruction used to guide the preset large language model to extract information about request parameters, parameter types and parameter sources contained in the code module body of the marked target code module;

[0163] S513, input the second context information packet and the second preset prompt information into the preset large language model to extract information from the comments of the marked target code module, and obtain the output comment analysis information;

[0164] The second preset prompt message is a prompt message or instruction used to guide the preset large language model to extract the parameter descriptions and summaries contained in the comments of the tagged target code module;

[0165] S514, based on the code module body analysis information and comment analysis information, the second extracted information is obtained;

[0166] The second extracted information contains the second parameter information of the tagged target code module;

[0167] S515, if the first context information packet does not contain corresponding extraction information, and if the marked target code module does not contain comments, the first context information packet and the first preset prompt information are input into the preset large language model to analyze the code module body of the marked target code module and obtain the second extraction information.

[0168] S516, traverse each application programming interface endpoint, and if the code module corresponding to the application programming interface endpoint is the marked target code module, merge the second extraction information and the first extraction information of the code module corresponding to the application programming interface endpoint to obtain merged information; perform format conversion and field filling processing on the merged information to obtain the information extraction result of the file to be processed;

[0169] S517, store the extracted information from the file to be processed in correspondence with the file to be processed.

[0170] For example, in an API information retrieval business scenario, an internet company needs to modernize an outdated core transaction system developed using the Spring framework. This system is old, the original developers have left, and API documentation is severely lacking. One core user information controller, UserController, has a very complex parameter definition in its getUserInfo query module; some parameters are directly obtained within the module body via the HttpServletRequest object. This poses significant challenges to API organization and integration.

[0171] For this business scenario, the method of this invention can be used to automatically and completely parse all API information of the getUserInfo code module in UserController.java.

[0172] First, the code processing module loads the complete content of UserController.java into local storage or memory via the file path, obtaining the UserController.java file and its corresponding source code, as shown below. In the code below, "#" represents... ":

[0173] @RestController@RequestMapping(" / api / user")public classUserController

【

[0174] / ##

[0175] # Obtain user information based on multiple criteria.

[0176] # Supports querying by user ID or login name.

[0177] # @param request HTTP request object, containing additional query parameters.

[0178] # @return Serialized user information.

[0179] # /

[0180] @GetMapping(" / info")public String getUserInfo(HttpServletRequestrequest, HttpServletResponse response) [

[0181] / / Retrieve the user ID from the request parameters; this is the primary query condition.

[0182] String userId = request.getParameter("userId");

[0183] / / Retrieve the authentication token from the request header for security verification.

[0184] String token = request.getHeader("Authorization");

[0185] / / Obtain a complex parameter object using the Helper class

[0186] AuthCookie authCookie = AuthCookieHelper.getAuthCookie(request);

[0187] / / Get a debug parameter, which may not exist.

[0188] if (request.getParameter("debug") != null) [

[0189] / / Execute debugging logic... 】

[0191] return "success"; / / Simplified return value 】 】

[0194] Then, the static AST structured analysis module calls the `javalang.parse.parse()` method to generate an AST for the above source code, obtaining the AST of `UserController`. The nodes in the AST are traversed, and the class declaration node "public class UserController" is identified by the keyword `class`. Within the class declaration node, the `@RestController` and `@RequestMapping(" / api / user")` annotations are found. The `@RestController` keyword confirms that this class declaration node is the controller class node. Based on the annotations `@RestController @RequestMapping(" / api / user")`, the base path corresponding to the controller class node is extracted: ` / api / user`.

[0195] The code module nodes within the controller class nodes are traversed. The `@GetMapping(" / info")` annotation on the code module declaration node is searched. The `@GetMapping` keyword identifies this code module declaration node as an API endpoint. The code module path ` / info` is extracted from the annotation of the API endpoint's corresponding code module. This code module path is concatenated with the extracted base path of the controller class node to obtain the path information of the API endpoint's corresponding code module: ` / api / user / info`. Based on the annotation name `@GetMapping`, the API endpoint's corresponding code module is determined to be a GET request type. The code module name `getUserInfo`, along with the parameters `HttpServletRequest` and `HttpServletResponse`, are extracted. The resulting parameter list includes the parameter names `request` and `response`, corresponding to the parameter types `HttpServletRequest` and `HttpServletResponse`, respectively. This represents the first extracted information from the `UserController.java` file.

[0196] The semantic analysis trigger module detects that the parameter type HttpServletRequest of the code module getUserInfo is a preset parameter type, marks the code module getUserInfo corresponding to the API endpoint, tags the code module with "requires code module body analysis", and records the key information:

"target_method": "getUserInfo", "request_variable": "request"

[0197] The context building module, based on the location information of the AST nodes, precisely extracts the complete code block from [ to ] of the getUserInfo code module from the aforementioned source code, i.e., the source code body of the getUserInfo code module. The getUserInfo code module contains comments, and thus, the ` / ##… # / ` (i.e., ...) is extracted from the `documentation` attribute of the getUserInfo code module node. The comments in the `getUserInfo` code module are used to generate a second context information package based on the source code of the `getUserInfo` code module, the variable name `request` corresponding to the object with the preset parameter type `HttpServletRequest`, and the comments. The generated second context information package is as follows:

【

[0199] "method_body": "

【

[0200] String userId = request.getParameter(\"userId\"); ... 】",

[0201] "request_variable_name": "request",

[0202] Javadoc: "Retrieving user information based on multiple criteria..." 】

[0204] Furthermore, the semantic enhancement module of the large language model inputs the second contextual information packet and the third preset prompt information into the preset large language model to extract information from the labeled target code module, obtaining the output second extracted information. The output second extracted information is as follows:

【

[0206] "summary": "Retrieves user information based on multiple criteria, supporting queries by user ID or login name."

[0207] "inferred_parameters": [

[0208]

"name": "userId", "in": "Query", "type": "String", "description": "Inferred from request.getParameter in the code module body"

[0209]

"name": "debug", "in": "Query", "type": "Boolean", "description": "Inferred from the code module body if(request.getParameter)"

[0210]

"name": "Authorization", "in": "Header", "type": "String", "description": "Inferred from request.getHeader in the code module body"

[0211] ],

[0212] "helper_usage": [

[0213]

"class": "AuthCookieHelper", "method": "getAuthCookie", "description": "Call AuthCookieHelper to obtain authentication cookie information"

[0216] The functional summary and implicit parameter list output by the large language model (second extracted information) are merged into the basic API data (first extracted information) to obtain merged information. This merged information undergoes format conversion and field filling to obtain a complete, structured JSON object, which is the final information extraction result. This information extraction result accurately describes the getUserInfo interface. The information extraction result is as follows:

【

[0218] "path": " / api / user / info",

[0219] "http_method": "GET",

[0220] "summary": "Retrieves user information based on multiple criteria, supporting queries by user ID or login name."

[0221] "parameters": [

[0222]

"name": "userId", "in": "Query", "type": "string", "required":true, "description": "User ID, main query conditions"

[0223]

"name": "debug", "in": "Query", "type": "boolean", "required": false, "description": "Whether to enable debug mode"

[0224] [ "name": "Authorization", "in": "Header", "type": "string", "required": true, "description": "Authentication Token" 】

[0225] ],

[0226] Notes: This interface also handles authentication cookies via AuthCookieHelper. 】

[0228] Through the above steps, this invention successfully and comprehensively parses an API interface containing implicit parameters and complex logic. It not only extracts explicit information (path, method) that the AST can recognize, but more importantly, by intelligently triggering the large language model as its core, it uncovers deep semantic information hidden in the code and comments (implicit parameters userId, debug, request header Authorization, and function summary). The final result is far more complete and accurate than any single technical solution, fully demonstrating the feasibility, advancement, and practical value of this invention.

[0229] Corresponding to the above method embodiments, the present invention also provides corresponding system embodiments.

[0230] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of an information extraction system according to an embodiment of the present invention. The information extraction system according to this embodiment includes:

[0231] The code acquisition module 601 is used to acquire the source code of the file to be processed.

[0232] Syntax tree generation module 602 is used to generate an abstract syntax tree corresponding to the source code;

[0233] The first information extraction module 603 is used to extract information from the abstract syntax tree to obtain first extracted information; the first extracted information includes path information of at least one target code module and first parameter information of the target code module; the first parameter information includes parameter name and parameter type; the target code module is used to perform a specified task or implement a specified function, and the target code module is the code module corresponding to the application programming interface endpoint in the abstract syntax tree;

[0234] The tagging module 604 is used to tag target code modules whose parameter type is a preset parameter type;

[0235] The second information extraction module 605 is used to generate a first context information package for each marked target code module based on the source code of the marked target code module and the variable name of the object corresponding to the target parameter type; and to extract information from the marked target code module by calling a preset large language model based on the first context information package and preset prompt information to obtain second extraction information; the second extraction information includes the second parameter information of the marked target code module; the target parameter type is the parameter type that is a preset parameter type in the first parameter information of the marked target code module;

[0236] The data integration module 606 is used to integrate the first extracted information and the second extracted information to obtain the information extraction result of the file to be processed.

[0237] Optionally, the first information extraction module 603 described above is specifically used for:

[0238] Identify the controller class nodes contained in the abstract syntax tree;

[0239] For each controller class node, the annotations corresponding to the controller class node are parsed to extract the basic path corresponding to the controller class node, and the code module nodes within the controller class node are traversed. Based on the annotations of each code module node, the application programming interface endpoints contained in the controller class node are determined.

[0240] For each application programming interface (API) endpoint, the code module path in the corresponding code module annotation is extracted, and combined with the base path corresponding to the controller class node, the path information of the code module corresponding to the API endpoint is obtained, and the first parameter information of the code module corresponding to the API endpoint is extracted.

[0241] Optionally, if the tagged target code module contains comments, the comments of the tagged target code module are tagged; the system further includes:

[0242] The information packet generation module is used to generate a second context information packet for each marked target code module, based on the source code of the marked target code module, the variable name of the object corresponding to the target parameter type, and the comment content.

[0243] Optionally, the second information extraction module 605 described above is specifically used for:

[0244] The first context information packet and the first preset prompt information are input into the preset large language model to analyze the code module body of the marked target code module, and the output code module body analysis information is obtained; the first preset prompt information is: prompt information or instruction used to guide the preset large language model to extract the information of request parameters, parameter types and parameter sources contained in the code module body of the marked target code module.

[0245] The second context information packet and the second preset prompt information are input into the preset large language model to extract information from the annotations of the marked target code module, and the output annotation analysis information is obtained; the second preset prompt information is a prompt or instruction used to guide the preset large language model to extract the parameter descriptions and summaries contained in the annotations of the marked target code module.

[0246] Based on the code module body analysis information and the annotation analysis information, the second extraction information is obtained.

[0247] Optionally, the second information extraction module 605 described above is specifically used for:

[0248] The second context information packet and the third preset prompt information are input into the preset large language model to extract information from the marked target code module, and the second extraction information is output. The third preset prompt information is a prompt or instruction used to guide the preset large language model to extract the request parameters, parameter types, parameter sources and summaries contained in the marked target code module.

[0249] Optionally, the data integration module 606 described above is specifically used for:

[0250] Traverse each application programming interface endpoint. If the code module corresponding to the application programming interface endpoint is a marked target code module, merge the second extracted information and the first extracted information of the code module corresponding to the application programming interface endpoint to obtain merged information.

[0251] The merged information is formatted and fields are filled to obtain the information extraction result of the file to be processed.

[0252] Optionally, the above system also includes:

[0253] The first determining module is used to determine whether each file to be processed has been modified; if the file to be processed has been modified, the syntax tree generation module 602 is triggered to generate an abstract syntax tree corresponding to the source code of the file to be processed; if the file to be processed has not been modified, the information extraction results corresponding to the stored file to be processed are obtained.

[0254] Optionally, the above system also includes:

[0255] The second determining module is used to query the correspondence between the context information package and the extracted information, and determine whether the first context information package corresponds to extracted information; if the first context information package corresponds to extracted information, the extracted information corresponding to the first context information package is obtained as the second extracted information; if the first context information package does not correspond to extracted information, the second information extraction module 605 is triggered to execute based on the first context information package and the preset prompt information, and call the preset large language model to extract information from the marked target code module to obtain the second extracted information.

[0256] This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0257] Memory 703 is used to store computer programs;

[0258] When the processor 701 executes the program stored in the memory 703, it implements any of the above methods to achieve the same technical effect.

[0259] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0260] The communication interface is used for communication between the aforementioned terminal and other devices.

[0261] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0262] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0263] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the methods described in the above embodiments to achieve the same technical effect.

[0264] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the methods described in the above embodiments to achieve the same technical effect.

[0265] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0266] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0267] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system and electronic device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0268] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.< / user>

Claims

1. An information extraction method, characterized in that, The method includes: Obtain the source code of the file to be processed; Generate the abstract syntax tree corresponding to the source code; Information is extracted from the abstract syntax tree to obtain first extracted information; the first extracted information includes path information of at least one target code module and first parameter information of the target code module; the first parameter information includes parameter name and parameter type; the target code module is used to perform a specified task or implement a specified function, and the target code module is the code module corresponding to the application programming interface endpoint in the abstract syntax tree; Mark the target code module whose parameter type is a preset parameter type; For each tagged target code module, a first context information package is generated based on the source code of the tagged target code module and the variable name of the object corresponding to the target parameter type; and based on the first context information package and preset prompt information, a preset large language model is invoked to extract information from the tagged target code module to obtain second extracted information; the second extracted information contains the second parameter information of the tagged target code module; the target parameter type is the parameter type that is a preset parameter type in the first parameter information of the tagged target code module; The first extracted information and the second extracted information are integrated to obtain the information extraction result of the file to be processed.

2. The method according to claim 1, characterized in that, The step of extracting information from the abstract syntax tree to obtain the first extracted information includes: Identify the controller class nodes contained in the abstract syntax tree; For each controller class node, the annotations corresponding to the controller class node are parsed to extract the basic path corresponding to the controller class node, and the code module nodes within the controller class node are traversed. Based on the annotations of each code module node, the application programming interface endpoints contained in the controller class node are determined. For each application programming interface (API) endpoint, the code module path in the corresponding code module annotation is extracted, and combined with the base path corresponding to the controller class node, the path information of the code module corresponding to the API endpoint is obtained, and the first parameter information of the code module corresponding to the API endpoint is extracted.

3. The method according to claim 2, characterized in that, If the tagged target code module contains comments, the comments of the tagged target code module are tagged; the method further includes: For each marked target code module, a second context information package is generated based on the source code of the marked target code module, the variable name of the object corresponding to the target parameter type, and the comment content.

4. The method according to claim 3, characterized in that, Based on the first context information packet and preset prompt information, a preset large language model is invoked to extract information from the marked target code module to obtain second extracted information, including: The first context information packet and the first preset prompt information are input into the preset large language model to analyze the code module body of the marked target code module, and the output code module body analysis information is obtained; the first preset prompt information is: prompt information or instruction used to guide the preset large language model to extract the information of request parameters, parameter types and parameter sources contained in the code module body of the marked target code module. The second context information packet and the second preset prompt information are input into the preset large language model to extract information from the annotations of the marked target code module, and the output annotation analysis information is obtained; the second preset prompt information is a prompt or instruction used to guide the preset large language model to extract the parameter descriptions and summaries contained in the annotations of the marked target code module. Based on the code module body analysis information and the annotation analysis information, the second extraction information is obtained.

5. The method according to claim 3, characterized in that, Based on the first context information packet and preset prompt information, a preset large language model is invoked to extract information from the marked target code module to obtain second extracted information, including: The second context information packet and the third preset prompt information are input into the preset large language model to extract information from the marked target code module, and the second extraction information is output. The third preset prompt information is a prompt or instruction used to guide the preset large language model to extract the request parameters, parameter types, parameter sources and summaries contained in the marked target code module.

6. The method according to claim 2, characterized in that, The step of integrating the first extracted information and the second extracted information to obtain the information extraction result of the file to be processed includes: Traverse each application programming interface endpoint. If the code module corresponding to the application programming interface endpoint is a marked target code module, merge the second extracted information and the first extracted information of the code module corresponding to the application programming interface endpoint to obtain merged information. The merged information is formatted and fields are filled to obtain the information extraction result of the file to be processed.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: For each file to be processed, determine whether the file has been modified. If the file to be processed is modified, an abstract syntax tree corresponding to the source code of the file to be processed is generated; If the file to be processed has not been modified, obtain the information extraction results corresponding to the stored file to be processed.

8. The method according to claim 1, characterized in that, After generating the first context information packet, the method further includes: Query the correspondence between the context information packet and the extracted information to determine whether the first context information packet corresponds to extracted information; If there is extraction information corresponding to the first context information packet, the extraction information corresponding to the first context information packet is obtained as the second extraction information; If there is no corresponding extraction information in the first context information packet, the system performs information extraction on the marked target code module by calling a preset large language model based on the first context information packet and preset prompt information, and obtains the second extraction information.

9. An information extraction system, characterized in that, The system includes: The code acquisition module is used to acquire the source code of the file to be processed. A syntax tree generation module is used to generate an abstract syntax tree corresponding to the source code. The first information extraction module is used to extract information from the abstract syntax tree to obtain first extracted information; the first extracted information includes path information of at least one target code module and first parameter information of the target code module; the first parameter information includes parameter name and parameter type; the target code module is used to perform a specified task or implement a specified function, and the target code module is the code module corresponding to the application programming interface endpoint in the abstract syntax tree; The tagging module is used to tag target code modules whose parameter types are preset parameter types; The second information extraction module is used to generate a first context information package for each marked target code module based on the source code of the marked target code module and the variable name of the object corresponding to the target parameter type; and to extract information from the marked target code module by calling a preset large language model based on the first context information package and preset prompt information to obtain second extracted information; the second extracted information includes the second parameter information of the marked target code module; the target parameter type is the parameter type that is a preset parameter type in the first parameter information of the marked target code module; The data integration module is used to integrate the first extracted information and the second extracted information to obtain the information extraction result of the file to be processed.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-8.

Images