Static defect analysis method and system for large language model-based agent development framework

Abstract

This invention provides a static defect analysis method and system for an agent development framework based on a large language model. The method includes: acquiring the agent development framework to be analyzed and the corresponding downstream applications; extracting the mapping relationship between modules and functions in the agent development framework from its technical documentation and constructing a module-function feature table; acquiring defect-related text describing the defects of the agent development framework in natural language, extracting initial semantic features reflecting the text's thematic structure, and inputting the defect-related text and the initial semantic features into a large language model after vectorization encoding to extract structured defect problems; constructing an abstract syntax tree for each downstream application corresponding to the agent development framework, identifying the actual function calls in each downstream application through the abstract syntax tree, locating the functions and modules associated with the actual defects, and generating static analysis results.

Description

Technical Field

[0001] This invention relates to the field of intelligent agent development technology, and in particular to a static defect analysis method and system for intelligent agent development frameworks based on large language models. Background Technology

[0002] Large language model-driven intelligent agent systems are widely used in task automation, intelligent business process orchestration, and complex decision execution. Intelligent agent development frameworks typically integrate core modules such as tool invocation, structured output, memory management, and multi-tool interfaces, enabling developers to build intelligent applications in a highly abstract manner and deploy them extensively in scenarios such as intelligent customer service, data analysis, and knowledge services. However, existing large language model-based intelligent agent development frameworks have the following drawbacks: 1) Developer feedback in the community is diverse and semantically ambiguous, lacking module mapping relationships; 2) Manual analysis struggles to efficiently extract typical defect patterns, making it difficult to comprehensively revise the intelligent agent development framework.

[0003] Therefore, designing a method for developing intelligent agents based on large language models that can comprehensively identify security risks and solve dependency analysis problems is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a static defect analysis method and system for an intelligent agent development framework based on a large language model, in order to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a static defect analysis method for an agent development framework based on a large language model. The method includes the following steps: obtaining the agent development framework to be analyzed and the corresponding downstream applications; extracting the mapping relationship between modules and functions in the agent development framework from its technical documentation, and constructing a module-function feature table based on the mapping relationship; obtaining defect-related text describing the defects of the agent development framework in natural language, extracting initial semantic features reflecting the text's thematic structure from the defect-related text, and inputting the defect-related text and the initial semantic features into a large language model after vectorization encoding to extract structured defect problems; the structured defect problems include defect types and defect-associated functions; constructing an abstract syntax tree for each downstream application corresponding to the agent development framework, identifying real function calls in each downstream application through the abstract syntax tree, matching the identified real function calls with the module-function feature table to locate real defect-associated functions and modules in the downstream applications, and then analyzing the call path of the real defect-associated functions to generate static analysis results including defect types, real defect-associated functions, real defect-associated modules, and the call path of the real defect-associated functions.

[0006] In some embodiments of the present invention, the step of extracting the mapping relationship between modules and functions in the intelligent agent development framework from the technical documentation of the intelligent agent development framework includes: extracting function definitions, module interfaces, parameter types and return value information from the technical documentation, extracting and cross-validating the functions and calling methods actually supported by the intelligent agent development framework in each module, thereby obtaining the mapping relationship between modules and functions in the intelligent agent development framework.

[0007] In some embodiments of the present invention, the step of extracting initial semantic features reflecting the text topic structure from the defect-related text includes: preprocessing the defect-related text, the preprocessing including word segmentation, stop word removal, removal of correction-based prompts, and removal of invalid links; constructing a word co-occurrence graph based on the preprocessed defect-related text according to the TextRank algorithm; iteratively calculating the node weights in the word co-occurrence graph, and selecting keywords and key phrases with weights higher than a preset threshold to constitute the initial semantic features.

[0008] In some embodiments of the present invention, the step of inputting the defect-related text and the initial semantic features into a large language model after vectorization encoding to extract the structured defect problem includes: using the defect-related text and the initial semantic features as input to the large language model after vectorization encoding; using the large language model to identify implicit defect semantics through cross-sentence reasoning; uniformly mapping the implicit defect semantics to a consistent defect type; using the large language model to extract abnormal triggering patterns, affected components, and contextual conditions; semantically matching the initial semantic features with the module-function feature table; establishing potential associations between defect types and functions and modules in the agent development framework by calculating vector similarity and / or nearest neighbor retrieval in the embedding space; identifying defect association functions corresponding to defect types; and constructing the structured defect problem from the defect types and the defect association functions.

[0009] In some embodiments of the present invention, the step of identifying the actual function calls in each downstream application through the abstract syntax tree includes: parsing the module import statements and function call expressions of the downstream application through the abstract syntax tree to identify the actual function calls in the downstream application.

[0010] In some embodiments of the present invention, the step of locating the real defect association function and the real defect association module in the downstream application, and then analyzing the call path of the real defect association function, specifically includes: constructing a function-level and module-level dependency graph based on the real defect association function and the real defect association module; analyzing the call path and upstream and downstream dependency structure of the real defect association function based on the function-level and module-level dependency graph; and then identifying the associated dependency links and potential cross-module impact range.

[0011] In some embodiments of the present invention, the static analysis results further include defect-related semantic fields described in natural language; after the step of analyzing the call path of the real defect association function, the method further includes: extracting defect-related semantic fields described in natural language from the code context of the downstream application.

[0012] In some embodiments of the present invention, the method further includes: determining, through a large language model, whether the defect-related semantic fields of the natural language description fully express the expected function and / or abnormal semantics, and checking whether there is a semantic conflict between role responsibilities, tool uses and task objectives, identifying semantic defects such as vague descriptions, incoherent logic or inconsistencies with code implementation, and then performing semantic enhancement on the defect-related semantic fields of the natural language description; after obtaining the static analysis results, the method further includes defect pattern recognition: determining the defect identification results of the intelligent agent development framework based on the static analysis results.

[0013] Corresponding to the above methods, the present invention also provides a static defect analysis system for an intelligent agent development framework based on a large language model, including a processor, a memory, and a computer program / instructions stored in the memory. The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of any of the methods described in the above embodiments.

[0014] In accordance with the above methods, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method as described in any of the above embodiments.

[0015] The static defect analysis method for intelligent agent development frameworks based on large language models proposed in this invention is designed with a combined defect identification process, which includes framework function extraction, defect-related text semantic parsing, and code structure analysis. It takes into account both the characteristics of natural language expression and actual calling behavior, and can generate comprehensive and well-structured defect analysis results. This can provide effective support for the quality diagnosis of intelligent agent development frameworks and help support the subsequent optimization of intelligent agent development frameworks.

[0016] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0017] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings: Figure 1 This is a flowchart of a static defect analysis method for an intelligent agent development framework in one embodiment of the present invention.

[0019] Figure 2 This is a diagram of the static defect analysis system architecture of the intelligent agent development framework in one embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the computer equipment included in the system. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0022] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0023] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0024] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0025] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0026] To overcome the problems of existing technologies, this invention proposes a static defect analysis method for intelligent agent development frameworks based on large language models. This method systematically maps and analyzes the specific modules and function calls of the intelligent agent development framework, and simultaneously covers comprehensive defect identification based on real-world usage and natural language expression, thereby improving the security of intelligent agent applications. Figure 1 This is a flowchart of a static defect analysis method for an intelligent agent development framework according to an embodiment of the present invention. The method includes the following steps: Step S110: Obtain the intelligent agent development framework to be analyzed, and obtain the downstream application corresponding to the intelligent agent development framework.

[0027] If static defect analysis is performed on multiple agent development frameworks simultaneously, the result is a set of agent development frameworks and a set of downstream applications. One agent development framework can correspond to one or more downstream applications. These agent development frameworks can be obtained from authoritative open-source communities such as GitHub. Candidate agent development frameworks can be retrieved by setting keywords (such as "agentframework", "LLM agent", and "multi-agent system"). Furthermore, quantitative filtering can be performed based on repository activity metrics (number of stars, contributor size, commit frequency, maintenance cycle, etc.).

[0028] Step S120: Extract the mapping relationship between modules and functions in the intelligent agent development framework from the technical documentation of the intelligent agent development framework, and construct a module-function feature table based on the mapping relationship between modules and functions.

[0029] The module-function feature table is constructed by collecting technical documents such as official documentation, technical podcasts, and example projects. It extracts and cross-verifies the functions and calling methods actually supported by the framework at each functional point (i.e., module), thereby storing accurate and reliable module-function mapping relationships in the module-function feature table.

[0030] Step S130: Obtain defect-related text describing the defects of the intelligent agent development framework in natural language, extract initial semantic features reflecting the text topic structure from the defect-related text, input the defect-related text and the initial semantic features into a large language model after vectorization encoding, and extract structured defect problems; the structured defect problems include defect types and defect association functions.

[0031] The defect-related text can be collected from open-source community resources (including Discussions, Issues, and Pull Requests) corresponding to the intelligent agent development framework. To address issues such as cross-module descriptions, terminology diversity, and repeated expressions of the same defect by different users in different ways, the defect-related text needs to be processed. The structured defect problem also includes key semantic fields and triggering features.

[0032] Step S140: Construct an abstract syntax tree for each downstream application corresponding to the intelligent agent development framework. Identify the real function calls in each downstream application through the abstract syntax tree. Match the identified real function calls with the module-function feature table to locate the real defect-related functions and modules in the downstream applications. Then analyze the call path of the real defect-related functions and generate static analysis results including defect type, real defect-related functions, real defect-related modules and the call path of the real defect-related functions.

[0033] Specifically, an Abstract Syntax Tree (AST) is constructed to comprehensively scan the code structure, function call chain, and module dependencies of downstream projects. This extracts the calling patterns and structured features of the intelligent agent development framework in real-world use, providing accurate foundational data for subsequent defect identification and pattern analysis. The calling patterns and structured features in real-world use are revealed through static analysis of the associated functions, modules, and call paths of these associated functions, which are linked to actual defects.

[0034] In some embodiments of the present invention, the defect type, defect association function, defect association module and the call path of the defect association function belong to structured defect information. The method also includes defect-related semantic fields, which belong to unstructured defect information.

[0035] The static defect analysis method for intelligent agent development frameworks based on large language models proposed in this invention is designed with a combined defect identification process, which includes framework function extraction, defect-related text semantic parsing, and code structure analysis. It takes into account both the characteristics of natural language expression and actual calling behavior, and can generate comprehensive and well-structured defect analysis results. This can provide effective support for the quality diagnosis of intelligent agent development frameworks and help support the subsequent optimization of intelligent agent development frameworks.

[0036] In some embodiments of the present invention, the step of extracting the mapping relationship between modules and functions in the intelligent agent development framework from the technical documentation of the intelligent agent development framework includes: extracting function definitions, module interfaces, parameter types and return value information from the technical documentation, extracting and cross-validating the functions and calling methods actually supported by the intelligent agent development framework in each module, thereby obtaining the mapping relationship between modules and functions in the intelligent agent development framework.

[0037] The intelligent agent development framework is generally composed of multiple modules. The above-mentioned module interface refers to the interface for accessing / calling the various modules that make up the intelligent agent development framework, and the function refers to the function used in the intelligent agent development framework.

[0038] In addition, you can analyze the configuration file examples provided in the official documentation to understand how to change the behavior, capabilities, and operating logic of the agent by modifying these parameters.

[0039] By employing this embodiment of the invention, the mapping relationship between modules and functions in the design of the intelligent agent development framework can be obtained by parsing technical documents and / or configuration file examples. Then, based on the module-function feature table, the real defect-related functions and real defect-related modules can be located, assisting in the static analysis of the intelligent agent development framework based on AST technology.

[0040] In some embodiments of the present invention, the step of extracting initial semantic features reflecting the text topic structure from the defect-related text includes: preprocessing the defect-related text, the preprocessing including word segmentation, stop word removal, removal of correction-based prompts, and removal of invalid links; constructing a word co-occurrence graph based on the preprocessed defect-related text according to the TextRank algorithm; iteratively calculating the node weights in the word co-occurrence graph, and selecting keywords and key phrases with weights higher than a preset threshold to constitute the initial semantic features.

[0041] By employing this embodiment of the invention, a semantic normalization and code association method that combines graph sorting with a large language model can be used to construct a word co-occurrence graph and calculate node weights in the graph. High-weight keywords and key phrases are then selected to effectively identify keywords and key phrases that are strongly related to defects, thereby extracting structured defect problems.

[0042] In some embodiments of the present invention, the step of inputting the defect-related text and the initial semantic features into a large language model after vectorization encoding to extract the structured defect problem includes: using the defect-related text and the initial semantic features after vectorization encoding as input to the large language model; using the large language model to identify implicit defect semantics through cross-sentence reasoning; uniformly mapping the implicit defect semantics to a consistent defect type; using the large language model to extract abnormal triggering patterns, affected components, and contextual conditions; semantically matching the initial semantic features with the module-function feature table; establishing potential associations between defect types and functions and modules in the agent development framework by calculating vector similarity and / or nearest neighbor retrieval in the embedding space; identifying defect association functions corresponding to defect types; and constructing the structured defect problem from the defect types and the defect association functions.

[0043] The implicit defect semantics described above may be expressed in different ways. Therefore, it is necessary to map the implicit defect semantics of different expressions to a consistent defect type. The defect type can be a tool execution anomaly such as "unstable tool behavior", "unresponsive function execution", or "action scheduling blocked".

[0044] In addition, step S130 may also include: generating a defect report based on the semantic matching results and defect types and the potential associations between the functions and modules in the intelligent agent development framework; semantically merging the semantic overlaps in the defect report; semantically completing the conceptual conflicts and ambiguous descriptions to obtain key semantic fields.

[0045] This invention enables the use of a large language model to further identify implicit defect semantics and uniformly map inconsistent implicit defect semantics to a consistent defect type. Furthermore, through semantic matching and potential association identification, the defect type and its related frame functions, frame modules, and key semantic fields can be determined. Additionally, the structured defect problem also includes triggering features corresponding to the defect type.

[0046] In some embodiments of the present invention, the step of identifying the actual function calls in each downstream application through the abstract syntax tree includes: parsing the module import statements and function call expressions of the downstream application through the abstract syntax tree to identify the actual function calls in the downstream application.

[0047] By employing this embodiment of the invention, it is possible to identify real function calls in downstream applications using abstract syntax trees, thereby providing accurate basic data for subsequent defect identification and pattern analysis.

[0048] In some embodiments of the present invention, the step of locating the real defect association function and the real defect association module in the downstream application, and then analyzing the call path of the real defect association function, specifically includes: constructing a function-level and module-level dependency graph based on the real defect association function and the real defect association module; analyzing the call path and upstream and downstream dependency structure of the real defect association function based on the function-level and module-level dependency graph; and then identifying the associated dependency links and potential cross-module impact range.

[0049] By employing this embodiment of the invention, the call path, dependency chain, and cross-module impact scope can be identified through static analysis of the dependencies between functions and modules, thereby enabling the adjustment of the agent development framework based on the static analysis results.

[0050] In some embodiments of the present invention, the static analysis results also include defect-related semantic fields described in natural language.

[0051] Accordingly, after the step of analyzing the call path of the real defect association function, the method further includes: extracting defect-related semantic fields in natural language description from the code context of the downstream application.

[0052] By employing this embodiment of the invention, semantic fields related to defects can be extracted for subsequent LLM processing. Through semantic verification and anomaly identification, the completeness of static analysis results can be improved.

[0053] In some embodiments of the present invention, the method further includes: determining, through a large language model, whether the defect-related semantic fields of the natural language description fully express the expected function and / or abnormal semantics, and checking whether there is a semantic conflict between role responsibilities, tool uses and task objectives, identifying semantic defects that are vague in description, logically incoherent or inconsistent with code implementation, and then performing semantic enhancement on the defect-related semantic fields of the natural language description.

[0054] By employing this embodiment of the invention, a large language model can be used to enhance the semantics of defect-related semantic fields in natural language descriptions, thereby improving the quality of defect-related semantic fields.

[0055] In some embodiments of the present invention, after obtaining the static analysis results, the method further includes defect pattern recognition: determining the defect recognition results of the intelligent agent development framework based on the static analysis results.

[0056] The defect identification results of the intelligent agent development framework include modules not being called effectively, abnormal calls to key functions, inconsistencies between tool semantics and role definitions, and abnormal output parsing behavior.

[0057] By employing this embodiment of the invention, the defects of the agent development framework can be analyzed based on the static analysis results of downstream applications, which helps to improve the agent development framework.

[0058] Figure 2 This is a system architecture diagram of a static defect analysis system for an agent development framework according to an embodiment of the present invention. The static defect analysis system of this agent development framework includes the following modules: 1) Data Acquisition and Preprocessing Module: Collects intelligent agent development frameworks and downstream applications, and performs screening, parsing and verification to form a high-quality data set.

[0059] 2) Functional Interface Modeling and Verification Module: By collecting official documentation, technical blogs and sample projects, we extract and cross-verify the functions and calling methods actually supported by the framework in each module, forming an accurate and reliable function-function mapping relationship.

[0060] 3) Defect semantic normalization and association module based on LLM: Using graph ordering and large language model, the community defect feedback is semantically standardized, type-unified and cross-representation merged, and the defect semantics are effectively associated with the module-function features of the framework.

[0061] 4) Static Analysis Module Based on AST: Based on AST technology, the module performs a comprehensive scan of the code structure, function call chain and module dependency relationship of the downstream project, extracts the calling pattern and structured feature information of the framework in real use, and thus provides accurate basic data for subsequent defect identification and pattern analysis.

[0062] 5) LLM-based semantic consistency verification module: Utilizes large language model analysis tools to describe the semantic and logical consistency of role definitions and task descriptions.

[0063] 6) Comprehensive Defect Identification Module: Combines static analysis results to identify cross-module call anomalies, semantic conflicts, and typical defect patterns, forming system-level defect analysis results.

[0064] Next, we will introduce each step of the static defect analysis of the intelligent agent development framework in conjunction with its specific implementation method.

[0065] First, candidate frameworks can be automatically retrieved from authoritative open-source communities such as GitHub based on keywords (e.g., "agent framework," "LLM agent," and "multi-agent system"), and then quantitatively filtered based on repository activity metrics (number of stars, contributor size, commit frequency, maintenance cycle, etc.). After filtering out irrelevant repositories and low-quality projects, a collection of representative intelligent agent development frameworks in the industry can be formed. ,in Indicates the first An independent open-source intelligent agent development framework.

[0066] In the collection of intelligent agent development frameworks Building upon existing methods, this invention identifies downstream projects of corresponding intelligent agent frameworks by analyzing the import path patterns, configuration file structures, and dependency files unique to each framework. To overcome the limitation of GitHub's REST API returning a maximum of 1000 projects per search, the `size` field in the repository metadata is used to divide all repositories into multiple fine-grained intervals based on size. A REST API search is then initiated for each size interval to obtain potential candidate repositories. Subsequently, import path matching is performed on the repositories returned for each interval to initially select downstream applications. The interval range is dynamically adjusted to ensure that the number of candidate repositories within each interval is less than the API return limit, thus achieving complete coverage of a massive number of repositories. Finally, the obtained repositories undergo configuration file verification and manual validation to select and eliminate projects that do not actually use the framework, ultimately forming a high-quality set of downstream applications. This set serves as the target project set for subsequent static analysis and defect identification based on AST. The first one directly related to a certain intelligent agent framework One downstream application project.

[0067] A collection of intelligent agent development frameworks Extract function definitions, module interfaces, parameter types, and return value information from technical documents (which may be official documents or technical white papers), parse configuration examples, and build a preliminary module-function feature table.

[0068] Next, we can further crawl technical blogs and example implementations, extract call examples and parameter descriptions, verify and supplement the differences between the official documentation and the actual implementation on GitHub, thereby constructing a complete module-function feature table. After completing the module-function feature modeling, we can then extract features from the intelligent agent development framework collection. Defect-related texts are automatically collected from the corresponding open-source community resources (including Discussions, Issues, and Pull Requests).

[0069] To address the issues raised in community feedback regarding cross-module descriptions, terminological diversity, and the repeated expression of the same defect by different users, a semantic normalization and code association method based on graph ranking and a large language model can be adopted. First, the collected text undergoes preprocessing, including word segmentation, stop word removal, removal of scripted prompts and invalid links, and a word co-occurrence graph is constructed based on the TextRank algorithm. High-weight keywords and key phrases are obtained through iterative calculation of node weights, forming an initial set of semantic features reflecting the text's thematic structure. Then, the vectorized encodings of the original text and keywords (and phrases) are input into the large language model. Implicit defect semantics are identified through cross-sentence reasoning, and different expressions are uniformly mapped to consistent defect types, such as "unstable tool behavior," "unresponsive function execution," and "blocked action scheduling"—tool execution anomalies. The system further extracts anomaly triggering patterns, affected components, and contextual conditions using a large language model. Keywords are then semantically matched with the aforementioned module-function feature table. Potential associations between defect types and specific functions or modules are established through vector similarity and / or nearest neighbor retrieval in the embedding space, thereby generating defect reports. Semantic overlaps in the defect reports are merged, and semantic completion is performed when conceptual conflicts or ambiguous descriptions occur across threads. Finally, a structured defect problem set is generated. Each It includes standardized defect types, key semantic fields, triggering features, and associated (or mapped) framework functions.

[0070] After completing module-function feature modeling and defect semantic normalization and correlation analysis, this invention utilizes a high-quality downstream application set. Using real engineering projects as the analysis object, the Abstract Syntax Tree (AST) technology is used to perform static scanning of their source code, extracting the call structure, dependency relationships and exception handling characteristics of the framework function interfaces in actual use scenarios, thereby providing a static analysis-level verification basis for the existing function-level correlation results in the defect problem set.

[0071] First, an Abstraction Framework (AST) is constructed for each downstream application project. Module import statements and function call expressions are parsed, and the identified function calls are matched against the module-function feature table to locate the defect problem set in the downstream project. The system already associates intelligent agent framework functions. Based on this, the number of calls and frequency of occurrence of defect-associated functions in different downstream projects are statistically analyzed, and the call frequency distribution at the function level and module level is obtained after normalization. Simultaneously, function-level and module-level dependency graphs are constructed based on the function call relationships extracted from the AST, focusing on analyzing the call paths of defect-associated functions and their upstream and downstream dependency structures to identify key dependency links and potential cross-module impact ranges. Further, exception handling statements, error branches, and log output locations are parsed within the function call context, and the exception handling structure is statistically correlated with the corresponding functions and call paths. This quantifies the distribution characteristics of defect-associated interfaces accompanying exception handling in real-world projects, thus forming a static analysis result enhanced with defect semantics. This result includes not only structured information such as the modules, functions, and call paths corresponding to identified defects located based on AST, but also defect-related semantic fields extracted from the code context and requiring subsequent semantic analysis via LLM.

[0072] Optionally, for the defect-related fields involving natural language descriptions extracted from the above static analysis results (including Tool descriptions, role definitions, task function descriptions, and anomaly information annotations, etc.), a large language model can be further introduced for semantic consistency and intent logic analysis. By performing semantic understanding on the extracted descriptive text through LLM, it is possible to determine whether it fully expresses the expected function or abnormal semantics, detect whether there are semantic conflicts between role responsibilities, tool uses, and task objectives, and identify semantic defects such as vague descriptions, incoherent logic, or inconsistencies with code implementation.

[0073] By employing this embodiment of the invention and leveraging the semantic enhancement capabilities of LLM, the problem that pure static structural analysis cannot provide in-depth explanations at the natural language level can be overcome, thereby obtaining more comprehensive and fine-grained module-level and project-level defect identification results.

[0074] Finally, based on the comprehensive static analysis results Combined with semantic enhancement analysis output, it forms the defect identification results of the intelligent agent development framework, which can discover typical problem patterns, such as modules not being called effectively, abnormal calls to key functions, inconsistencies between tool semantics and role definitions, and abnormal output parsing behavior, providing systematic, end-to-end data support for intelligent agent system architecture optimization, functional improvement and quality assessment.

[0075] Corresponding to the above method, the present invention also provides a static defect analysis system for an intelligent agent development framework based on a large language model. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0076] Figure 3 See the schematic diagram of the computer equipment included in the system. Figure 3 The computer device 00 includes: a processor 01, a memory 02, and a computer program stored on the memory 02 and executable on the processor 01. When the processor 01 executes the computer program, it implements the method steps proposed in any of the above embodiments.

[0077] The processor 01 is connected to the memory 02, such as via a bus 03. The processor 01 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 01 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The bus 03 may include a pathway for transmitting information between the aforementioned components. The bus 03 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 03 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The text uses only a single thick line to represent a bus, but this does not imply that there is only one bus or one type of bus. Memory 02 stores a computer program corresponding to the human factors data server access control method described in the above embodiments of this application. This computer program is executed under the control of processor 01. Processor 01 executes the computer program stored in memory 02 to implement the content shown in the aforementioned method embodiments.

[0078] Corresponding to the methods described above, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method as described in any of the above embodiments. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0079] Corresponding to the above methods, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any of the above embodiments.

[0080] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0081] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0082] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A static defect analysis method for an intelligent agent development framework based on a large language model, characterized in that, The method includes: Obtain the intelligent agent development framework to be analyzed, and obtain the corresponding downstream applications of the intelligent agent development framework; Extract the mapping relationship between modules and functions in the intelligent agent development framework from the technical documentation of the intelligent agent development framework, and construct a module-function feature table based on the mapping relationship between modules and functions; Obtain defect-related text describing the defects of the intelligent agent development framework in natural language; extract initial semantic features reflecting the text's topic structure from the defect-related text; input the defect-related text and the initial semantic features into a large language model after vectorization encoding to extract structured defect problems; the structured defect problems include defect types and defect association functions; An abstract syntax tree is constructed for each downstream application corresponding to the intelligent agent development framework. The real function calls in each downstream application are identified through the abstract syntax tree. The identified real function calls are matched with the module-function feature table to locate the real defect-related functions and modules in the downstream applications. Then, the call path of the real defect-related functions is analyzed to generate static analysis results including defect type, real defect-related functions, real defect-related modules, and the call path of the real defect-related functions.

2. The method according to claim 1, characterized in that, The step of extracting the mapping relationship between modules and functions in the intelligent agent development framework from the technical documentation of the intelligent agent development framework includes: Extract function definitions, module interfaces, parameter types, and return value information from the technical documents, extract and cross-verify the functions and calling methods actually supported by the intelligent agent development framework in each module, and thus obtain the mapping relationship between modules and functions in the intelligent agent development framework.

3. The method according to claim 1, characterized in that, The steps for extracting initial semantic features reflecting the text's topic structure from the defect-related text include: The defect-related text is preprocessed, including word segmentation, stop word removal, removal of proofreading prompts, and removal of invalid links. A word co-occurrence graph is constructed based on preprocessed defect-related text using the TextRank algorithm; The node weights in the word co-occurrence graph are calculated iteratively, and keywords and key phrases with weights higher than a preset threshold are selected to form the initial semantic features.

4. The method according to claim 1 or 3, characterized in that, The step of inputting the defect-related text and the initial semantic features into a large language model after vectorization encoding to extract the structured defect problem includes: The defect-related text and the initial semantic features are vectorized and encoded as input to the large language model. The large language model is used to identify implicit defect semantics through cross-sentence reasoning and to uniformly map the implicit defect semantics to a consistent defect type. Using a large language model, abnormal triggering patterns, affected components, and contextual conditions are extracted. The initial semantic features are semantically matched with the module-function feature table. By calculating vector similarity and / or embedding space nearest neighbor retrieval, the potential association between defect types and functions and modules in the agent development framework is established, and the defect association function corresponding to the defect type is identified. The structured defect problem is constituted by the defect type and the defect association function.

5. The method according to claim 1, characterized in that, The step of identifying the actual function calls in each downstream application through an abstract syntax tree includes: The abstract syntax tree is used to parse the module import statements and function call expressions of downstream applications to identify the actual function calls in the downstream applications.

6. The method according to claim 1, characterized in that, The steps of locating the actual defect correlation function and the actual defect correlation module in downstream applications, and then analyzing the call path of the actual defect correlation function, specifically include: Based on the real defect association functions and modules, construct function-level and module-level dependency graphs. Analyze the call paths and upstream and downstream dependency structures of the real defect association functions based on these graphs, thereby identifying the associated dependency links and potential cross-module impact ranges.

7. The method according to claim 6, characterized in that, The static analysis results also include defect-related semantic fields described in natural language. After the step of analyzing the call path of the function associated with the actual defect, the method further includes: extracting defect-related semantic fields in natural language description from the code context of the downstream application.

8. The method according to claim 7, characterized in that, The method further includes: using a large language model to determine whether the defect-related semantic fields of the natural language description fully express the expected function and / or abnormal semantics, and checking whether there is a semantic conflict between role responsibilities, tool uses and task objectives, identifying semantic defects such as vague descriptions, incoherent logic or inconsistencies with code implementation, and then performing semantic enhancement on the defect-related semantic fields of the natural language description. After obtaining the static analysis results, the method further includes defect pattern recognition: determining the defect recognition results of the intelligent agent development framework based on the static analysis results.

9. A static defect analysis system for an intelligent agent development framework based on a large language model, comprising a processor, a memory, and computer programs / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 8.

Images