METHODS AND SYSTEMS FOR DETECTING SECURITY VULNERABILITIES BASED ON INTERACTIVE GRAPHS FOR CHATBOT APPLICATIONS USING LARGE-SCALE LANGUAGE MODELS

Abstract

The invention proposes a method and system for detecting security vulnerabilities in chatbot applications using a large language model (LLM) running on a web platform. The method includes the following steps: attack surface reconnaissance by automatically identifying endpoints, parameters, and information of the LLM model through source code and HTTP traffic analysis; constructing an attack plan in the form of an exploitation graph consisting of task nodes and dependency relationships; sequentially executing tasks on the graph, while using a self-correcting mechanism based on Retrieval Augmented Generation (RAG) combined with the K-Nearest Neighbors (KNN) algorithm to modify statements upon failed execution; validating the results of each task using the LLM model; and quantifying the severity of the vulnerability through an LLMVSS scoring system.The invention also proposes a computer system configured to perform a vulnerability detection method composed of reconnaissance, graph management, execution, knowledge store, and evaluation modules. The invention automates the security testing process, increases coverage of multi-step attacks, reduces reliance on testing experts, and provides a risk assessment mechanism suitable for chatbot systems using LLM.

Images