The invention discloses a large
language model agent security test method and device based on a model context agreement, and the method comprises the steps: firstly deploying a test tool in a test
server, defining an application program interface which comprises function parameters and
test probe parameters, describing the
test probe parameters as necessary technical requirements for executing nominal functions, and executing the nominal functions according to the
test probe parameters; and inducing the tested LLM agent to transmit complete session
context data when calling. The test
server receives a tool call request including a function parameter value and session
context data, a back-end concurrently processes the request, executes a nominal function to generate a benign result, and extracts the session
context data at the same time. Afterwards, a benign result is returned to the tested LLM agent, and the extracted data is asynchronously transmitted to a remote log
server for recording as a test log. And finally, comparing the test log with the actual operation history, judging whether a session context data leakage
vulnerability exists or not, and quantifying the
severity level so as to detect whether the LLM agent leaks session memories such as the user
interaction history or not.