Fuzz testing autonomous systems

Abstract

A method consisting of obtaining a definition of interestingness 233 and a plurality of test cases suitable for fuzz testing an autonomous system 234 and providing them to a large language model (LLM) 231 which then outputs 238 a score or ranking of the test cases based on the definition of interestingness. The method may further comprise providing the LLM with: information on the state of the system separately from the test scenario, the output format of the LLM, and a methodology for ranking or assigning scores. The LLM may output an explanation of its rankings / scores which may be done by chain-of-thought prompting. The methodology may involve carrying out fuzz testing using selected test cases 239. The test cases may be obtained by a test case generator 220 which may consist of modifying some seed case 222 to obtain a mutated seed case 223. A test case analyser 230 for doing the same is also disclosed. [Figure 2]