Curiosity Agents for Automated Game Environment Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for testing and validating video games, particularly those using automated learning agents, often fail to thoroughly explore virtual interactive environments due to limitations in navigation meshes and reinforcement systems, leading to untested aspects of the game.
Innovation Solution
The deployment of curiosity agents in video games, which utilize reinforcement learning models to explore environments novelly, store exploration data in buffers, and adjust spawn points based on this data to maximize coverage, ensuring unexplored areas are investigated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated learning agents are used for testing video games, then testing efficiency is improved, but exploration coverage deteriorates due to navigation mesh limitations
Solution Approach 1:
The patent divides the virtual environment into discrete locations and uses a visitation state buffer to track explored areas. The curiosity agent segments the exploration space by maintaining a set of unexplored locations and systematically targeting them, thereby achieving complete coverage without being constrained by traditional navigation mesh limitations.
Solution Approach 2:
The patent implements feedback mechanisms where the curiosity agent receives information about visited locations from the visitation state buffer and adjusts its navigation accordingly. The agent uses this feedback to identify and target unexplored locations, ensuring systematic coverage of the entire virtual environment while maintaining efficient automated testing.
2Speed
If reinforcement systems are used to guide agents, then navigation efficiency is improved, but ability to discover novel actions deteriorates
Solution Approach 1:
Instead of using reinforcement systems to guide the agent toward predetermined goals or paths, the patent inverts the approach by having the curiosity agent actively seek out unexplored locations based on the visitation state buffer. This inversion enables the agent to discover novel actions and paths that were not anticipated by traditional reinforcement systems, while still maintaining efficient navigation through systematic targeting of unexplored areas.
Solution Approach 2:
The curiosity agent autonomously determines its own navigation targets by querying the visitation state buffer to identify unexplored locations. The agent self-manages its exploration strategy without external guidance, allowing it to naturally discover novel actions and adapt to the environment while maintaining efficient navigation through its own decision-making processes.
3Reliability
If manual testing is used, then thorough validation is achieved, but time consumption increases
Solution Approach 1:
The curiosity agent autonomously performs comprehensive validation of the virtual environment by independently identifying and exploring unexplored locations using the visitation state buffer. This self-service approach achieves thorough validation comparable to manual testing while eliminating the time consumption associated with human testers, as the agent operates continuously and systematically covers the entire environment.
Solution Approach 2:
The patent replaces the mechanical system of manual human testing with an automated curiosity agent that uses algorithmic decision-making based on the visitation state buffer. This substitution maintains the thoroughness of validation by systematically exploring all locations while dramatically reducing time consumption through automated, continuous operation without human intervention.
Data Source
AI summary
Systems and methods of curiosity driven reinforcement learning agents promote novel exploration of a virtual interactive environment. The data of the exploration can be stored in a buffer to determine, generate, and display visualizations in the virtual interactive environment. The visualizations can correspond to identify issues in the virtual interactive environment and/or identify relationships between regions of the virtual interactive environment.