Reinforcement Learning Agent for Software License Usage Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Organizations face challenges in determining the appropriate number of software licenses to acquire, as users often remain logged in without actively using the software, making it difficult to estimate the need for floating licenses and leading to inefficient resource allocation and increased costs.
Innovation Solution
A reinforcement learning-based method is employed to evaluate software product usage by analyzing key performance indicators, determining login states, and adjusting license types (permanent or floating) based on expected utility scores, allowing for more accurate assessments and efficient license management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If users remain logged in to the software product, then license availability is maintained, but inaccurate license usage estimation occurs and costs increase
Solution Approach 1:
The system continuously monitors user interactions with the software product and feeds this information back to the reinforcement learning agent. The agent uses this feedback to update its policy for determining active vs. inactive users, thereby improving license usage estimation accuracy over time while preventing license waste from falsely logged-in users.
Solution Approach 2:
The reinforcement learning agent autonomously determines whether users are actively using the software based on monitored interaction patterns. The system self-adjusts license allocation decisions without manual intervention, automatically identifying inactive users and freeing their licenses while maintaining accurate usage estimates.
2Measurement precision
If reinforcement learning is used to evaluate login states, then license allocation accuracy improves, but system complexity increases
Solution Approach 1:
The reinforcement learning agent acts as an intermediary layer between raw user interaction data and license allocation decisions. This mediator processes complex patterns of user behavior and translates them into simple login state determinations, achieving high measurement precision while managing system complexity through a dedicated specialized component.
Solution Approach 2:
The system monitors changes in user interaction parameters over time to determine active vs. inactive states. By tracking multiple parameters (interaction frequency, types of actions, time patterns) and their changes, the system achieves accurate detection without requiring overly complex infrastructure, leveraging parameter evolution as the key differentiator.
3Productivity
If manual monitoring of software usage is performed, then implementation simplicity is maintained, but productivity and accuracy decrease
Solution Approach 1:
The system automatically monitors user interactions and performs license allocation decisions without manual intervention. The reinforcement learning agent continuously evaluates usage patterns and self-manages license distribution, dramatically improving productivity while eliminating the time-consuming manual evaluation process.
Solution Approach 2:
The reinforcement learning agent operates continuously, constantly monitoring user interactions and adjusting license allocations in real-time. This continuous automated operation eliminates idle time between manual evaluations and ensures license optimization is an ongoing process rather than a periodic task, significantly improving overall efficiency.
Data Source
AI summary
Techniques are provided for reinforcement learning-based evaluation of software product usage. One method comprises obtaining key performance indicators indicating software product usage by a user; determining, for a predefined time window: (i) a mean and/or a median of the obtained KPIs; (ii) an amount of time that the software product was active; and (iii) an amount of interactions by the user with a user interface; evaluating possible login states of the software product using at least one reinforcement learning agent, wherein the evaluating comprises (a) observing the plurality of possible login states, including a current state comprising a current login state of the software product, and (b) obtaining an expected utility score for changing from the current login state to a different login state of the software product; and determining whether to change from the current login state to a different login state of the software product based on the expected utility score.