Reinforcement Learning Agents in Dynamic Resource Matching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Machine learning agents trained on historical data fail to effectively capture the dynamic and interactive nature of environments like electronic trading, where resources are competed for, leading to inadequate responses to dynamic changes in resource availability.

Innovation Solution

A computer-implemented system integrating a reinforcement learning process with a matching engine to create a simulation environment, where reinforcement learning agents can interact and compete for resources, using a resource generating agent to simulate dynamic resource tasks and provide feedback for improved training.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If machine learning agents are trained on historical data, then training data availability is improved, but the ability to capture dynamic and interactive nature of trading environments deteriorates

Engineering Contradiction:

Improvetraining data availabilityVSAvoidresponse to dynamic changes

Core Design Contradiction:

Quantity of substanceVSAdaptability or versatility

Solution Approach 1:

The system transitions from static historical data to a dynamic simulation environment where resource availability changes over time based on agent actions. The matching engine continuously updates resource states, creating a living training environment that adapts to agent behaviors rather than remaining fixed like historical datasets.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

A simulation environment is introduced as an intermediary between historical data and real trading. This simulation layer captures the dynamic and interactive nature of trading environments by modeling resource competition, while still allowing agents to be trained on structured data that reflects realistic market conditions.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If a simulation environment with multiple agents competing for resources is created, then the dynamic and interactive nature of trading environments is captured, but system complexity increases

Engineering Contradiction:

Improvecapture of dynamic natureVSAvoidsystem structure

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system is divided into distinct modular components: the matching engine that handles resource allocation, the resource generating agent that manages resource availability, and multiple reinforcement learning agents that compete for resources. This segmentation allows each component to be independently developed, tested, and optimized while maintaining overall system functionality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of directly implementing a full trading exchange, the system creates a simplified copy or abstraction of the trading environment. The matching engine replicates the essential resource competition dynamics of real markets without including all the complexity of actual trading infrastructure, allowing agents to learn dynamic interactions in a controlled setting.

Inventive Principle:

Principle #26Copying

3Reliability

If reinforcement learning agents compete for resources in a simulation, then learning quality improves, but training time and computational resources increase

Engineering Contradiction:

Improvelearning qualityVSAvoidtraining duration

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The simulation environment enables continuous training of reinforcement learning agents through uninterrupted interaction with the dynamic resource system. Multiple agents can train simultaneously in parallel, maximizing utilization of computational resources and reducing overall training time compared to sequential training approaches.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The system allows reinforcement learning agents to self-train through autonomous interaction with the simulation environment. Agents independently learn optimal strategies by competing for resources and receiving feedback from the matching engine, eliminating the need for manual intervention or curated training datasets, thereby improving learning quality while managing computational efficiency.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20250013496A1Trade platform with reinforcement learning network and matching engine

Publication Date: 2025.01.09 ROYAL BANK OF CANADA

AI summary

A system for reinforcement learning in a dynamic resource environment includes at least one memory device and at least one processor configured to provide an electronic resource environment comprising: a matching engine and the resource generating agent configured for: obtaining from a historical data processing task database a plurality of historical data processing tasks, each historical data processing task including respective task resource requirement data; for a historical data processing task of the plurality of historical data processing tasks, generating layers of data processing tasks wherein a first layer data processing task has an incremental variant in its resource requirement data relative to resource requirement data for a second layer data processing task; and providing the layers of data processing tasks for matching by the machine engine.