Reinforcement learning systems and methods for inventory control and optimization

Abstract

A method of reinforcement learning for a resource management agent in a system for managing an inventory of perishable resources having a sales horizon, while seeking to optimize revenue generated therefrom. The inventory has an associated state. The method comprises generating a plurality of actions. Responsive to the actions, corresponding observations are received, each observation comprising a transition in the state associated with the inventory and an associated reward in the form of revenues generated from sales of the perishable resources. The received observations are stored in a replay memory store. A randomised batch of observations is periodically sampled from the replay memory store according to a prioritised replay sampling algorithm wherein, throughout a training epoch, a probability distribution for selection of observations within the randomised batch is progressively adapted. Each randomised batch of observations is used to update weight parameters of a neural network that comprises an action-value function approximator of the resource management agent, such that when provided with an input inventory state and an input action, an output of the neural network more closely approximates a true value of generating the input action while in the input inventory state. The neural network may thereby be used to select each of the plurality of actions generated depending upon a corresponding state associated with the inventory.

Description

technical field [0001] The present invention relates to technical methods and systems for improved inventory control and optimization. In particular, embodiments of the present invention employ machine learning techniques, particularly reinforcement learning, in the implementation of an improved revenue management system. Background technique [0002] Inventory systems are employed in many industries to control the availability of resources, for example through pricing and yield management and any associated calculations. Inventory systems enable customers to purchase or reserve available resources or goods from providers. Additionally, inventory systems allow providers to manage available resources and maximize revenue and profits by making these resources available to customers. [0003] In this context, the term "revenue management" refers to the application of data analytics to predict consumer behavior and optimize product offerings and pricing to maximize revenue gro...

AI Technical Summary

Problems solved by technology

Furthermore, conventional RMS models treat markets as monopolies under the assumption that competitors' actions are implicitly considered in customer behavior

[0007] Another disadvantage of the conventional approach to RMS is that there are often interdependencies between the model and its inputs such that any change in the available input data requires the model to be modified or rebuilt to take advantage of the new or changed information

Furthermore, without human intervention, the modeled system is slow to respond to changes in requirements that are poorly or not represented in the historical data on which the model is based

Images