Computer System For Data Manipulation And Related Methods

Abstract

A computer system (100) for data management arranged to provide at least four components: a generator component (202), a solver component (204), a simulation component (206) and a visualisation component (208), wherein the simulation component (206) is arranged to simulate stochastic processes and output simulation data of the simulated process to the generator component (202); the generator component (202) is arranged to process a model input thereto and to generate a Dynamic Stochastic Program (DSP) therefrom and also to process the simulation data received from the simulation component and output the DSP together with the processed simulation data to the solver component (204); the solver component being arranged to receive the DSP and the processed simulation data and to process the DSP together with the simulation data; and the visualisation component (208) being arranged to allow the data being processed by the solver to be visualised.

Description

FIELD OF THE INVENTION [0001] This invention relates to a computer system for data manipulation and in particular, but not exclusively, to a computer system arranged to manipulate data to ascertain future trends. Related methods are also provided. BACKGROUND OF THE INVENTION [0002] Linear, and more generally mathematical, programming optimization techniques have been in steady commercial use on the computing systems of the day since the mid 1950s. Sometime in the mid 1970s, independently at both the US Energy Department and the World Bank, the concept of a modelling language for mathematical programming arose. [0003] Representation of a model in such a language is to the numerical optimization of a mathematical program the analogue of what a database management system schema is to a flat file. It is straightforward for the analyst to turn a mathematical description of a model into statements in a modelling language which then communicates directly with the solver—optimization routin...

AI Technical Summary

Benefits of technology

[0013] An advantage of such a method is that it can improve the efficiency and effectiveness of Dynamic Stochastic Programming (DSP) for real-world planning problems running on equivalent hardware to prior solutions. The components may resolve the hitherto intractable issues of problem size and complexity by supporting stages of an iterative process of finding a robust solution with an integrated set of tools that are specifically designed for each modelling task: simulation, generation, optimization, and visualisation.

[0030] The Stochastics Suite includes several solvers that exploit the nodal structure of the DSP, as expressed in the programming language and in the separate scenario tree structure. These solvers achieve dramatic increases in the speed of solving large problems: ten to one hundred times faster than other general-purpose solvers. Not only does this allow real-world problems to be solved in acceptable times, it facilitates solving complex problems that are otherwise insoluble using other solvers.

Problems solved by technology

However, solving such problems is computationally intensive, making it nigh impossible to solve realistic problems in acceptable time.

The present difficulty of the method stems from a number of issues with current DSP tools.

It has therefore been a problem to solve such problems involving uncertainty, now and in the future, in an acceptable time or at all.

There is an issue of how to represent uncertain future states of the world.

Here lies the problem: too few scenarios and the solution is not robust, too many and the problem cannot be solved in reasonable time.

This is particularly problematic for DSP in which there are multiple time-steps and the number of scenarios grows geometrically.

The second issue is the choice of a suitable solver: one that is adapted to the nature of the problem.

General-purpose solvers attempt to map the entire DEP into memory, which constrains the practitioner to unrealistically simple DSP's or large amounts of memory and computing power would be required to solve DEP of more complexity.

The problem thus arises as to how a realistic problem can be solved in reasonable time with the computing power that it is currently readily available.

Images