A big data service system based on reinforcement learning

Abstract

The invention provides a big data service system based on reinforcement learning. The big data service system comprises an interaction layer, a service layer, a function layer, a storage layer, a sorting layer, a model layer and a buffer layer, wherein the interaction layer, the service layer, the function layer and the storage layer are arranged in the same server group, and the arrangement layer, the model layer and the buffer layer are arranged in another server group; according to the invention, the setting of different functions is operated based on the two sets of server groups, so thatthe overall cost of a hardware environment required for providing an operation reinforcement learning algorithm in a big data service system can be greatly reduced.

Description

technical field [0001] The invention relates to a big data service system based on reinforcement learning. Background technique [0002] At present, the architecture of big data service systems is generally designed for the main purpose of storing large amounts of data and batching simple processing. The development of reinforcement learning, especially deep reinforcement learning, has put forward new requirements for hardware. , A large amount of data storage and simple batch processing mainly have high requirements for the number of CPU cores and hard disk capacity, while reinforcement learning algorithms mainly have high requirements for such as GPU and memory. Reinforcement learning algorithms often require a large amount of data as support. Based on the data requirements of reinforcement learning algorithms, it is an ideal choice to run on a big data service system. The big data service system puts forward new hardware requirements. The common practice is to add GPU ...

AI Technical Summary

Problems solved by technology

Reinforcement learning algorithms often require a large amount of data as support. Based on the data requirements of reinforcement learning algorithms, it is an ideal choice to run on a big data service system. The big data service system puts forward new hardware requirements. The common practice is to add GPU and memory to some server hosts, and then implement deployment through additional functional module components, but this will lead to the development of additional functional module components on the one hand. The cost is greatly increased. On the other hand, the performance loss of the server will also be caused by the operation of the additional functional module components used for deployment. This part of the performance loss will inevitably be reflected in the hardware cost, resulting in an additional increase in the hardware cost of the service provider. The additional increase in hardware costs is often not solved by simply increasing the number of server hosts, and the increase in the number of server hosts also increases the cost of computer rooms, cooling equipment, etc.

Images