Virtual memory management method and virtual memory management device for mass data processing

Abstract

The invention discloses a virtual memory management method and a virtual memory management device for mass data processing, belongs to the technical field of computer system optimization and is mainly applied to one-step real-time processing of computer mass data in quantity higher than that of available physical memory. When memory allocation reaches an assigned critical value, the most suitable physical memory is found to substitute for disc virtual memory according to memory block access correlation composite index and memory allocation application size. By a memory allocation method based on size matching, a front segment of an allocation unit is allocated to an object having high requirement for memory while a rear segment is allocated to an object having low requirement for memory; thus transform matching degree of the virtual memory is increased while data movement amount is reduced. By the virtual memory management method, during allocation of mass memories which are of capacity ranging from 100KB to 10MB and of the total quantity higher than the available physical memory, Thrashing in scheduling of the virtual memory is decreased, scheduling is accurate and predictable to some degree, and system performance is improved remarkably.

Description

technical field [0001] The invention belongs to the technical field of computer system optimization, and in particular relates to a virtual memory management method in big data processing and a device thereof. Background technique [0002] Although computer hardware develops rapidly and memory capacity increases rapidly, from KB level to MB level, and then to GB level, a level increases by a thousand times, but the demand is also constantly increasing. Now that we have entered the era of big data processing, no amount of memory configuration can meet all application requirements. On the other hand, a large-capacity memory configuration also means higher hardware investment, and people always hope that a computer with a general configuration can also process a large amount of data. [0003] Compared with the capacity and cost of memory, disk has the characteristics of low cost and large capacity. Under this background, virtual memory technology has emerged. Virtual memory r...

AI Technical Summary

Problems solved by technology

However, the virtual memory management of the operating system cannot optimally handle this situation. The low efficiency of virtual memory scheduling, frequent call-in and call-out, and excessive data movement will cause this type of big data processing to be performed in the case of a large amount of data. , which is very slow and cannot even handle

Images