Efficient image dividing method and system for larger image calculation

Abstract

The invention provides an efficient image dividing method and system for larger image calculation. The method includes the steps: dividing image data into a plurality of vertexes, and randomly sortingthe vertexes into a queue; performing partition distribution on the first vertex according to a queue sequence, namely, distributing the first vertex to a processing unit, taking partition information of the first vertex as a value, taking adjacent vertexes of the first vertex as keys, and storing the information into a DRAM (dynamic random access memory) or NVM (non-volatile memory) in a dictionary entry mode after distribution; judging whether entries taking the subsequent vertexes as keys are present in the DRAM or NVM or not, and directly adding the partition information of the vertexes to the corresponding entries in the DRAM or NVM if the entries taking the subsequent vertexes as the keys are present in the DRAM or NVM; distributing the vertexes to the processing unit with minimum load if not. Partition information of each distributed vertex serves as a value, the adjacent vertexes of the vertex serve as keys, and the vertex is stored into a corresponding buffer memory in the dictionary entry mode. According to the method, the corresponding entry taking a vertex as the key can be directly searched according to the current vertex every time, and efficiency is improved.

Description

technical field [0001] The invention relates to the field of computers, in particular to a method and system for large-scale computing and high-efficiency graph division. Background technique [0002] At present, the scale of graphs is huge and growing. For example, a graph composed of brain nerves can reach hundreds of terabytes. The most typical is the World Wide Web. Search engines can capture about 1 trillion link graphs. According to It is estimated that the future scale will exceed ten trillion. The world's largest social network Facebook currently has about 1 billion users, corresponding to tens of billions of relationship links. Ordinary computers cannot process these graphs (large graphs) normally due to memory limitations, which poses severe challenges to common graph calculations (such as finding connected components, calculating triangles and Pagerank). A standard solution is to divide graph data into multiple subgraphs and load them into different processing u...

AI Technical Summary

Problems solved by technology

At the same time, NVM is similar to DRAM in terms of access delay and read and write performance, and its static power consumption is lower than that of DRAM. The graph division is a read and write intensive calculation. Due to the poor write performance of NVM, high write power consumption and limited number of write operations, If NVM is directly used as memory, the service life of NVM will be directly affected due to high-frequency write operations

Images