An Octree Grid Layer-by-Layer Load Balancing Method

Abstract

The octree grid layer-by-layer load balancing method provided by the present invention relates to a load balancing method. It decouples adaptivity and load balancing through two groups of octrees. One group stores all nodes of the octree in a tree-shaped data structure and is responsible for the adaptivity function of the octree; the other group stores the geometric information of the leaf nodes of the octree in a one-dimensional array and is responsible for the layer-by-layer load balancing function. The adaptivity and layer-by-layer load balancing of the octree are realized through the original octree and the actual octree. Moreover, the adaptivity function is not restricted, the parallel performance is not restricted, and it is completely distributed. Theoretically, there is no restriction on the parallel scale, and the parallel efficiency is high.

Description

Technical Field

[0001] The invention relates to a load balancing method, and in particular to an octree grid layer-by-layer load balancing method. Background Art

[0002] Octree is a tree data structure. The number of child nodes of each node (octant) in the tree must be eight or zero. Octree is mainly used for 3D space related operations, such as discretization, partitioning, traversal, search, etc. in 3D space. Octree allows these operations to be performed recursively. The concept of octree is closely related to 3D space. In 2D space, the tree data structure is represented by a quadtree.

[0003] (quadtree), which is represented as a binary tree in 1D space. Octrees are widely used in computational graphics (CG), computer vision (CV), and computer-aided engineering (CAE). In CAE, octrees can be used for spatial discretization. The figure shows the octree mesh generated after the computational domain is discretized. The characteristic of the octree is that its nodes can be encrypted and coarsened. One encryption means that a leaf node generates 8 child nodes, all of which are leaf nodes, and the node itself becomes an ordinary node. Coarsening means that eight leaf nodes with a common parent node are deleted, and the parent node changes from an ordinary node to a leaf node. The encryption and coarsening function of the octree has an important application in mesh division, which can make the mesh units efficiently distributed in space.

[0004] Distributed mainly refers to distributed memory parallel computers. Programs running on distributed memory computers are called distributed programs, and the corresponding algorithms are called distributed parallel algorithms. Distributed octrees refer to octree algorithms or programs that run on distributed memory computers, are a complete octree as a whole, and have octree functions. When octrees are applied to certain fields that rely on large-scale parallel computing, octrees must be distributed. At this time, in order to be able to be large-scale parallel, the algorithms and data involved in the program are distributed. For example, in numerical simulations, in order to achieve higher accuracy, the number of grid units generated by spatial discretization is extremely large, and a single computer node is not enough to accommodate the grid and field data. At this time, the program must run in parallel on multiple computer nodes. At this time, the octree used for spatial discretization must also be distributed, distributed on each node of the computer, and each node only retains a part of the octree and cannot overlap. Distributed octrees bring about load balancing problems.

[0005] Load balancing means that the data and task load on each node of a distributed parallel computer are relatively balanced, so that the operation of parallel programs can achieve higher efficiency. The reason is similar to the "barrel effect". For distributed octrees, load balancing means that the number of octree leaf nodes retained by each computer node is as balanced as possible. The octree data structure has a significant layered characteristic, and each node is on a specific layer. Reflected on the grid, grids of the same scale belong to the same layer. Layer-by-layer load balancing means that the leaf nodes (note that they are leaf nodes) of each layer of the octree meet load balancing, and the overall load balancing is also met. The reason for layer-by-layer is that for some numerical calculation algorithms, due to the dependency between grid layers during traversal calculations, traversal must be performed layer by layer. For example, the current coarse layer can only be updated after the next fine layer is updated. At this time, the distribution of each layer of the octree on each node of the computer must be load balanced.

[0006] Adaptation means that the octree is dynamically encrypted and coarsened according to the feedback of the distribution of a certain characteristic quantity in space. For example, in numerical simulation of computational fluid dynamics, the grid in the area with large gradient in the calculation domain is encrypted. Because the large gradient means that the change here is drastic, more grids are needed to capture the details. The grid in the area with small gradient may have honors, so it should be coarsened to save calculations. Adaptation means that the encryption and coarsening of the grid are carried out alternately with the evolution process of the field. Since the octree can adapt, the evolution of the field is more accurate. The octree is adaptively encrypted according to the feedback of the field evolution process. The adaptability of the grid is crucial to the accuracy of the numerical calculation results. The current "adaptation" mainly refers to encryption and coarsening.

[0007] Layer-by-layer load balancing and adaptive coupling, that is, considering the distribution of leaf nodes at the same time when encrypting and coarsening, makes the complexity of the problem rise sharply. In the existing technology, it is difficult to achieve both at the same time. The load balancing that can be layered cannot be encrypted and coarsened, and static grids are used; the load balancing with adaptive encryption function does not have the layer-by-layer characteristics, resulting in poor performance in problems with inter-layer dependencies. The existing technology status is divided into three types: ① Free adaptation is realized and layer-by-layer load balancing is abandoned. It is only applicable to top-level algorithms with no inter-layer dependencies. For algorithms with inter-layer dependencies, there are a large number of processes waiting in the parallel process, and the parallel performance is poor. ② Layer-by-layer load balancing is realized, but the adaptive function is limited. The main reason is that the coarsening function is limited because the eight sub-nodes to be coarsened and the data are not in the same process, and coarsening cannot be performed or data migration must be repeated multiple times before coarsening. ③ Free adaptation and layer-by-layer load balancing are realized through data concentration, but not distributed parallelism. Each process retains a complete octree and stores redundant data. The memory occupied by the octree itself in a single process and the amount of octree operation tasks increase with the increase of the problem scale. When large-scale parallelism occurs, the parallel efficiency is significantly reduced. ④ Set different weights for different layers to balance the overall load. The load balance within the layer has been improved to a certain extent, but the problem has not been fundamentally solved. Summary of the Invention

[0008] In view of the above technical problems, the present invention provides an octree grid layer-by-layer load balancing method, which realizes unrestricted self-adaptation and layer-by-layer load balancing without affecting the parallel performance.

[0009] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0010] An octree grid layer-by-layer load balancing method provided by the present invention includes an original octree and an actual octree; the original octree stores all nodes of the octree in a tree-shaped data structure and is responsible for the self-adaptive function; the actual octree stores the geometric information of the leaf nodes of the original octree in a one-dimensional array and is responsible for layer-by-layer load balancing; the leaf nodes in the original octree are original leaf nodes; the leaf nodes in the actual octree are actual leaf nodes; data conversion can be performed between the original octree and the actual octree;

[0011] The data conversion between the original octree and the actual octree includes the following steps:

[0012] Calculate the layer-by-layer load balancing partition according to the hierarchical statistical information of the original leaf nodes in the original octree, and determine the number of leaf nodes owned by each process in each layer of the actual octree;

[0013] Allocate actual leaf node numbers to each original leaf node according to the layer-by-layer load balancing partition, determine which process each original leaf node is assigned to, and determine its serial number in this process;

[0014] Calculate the communication topology of the data conversion between the original octree and the actual octree according to the original partition and the layer-by-layer load balancing partition, and determine the process number and local number of the process where each actual leaf node and the corresponding original leaf node are located;

[0015] Perform data conversion between the original octree and the actual octree according to the communication topology.

[0016] In the octree grid layer-by-layer load balancing method provided by the present invention, preferably, the distribution of the original octree meets the coarsening requirement, and eight leaf nodes with the same parent node are stored in the same computer node; wherein, the original octree can meet the overall load balancing.

[0017] In the octree grid layer-by-layer load balancing method provided by the present invention, preferably, the actual octree meets the layer-by-layer load balancing; wherein, the actual octree can meet the requirement of the best overall distribution local optimization.

[0018] The octree grid layer-by-layer load balancing method provided by the present invention preferably includes a data migration method for encrypting and coarsening an actual octree; each node of the original octree has a data area for storing external data related to the corresponding node.

[0019] The data migration method includes the following steps:

[0020] Before encryption and coarsening, fill the information of the actual leaf nodes into the data areas of the original leaf nodes in the corresponding original octree.

[0021] Perform encryption and coarsening on the actual octree to obtain a new actual octree after encryption and coarsening, and the nodes of the new actual octree are new actual leaf nodes.

[0022] Fill the information of the new actual leaf nodes into the data areas of the original leaf nodes in the corresponding original octree.

[0023] Traverse the original octree to obtain the correspondence between the actual leaf nodes and the new actual leaf nodes, and use this correspondence as the communication topology to perform data migration.

[0024] The above technical solution has the following advantages or beneficial effects:

[0025] The octree grid layer-by-layer load balancing method provided by the present invention decouples adaptability and load balancing through two groups of octrees. One group stores all the nodes of the octree in a tree-shaped data structure and is responsible for the adaptability function of the octree; the other group stores the geometric information of the octree leaf nodes in a one-dimensional array manner and is responsible for layer-by-layer load balancing. The adaptability and layer-by-layer load balancing of the octree are realized through the original octree and the actual octree. Moreover, the adaptability function is not restricted, the parallel performance is not restricted, and it is completely distributed. Theoretically, there is no limit to the parallel scale, and the parallel efficiency is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, the present invention and its features, shapes, and advantages will become more obvious. The same reference numerals indicate the same parts in all the drawings. The drawings are not deliberately drawn to scale, and the focus is on showing the gist of the present invention.

[0027] Figure 1 FIG. is a flowchart of an octree grid layer-by-layer load balancing method provided by an embodiment of the present invention.

[0028] Figure 2 FIG. is a flowchart of data migration before and after encryption and coarsening of an octree grid layer-by-layer load balancing method provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0029] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but it is not intended to limit the present invention.

[0030] Embodiment 1:

[0031] An octree grid layer-by-layer load balancing method provided by an embodiment of the present invention includes an original octree and an actual octree; the original octree stores all nodes of the octree in a tree-shaped data structure and is used to be responsible for the adaptive function; the actual octree stores the geometric information of the leaf nodes of the original octree in a one-dimensional array and is used to be responsible for layer-by-layer load balancing; the leaf nodes in the original octree are original leaf nodes; the leaf nodes in the actual octree are actual leaf nodes; data conversion can be performed between the original octree and the actual octree;

[0032] The data conversion between the original octree and the actual octree includes the following steps:

[0033] S101: Calculate the layer-by-layer load balancing dissection according to the hierarchical statistical information of the original leaf nodes in the original octree, and determine the number of leaf nodes owned by each process in each layer of the actual octree;

[0034] S102: Allocate actual leaf node numbers to each original leaf node according to the layer-by-layer load balancing dissection, determine which process each original leaf node is assigned to, and determine its serial number in this process;

[0035] S103: Calculate the communication topology of the data conversion between the original octree and the actual octree according to the original dissection and the layer-by-layer load balancing dissection, and determine the process number and local number of the corresponding original leaf node where each actual leaf node is located;

[0036] S104: Perform data conversion between the original octree and the actual octree according to the communication topology.

[0037] In Embodiment 1 of the present invention, the adaptation and load balancing are decoupled through two groups of octrees. One group stores all the nodes of the octree in a tree-shaped data structure and is responsible for the adaptation function of the octree. The other group stores the geometric information of the leaf nodes of the octree in a one-dimensional array and is responsible for layer-by-layer load balancing. The octree responsible for the adaptation function is called the original octree (original octree, og), or the og form of the octree. The octree responsible for layer-by-layer load balancing is called the actual octree (actual octree, ac), or the ac form of the octree. Since the tree-shaped data structure is the fundamental storage method of the octree, the octree with this storage method is called the original octree. The leaf nodes are part of the octree, and the geometric information of the leaf nodes is directly related to the upper-layer actual application and can be stored using a common array. Therefore, the geometric information of the leaf nodes stored in a one-dimensional continuous manner is called the actual octree. The actual octree is generated from the original octree and is responsible for docking the upper-layer application data. The original octree stores the complete information of the octree in a tree-shaped manner but has no direct connection with the application data. The adaptation and layer-by-layer load balancing of the octree are achieved through the original octree and the actual octree. The adaptation function is not restricted, the parallel performance is not restricted, and it is fully distributed. Theoretically, there is no restriction on the parallel scale, and the parallel efficiency is high.

[0038] The adaptation and load balancing are effectively decoupled through the original octree and the actual octree, but the distributions of their leaf nodes are different. The distribution of the original octree does not need to consider load balancing, let alone layer-by-layer load balancing. Therefore, in this embodiment, the distribution of the original octree meets the coarsening requirement, and the eight leaf nodes with the same parent node are stored in the same computer node. Among them, on the premise of meeting the coarsening requirement, the original octree can meet the overall load balancing. The actual octree is responsible for the load balancing of the octree. Therefore, in this embodiment, the actual octree needs to meet the layer-by-layer load balancing. On this basis, it can further meet the requirement of the best overall distribution localization, but it does not need to meet the coarsening requirement.

[0039] Although the original octree and the actual octree are decoupled, information needs to be converted between them. For example, converting the geometric information of the leaf nodes of the original octree into the geometric information of the actual octree, and converting the encrypted criterion data in the form of the actual octree into the form of the original octree. Therefore, in this embodiment, the data conversion between the original octree and the actual octree is achieved through steps S101 to S104.

[0040] During the encryption and coarsening process, a series of operations are performed on the original octree, including encryption, coarsening, smoothing, and dissection. These operations change the distribution and topology of the original octree, and correspondingly, the distribution and topology of the actual octree change. The data field before encryption and coarsening corresponds to the actual octree before encryption and coarsening. The data field after encryption and coarsening corresponds to the actual octree after encryption and coarsening. The so-called "correspondence" means that the data field is in units of grid cells or grid blocks, and its quantity and storage order are the same as those of the actual leaf nodes. The data field after encryption and coarsening is generated by data migration and data encryption and coarsening reconstruction from the previous data field. However, there is no direct connection between the actual octrees before and after encryption and coarsening, so the data field after cannot be directly obtained from the previous data field. It is necessary to establish a connection between the two actual octrees before and after, that is, generate the correspondence relationship between the actual leaf nodes before and after encryption and coarsening, and perform data migration according to this correspondence relationship. Therefore, in this embodiment, data migration between the actual leaf nodes before and after encryption and coarsening is performed through the data area owned by each node of the original octree for storing external data related to the corresponding node. The specific steps are as follows:

[0041] S201: Before encryption and coarsening, fill the information of the actual leaf nodes into the data area of the original leaf nodes in the corresponding original octree;

[0042] S202: Perform encryption and coarsening on the actual octree to obtain a new actual octree after encryption and coarsening, and the nodes of the new actual octree are new actual leaf nodes;

[0043] S203: Fill the information of the new actual leaf nodes into the data area of the original leaf nodes in the corresponding original octree;

[0044] S204: Traverse the original octree to obtain the correspondence relationship between the actual leaf nodes and the new actual leaf nodes, and use this correspondence relationship as the communication topology to perform data migration.

[0045] In this embodiment, the core of establishing the correspondence relationship before and after encryption and coarsening is to fill the information of the two groups of actual leaf nodes before and after into the data area of the same original octree. The original octrees before and after encryption and coarsening can be considered as the same original octree. Before the start of encryption and coarsening, fill the information of the old actual leaf nodes into the original octree data area and make it change with the change of the original octree. After encryption and coarsening, fill the information of the new actual leaf nodes into the original octree data area. At this time, the information of the new and old actual leaf nodes is stored in the data area of the original octree together and corresponds one by one. Traverse the original octree, extract it, and process it to obtain the correspondence relationship between the actual leaf nodes before and after encryption and coarsening. Use this correspondence relationship as the communication topology to perform data migration and reconstruct the data field after encryption and coarsening.

[0046] Those skilled in the art should understand that those skilled in the art can implement the variation examples in combination with the prior art and the above embodiments, which will not be elaborated herein. Such variation examples do not affect the essence of the present invention and will not be elaborated herein.

[0047] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific implementation manners, and the devices and structures not described in detail should be understood to be implemented in a common manner in the art; any person skilled in the art can make many possible changes and modifications without departing from the technical solution of the present invention, or modify it into an equivalent embodiment with equivalent changes, which does not affect the essence of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the technical solution of the present invention still fall within the scope of protection of the technical solution of the present invention.

Claims

1. An octree grid layer-by-layer load balancing method, characterized in that It includes an original octree and an actual octree; the original octree stores all nodes of the octree in a tree - shaped data structure and is responsible for the adaptive function; the actual octree stores the geometric information of the original leaf nodes of the original octree in a one - dimensional array and is responsible for layer - by - layer load balancing; the leaf nodes in the original octree are original leaf nodes; the leaf nodes in the actual octree are actual leaf nodes; data conversion can be performed between the original octree and the actual octree; The data conversion between the original octree and the actual octree includes the following steps: Calculate the layer - by - layer load - balancing dissection according to the hierarchical statistical information of the original leaf nodes in the original octree, and determine the number of leaf nodes owned by each process in each layer of the actual octree; Allocate actual leaf node numbers to each original leaf node according to the layer - by - layer load - balancing dissection, determine which process each original leaf node is assigned to, and determine its sequence number in this process; Calculate the communication topology of the data conversion between the original octree and the actual octree according to the original dissection and the layer - by - layer load - balancing dissection, and determine the process number and local number of the process where each actual leaf node and its corresponding original leaf node are located; Perform data conversion between the original octree and the actual octree according to the communication topology; Among them, it also includes a data migration method for encrypting and coarsening the actual octree; each node of the original octree has a data area, and the data area is used to store external data related to the corresponding node; The data migration method includes the following steps: Before encryption and coarsening, fill the information of the actual leaf nodes into the data areas of the original leaf nodes in the corresponding original octree; Perform encryption and coarsening on the actual octree to obtain a new actual octree after encryption and coarsening, and the nodes of the new actual octree are new actual leaf nodes; Fill the information of the new actual leaf nodes into the data areas of the original leaf nodes in the corresponding original octree; Traverse the original octree, and the corresponding relationship between the actual leaf nodes and the new actual leaf nodes can be obtained, and this corresponding relationship is used as the communication topology to perform data migration.

2. The octree grid layer-by-layer load balancing method according to claim 1, wherein The distribution of the original octree meets the coarsening requirements, and eight leaf nodes with the same parent node are stored in the same computer node; among them, the original octree can meet the overall load - balancing requirement.

3. The octree grid layer-by-layer load balancing method according to claim 1, wherein The actual octree meets the layer - by - layer load - balancing requirement; among them, the actual octree can meet the requirement of the best overall distribution and local optimization.

Images