Multi-node parallel acceleration method, device and equipment based on high-performance computing platform and RoCE network and medium

Abstract

The embodiment of the invention provides a multi-node parallel acceleration method, device and equipment based on a high-performance computing platform and an RoCE network and a medium, and relates to the technical field of cluster computing, and the method comprises the following steps: constructing a communication stack based on the RoCE network, and realizing an interconnection transmission network between nodes from a sending end to a receiving end; a computing task and a communication transmission task of the high-performance computing platform are executed through a CPU core binding technology of the high-performance computing platform; based on the transparent large page function, processing a memory allocation request initiated by an application program of an operating system and a request of the application program of the operating system for accessing a memory through a virtual address; before a main computing task of the high-performance computing platform starts, pre-processing is executed through multiple threads, open type multi-processing threads are started, the main computing task is executed through the multiple open type multi-processing threads, and a next round of pre-processing task is started in parallel. According to the scheme, parallel acceleration of multiple nodes in cluster calculation is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cluster computing, and in particular to a multi-node parallel acceleration method and device based on a high-performance computing platform and RoCE network, equipment and medium. BACKGROUND

[0002] The core idea of multi-node parallel acceleration of cluster computing is to divide a huge computing task into many smaller sub-tasks, then distribute these sub-tasks to multiple computing nodes in the cluster for simultaneous execution (parallel computing), and finally aggregate the computing results of each node to significantly shorten the overall computing time. Achieving ideal acceleration effect is not easy, and the following main challenges will be encountered: 1. The time spent on exchanging data between nodes, if the communication is too frequent or the data volume is too large, the nodes may spend a lot of time waiting for data instead of computing.

[0003] 2. If the task decomposition is uneven, some nodes complete the work early and enter an idle state, while other nodes are still busy, which wastes resources.

[0004] 3. Amdahl's Law: the speedup of a system is limited by the proportion of its serial part.

[0005] 4. There are many cluster nodes, and the probability of hardware failure of a single node is greatly increased. Failure of a node can cause the entire large computing task to fail. SUMMARY

[0006] Therefore, the embodiments of the present application provide a multi-node parallel acceleration method based on a high-performance computing platform and RoCE network to solve the technical problems of long data exchange time between nodes, uneven task decomposition, and high failure rate in the prior art. The method comprises: Building a communication stack based on the RoCE network, and building an interconnection transmission network from the sending end to the receiving end between the cluster nodes based on the communication stack; Executing the computing task of the high-performance computing platform through the CPU binding core technology and the non-uniform memory access architecture of the CPU, and executing the communication transmission task between the high-performance computing platforms based on the communication stack of the RoCE network, the non-uniform memory access architecture and the CPU binding core technology; Turning on the transparent large page function in the operating system of the high-performance computing platform, and processing the memory allocation request initiated by the application program of the operating system and the request for accessing memory through a virtual address by the application program of the operating system based on the transparent large page function; Before starting the main computing task of the high-performance computing platform, the data loading task, the format conversion task and the initialization computing task in the pre-processing are executed by using multi-thread, the pre-processing data is generated, the threads of the open multi-processing are started, the main computing task is executed by using multiple threads of the open multi-processing, and the next round of pre-processing task is started in parallel.

[0007] The embodiment of the application further provides a multi-node parallel acceleration device based on a high-performance computing platform and a RoCE network, to solve the technical problems of long data exchange time between nodes, uneven task decomposition and high failure rate in the prior art. The network deployment module is configured to construct a communication stack based on the RoCE network, and construct an interconnection transmission network from a sending end to a receiving end between cluster nodes based on the communication stack. The core binding strategy configuration module is configured to execute the computing task of the high-performance computing platform by using the CPU core binding technology and the non-uniform memory access architecture of the CPU, and execute the communication transmission task between the high-performance computing platforms based on the communication stack of the RoCE network, the non-uniform memory access architecture and the CPU core binding technology. The transparent large page enabling module is configured to start the transparent large page function in the operating system of the high-performance computing platform, and process the memory allocation request initiated by the application program of the operating system and the request for accessing memory by the application program of the operating system through the virtual address based on the transparent large page function. The architecture adaptation module is configured to execute the data loading task, the format conversion task and the initialization computing task in the pre-processing by using multi-thread before starting the main computing task of the high-performance computing platform, generate the pre-processing data, start the threads of the open multi-processing, execute the main computing task by using multiple threads of the open multi-processing, and start the next round of pre-processing task in parallel.

[0008] The embodiment of the application further provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor.

[0009] The embodiment of the application further provides a computer readable storage medium, which stores a computer program for executing the multi-node parallel acceleration method based on the high-performance computing platform and the RoCE network.

[0010] Compared with the prior art, the at least one technical solution adopted by the embodiment of the present specification can achieve the beneficial effects at least including: By implementing RoCE high-speed network, CPU binding core, transparent large page and application parameter adaptation on a high-performance computing platform, a multi-node parallel acceleration framework for CAE simulation of hundreds of millions is constructed. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0012] Figure 1 is a flowchart of a multi-node parallel acceleration method based on a high-performance computing platform and a RoCE network provided by the embodiment of the present application; Figure 2 is a structural block diagram of a computer device provided by the embodiment of the present application; Figure 3 is a structural block diagram of a multi-node parallel acceleration device based on a high-performance computing platform and a RoCE network provided by the embodiment of the present application. DETAILED DESCRIPTION

[0013] The embodiments of the present application will be described in detail below with reference to the drawings.

[0014] The embodiments of the present application are described below through specific concrete examples, and those skilled in the art can easily understand other advantages and effects of the present application from the disclosure of the present specification. Obviously, the described embodiments are only some of the embodiments of the present application, not all. The present application can also be implemented or applied by other different specific embodiments, and each detail in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0015] For the software development of CAE simulation of hundreds of millions, a general multi-node parallel acceleration method is provided to quickly identify the performance bottleneck of multi-node parallel computing, locate the abnormal point to the code line level, and give optimization suggestions. Support real-time performance analysis of ten million process jobs.

[0016] The cluster of the domestic high-performance computing platform is closely related to the underlying hardware (computing nodes, high-speed network, storage) and basic software (mathematical library, compiler, parallel library, solver). The software development process can flexibly use the characteristics of the domestic self-research. The present application fully combines the characteristics of domestic software and hardware, and optimizes the final software performance through software and hardware combination. Through the combination with domestic hardware, the resource occupation of CPU, memory, network, disk IO and other resources in the performance analysis process is reduced to the minimum, the influence on the actual running of the software is reduced, and the actual running state of the software is restored.

[0017] Relying on the domestic high-performance computing software and hardware ecological system, especially the computing nodes taking the Huawei high-performance computing platform processor as the core, the high-speed Ethernet interconnection architecture supporting the RoCE protocol, the domestic Linux operating system and the matching DevKit performance analysis tool. In the typical high-performance computing application scenarios such as CAE simulation facing the grid scale of hundreds of millions, the overall performance of the system is not determined by a single factor, but is comprehensively affected by multiple dimensions such as computing, communication, memory access and task scheduling. Therefore, the implementation of the present application is not to apply a single optimization technology in isolation, but to systematically integrate several key technical measures to build an end-to-end collaborative optimization running environment, so as to realize the acceleration effect close to the theoretical limit in multi-node parallel computing.

[0018] In order to realize efficient multi-node parallel acceleration, in the specific implementation process, the present application focuses on the following four aspects: deployment and communication protocol optimization of RoCE high-speed network, configuration of CPU binding core strategy, enabling of transparent large page (THP), and deep adaptation of application running parameters and high-performance computing platform architecture characteristics. The four technologies respectively act on the cluster communication layer, process scheduling layer, memory management layer and application execution layer, and have clear dependence relationship and synergistic enhancement mechanism with each other, and together constitute an organic whole. The implementation principle, configuration logic, performance influence mechanism and positioning in the whole parallel acceleration framework of each technology will be described in detail below, and the internal relationship between them will be mainly explained.

[0019] In the embodiment of the present application, a multi-node parallel acceleration method based on a high-performance computing platform and a RoCE network is provided, as shown in Figure 1 The method comprises the following steps: Step S101: constructing a communication stack based on the RoCE network, and constructing an interconnection transmission network from a sending end to a receiving end between cluster nodes based on the communication stack; Step S102: performing the computing task of the high-performance computing platform through the CPU binding core technology and the non-uniform memory access architecture of the CPU, and performing the communication transmission task between the high-performance computing platforms based on the communication stack of the RoCE network, the non-uniform memory access architecture and the CPU binding core technology; Step S103: starting the transparent large page function in the operating system of the high-performance computing platform, and processing the memory allocation request initiated by the application program of the operating system and the request of the application program of the operating system to access the memory through the virtual address based on the transparent large page function; Step S104: before the main computing task of the high-performance computing platform starts, using multiple threads to execute the data loading task, format conversion task and initialization computing task in the pre-processing, generating pre-processing data, and starting the open multi-processing thread, executing the main computing task through multiple open multi-processing threads, and starting the next round of pre-processing task in parallel.

[0020] In specific implementation, the following steps are used to build the communication stack based on the RoCE network and build the interconnection transmission network from the sending end to the receiving end between the cluster nodes based on the communication stack: A high-performance communication abstraction layer is built, and the underlying transmission protocol is selected through the high-performance communication abstraction layer, wherein the underlying transmission protocol is the UD mode; A communication stack including an application layer, an RDMA transmission protocol layer, a network layer and a data link layer is built, and the high-performance communication abstraction layer is inserted between the application layer and the RDMA transmission protocol layer; The to-be-transmitted data of the sending end is stored in the RDMA memory area of the sending end, and a UD data packet is built through the to-be-transmitted data; the UD data packet is submitted to the sending queue of the RoCE network, the UD data packet is encapsulated into a network data frame supporting the RDMA transmission protocol through the RDMA transmission protocol layer of the communication stack, wherein the network data frame includes a UDP header; the network data frame is sent to the receiving end through the RoCE network, the UD data packet of the network data frame is verified and extracted, and the UD data packet is stored in the RDMA memory area of the receiving end.

[0021] In large-scale MPI (Message Passing Interface) parallel computing, the computing task is decomposed into multiple sub-tasks, which are distributed on different computing nodes in the cluster and executed in parallel. However, the boundary data between nodes needs to be frequently exchanged, the global reduction or synchronization operation needs to be performed, which constitutes an important overhead in the whole computing process. The traditional Ethernet communication mechanism based on TCP / IP protocol stack faces significant bottlenecks in the high-performance computing scenario. First, TCP / IP communication relies on the operating system kernel protocol stack to process network packets, and each transmission needs to perform multiple memory copies between the user state and the kernel state, and trigger context switching, resulting in high CPU resource consumption. Second, the processing logic of the protocol stack itself (such as verification, retransmission, congestion control, etc.) introduces hundreds of microseconds or even milliseconds of delay, which is difficult to meet the demand of HPC application for microsecond-level communication response. Third, in the high-concurrency and small-message communication scenario, the protocol header overhead ratio is significant, and the effective bandwidth utilization is far lower than the theoretical value of the physical link, and these problems are particularly prominent in the billion-grid CAE simulation.

[0022] To break through the above communication bottlenecks, the application adopts RoCE (RDMA over Converged Ethernet) as the high-speed interconnection network between cluster nodes. RoCE is the implementation of RDMA (Remote Direct Memory Access) technology on standard Ethernet, and its core idea is to allow the network card to directly read and write the application memory of the remote node, without the participation of the remote CPU, without the intervention of the operating system kernel, and without intermediate data copying.

[0023] The traditional TCP / IP communication path needs to go through multiple layers of forwarding such as “application program → kernel protocol stack → network card driver → network card”, while RoCE registers the memory region by the network card to directly initiate the read and write operation on the remote memory address, greatly simplifying the communication path. This mechanism significantly reduces the communication delay (which can be reduced to microseconds), improves the bandwidth utilization (close to the line speed), and greatly reduces the CPU overhead, thereby releasing more computing resources for actual computing tasks.

[0024] The current mainstream RDMA implementations include InfiniBand, iWARP and RoCE. InfiniBand is designed for RDMA, has the best performance but high cost, and needs special network cards and switches; iWARP is based on TCP, although it is compatible with ordinary Ethernet switches, but a large number of TCP connections will consume a large amount of memory resources in large-scale deployment; RoCE is based on UDP / IP or Ethernet link layer, and achieves a good balance between performance, cost and compatibility. RoCE is close to InfiniBand in performance, lower than iWARP in cost, and can reuse the existing Ethernet infrastructure (only need to support PFC, ECN and other lossless Ethernet features of the switch), which is the mainstream choice of current domestic HPC clusters.

[0025] RoCE contains two versions: RoCEv1 works in the Ethernet link layer (Layer 2), and requires the switch to support PFC (Priority Flow Control) and other flow control mechanisms to avoid packet loss; RoCEv2 is encapsulated in the UDP / IP layer (Layer 3), introduces IP addressing, supports cross-subnet communication, and has better scalability, which is suitable for large-scale distributed deployment. The present application prefers to use RoCEv2 scheme to balance performance and networking flexibility.

[0026] Only deploying RoCE hardware cannot automatically activate its performance advantages, and the upper communication software stack must be adapted. The specific implementation of the present application includes the following two key steps: First, upgrade the MPI library to a version that supports UCX (Unified Communication X). UCX is a high-performance communication abstraction layer that can automatically select the optimal underlying transmission protocol (such as RC or UD mode of RoCE). Many domestic CAE software still rely on Intel MPI 2018 or earlier versions, which do not integrate UCX support, and communication still defaults to TCP / IP path, which cannot take advantage of RoCE capabilities. Therefore, the present application requires replacing the MPI library with Intel MPI 2021 and later versions, or OpenMPI 4.x and later versions to obtain native support for RoCE.

[0027] Secondly, the UD (Unreliable Datagram) transmission mode using RoCE is explicitly specified at runtime. The UD mode adopts a connectionless design, does not need to maintain the end-to-end connection state, has the lowest communication delay, and the connection overhead is linearly related to the number of processes, which is very suitable for large-scale parallel scenarios with ten thousand processes. In contrast, the RC (Reliable Connected) mode provides reliable transmission, but needs to maintain an independent connection for each pair of processes, and the number of connections grows quadratically with the number of processes (N²), which has huge memory overhead in large-scale deployment and is not conducive to system expansion. Therefore, the UD mode is enabled in the configuration to achieve the best communication scalability.

[0028] Through the above implementation, MPI communication will bypass the traditional TCP / IP protocol stack and directly access remote memory through the RoCE network card, significantly reducing communication delay and CPU overhead, providing a high-bandwidth, low-latency communication foundation for multi-node parallelism, and fundamentally alleviating the communication bottleneck between multiple nodes.

[0029] In specific implementation, the CPU binding core technology and the non-uniform memory access architecture of the CPU are used to execute the computing task of the high-performance computing platform by the following steps: The topology of the non-uniform memory access architecture of the CPU is analyzed to determine the CPU core set and local memory area corresponding to each node of the non-uniform memory access architecture. A mapping relationship table of the message passing interface process and the node of the non-uniform memory access architecture is established, which is used to one-to-one correspondence between each message passing interface process and the node of the non-uniform memory access architecture. A message passing interface process cluster is started, and each message passing interface process is fixed to a specific CPU core of the target node of the non-uniform memory access architecture through CPU affinity binding technology according to the mapping relationship table. The memory allocation strategy of each message passing interface process is configured to make the message passing interface process use the local physical memory of the bound node of the non-uniform memory access architecture. Based on the specific CPU core and the memory allocation strategy, the computing task of the high-performance computing platform is executed.

[0030] In specific implementation, the communication stack based on the RoCE network, the non-uniform memory access architecture, and the CPU binding core technology are used to execute the communication transmission task between the high-performance computing platforms by the following steps: The RoCE network is set to an unreliable datagram transmission mode, and each message passing interface process is bound to a RoCE network card port of a node of the non-uniform memory access architecture in a one-to-one manner; when communication is performed between the message passing interface processes, the message passing interface process of the sending end obtains to-be-sent data from a local memory of the non-uniform memory access architecture node bound to the message passing interface process, encapsulates the to-be-sent data into a RoCE unreliable datagram packet through a local RoCE network card port, and sends the to-be-sent data; and the message passing interface process of the receiving end receives the to-be-sent data through the local RoCE network card port of the non-uniform memory access architecture node bound to the message passing interface process, generates received data, and stores the received data into the local memory of the non-uniform memory access architecture node bound to the message passing interface process.

[0031] Even if the inter-node communication bottleneck is solved by RoCE, the resource scheduling problem within a single computing node can still become a new performance constraint factor. Modern servers (such as nodes equipped with high-performance computing platform 920 processors) usually adopt a multi-Non-Uniform Memory Access (NUMA) architecture. Taking a typical 96-core high-performance computing platform server as an example, it contains multiple NUMA nodes, each of which integrates several CPU cores, shares an L3 cache, and is directly connected to a local memory channel. If the operating system does not perform affinity constraints on processes / threads, it will lead to loss of cache locality, increased cross- NUMA memory access, and intensified resource competition. Thread migration between different cores will cause frequent invalidation of L1 / L2 caches; a process running on NUMA 0 frequently accesses the memory of NUMA 1, and the access delay increases by 2-3 times; multiple MPI (Message Passing Interface) processes or OpenMP threads compete for the same physical core, causing context switching overhead. The above problems are particularly significant in the hybrid parallel (MPI + OpenMP) model, directly affecting the computing efficiency of a single node.

[0032] CPU affinity refers to a technique of fixing the scheduling of a process or thread to a specific CPU core or core set. Its core goal is to improve the locality of computing tasks and reduce performance jitter caused by scheduling uncertainty. Through CPU affinity, a thread can continuously use the private cache of the same core, avoiding cache thrashing; a process can be located on the same NUMA node as its assigned memory, ensuring the locality of memory access; at the same time, task migration is prohibited, and context switching overhead is significantly reduced.

[0033] NUMA topology of high-performance computing platform, develop a fine binding core scheme. Each MPI process is bound to a specific NUMA node, and its memory allocation is limited to the local, to ensure that the calculation and memory access are completed locally. At the same time, the OpenMP thread is bound, and the thread migration is prohibited, so as to ensure that each thread exclusively occupies a physical core and avoid resource contention (OpenMPI can use the-bind-to core parameter, and other MPI can use the numactl command).

[0034] The NUMA analysis module of DevKit can monitor the proportion of cross- NUMA memory access, L3 cache hit rate, and core CPU utilization balance to verify the binding effect of the core.

[0035] Binding the core not only improves the efficiency of single-node computing, but also indirectly optimizes the communication performance. Stable CPU affinity enables RoCE network card interrupts (IRQ) to be bound to dedicated cores, avoiding resource contention with computing threads; at the same time, MPI communication threads can process RoCE events more timely due to stable scheduling, reducing communication delay jitter. Therefore, binding the core provides a stable computing environment foundation for the efficient operation of RoCE.

[0036] In implementation, the following steps are taken to enable the transparent large page function in the operating system of the high-performance computing platform, and based on the transparent large page function, the memory allocation request initiated by the application program of the operating system and the request for accessing memory by the application program of the operating system through a virtual address are processed: In the operating system of the high-performance computing platform, the transparent large page function is enabled, and the mode of the transparent large page is set. When the application program of the operating system initiates a memory allocation request, a fixed-size dedicated memory region is determined, the dedicated memory region is used as a requested memory region, adjacent regular memory pages are detected, and multiple regular memory pages are merged into a single transparent large page. A direct mapping relationship between the application program virtual address space and the transparent large page is established. When the application program of the operating system accesses memory through a virtual address, the memory management unit performs address translation, finds a physical address through a page table buffer table item of the transparent large page, and accesses a continuous physical memory region corresponding to the transparent large page.

[0037] Specifically, the intermediate page table level is reduced through the direct mapping relationship. The TLB hit rate is significantly improved by finding the physical address through the page table buffer table item of the transparent large page, the number of page table traversals is reduced, the address translation overhead is reduced, and the physical address is quickly obtained. The continuous physical memory region ensures that the prefetch mechanism works efficiently, and improves the cache line filling efficiency.

[0038] High performance computing applications usually deal with super large scale contiguous memory blocks (e.g. billion grid data structure). Linux uses 4KB small pages for memory management by default, which faces significant bottleneck in large memory scenarios. First, the number of TLB (Translation Lookaside Buffer) entries is limited (usually thousands), and 4KB pages need a large number of TLB entries to cover large memory regions, and frequent TLB misses will trigger multi-level page table traversal, significantly increasing memory access latency. Second, the x86_64 and ARM64 architectures use a four-level page table, and the address translation process is complex and has high overhead. Third, the page table itself occupies a large amount of memory space, for example, 1TB memory needs about 256MB page table, wasting valuable memory resources.

[0039] Transparent Huge Pages (THP) automatically merges adjacent 4KB pages into 2MB large pages (1GB is supported on some platforms). Its advantages include 512 times improvement in TLB coverage, significantly reducing TLB misses, reducing page table levels, and accelerating virtual address to physical address translation; reducing page table memory usage and improving effective memory bandwidth. THP is transparent to applications and does not require source code modification, and is enabled through kernel parameters.

[0040] The embodiment of the application enables THP mode (set to always mode) on a high-performance computing platform to maximize memory access efficiency. It should be noted that THP merging relies on contiguous physical memory, and memory fragmentation may occur after long-term operation, resulting in merging failure. Therefore, it is recommended to use the defrag strategy (set to always) to perform memory defragmentation when necessary. In addition, THP is suitable for large block contiguous memory allocation scenarios (such as HPC and scientific computing), and may be counterproductive for applications that frequently allocate / release small memory (such as some databases). However, CAE type applications are highly adapted to THP, and actual measurements show that it can significantly improve memory access performance.

[0041] THP, binding cores, and RoCE form a close linkage: binding cores ensure that process memory allocation is located on the local NUMA node, improving the availability of local contiguous physical memory and thus improving the success rate of THP merging; the improved memory bandwidth and low latency characteristics of THP provide strong support for the high-speed DMA (Direct Memory Access) data transfer of RoCE network cards; and the low CPU overhead characteristics of RoCE release more computing resources for THP background memory defragmentation. The three together build a high-bandwidth, low-latency, and high-locality memory subsystem, providing a solid foundation for parallel computing.

[0042] In specific implementation, the following steps are taken to achieve the multi-threaded execution of data loading tasks, format conversion tasks and initialization calculation tasks in pre-processing before the start of the main calculation task on the high-performance computing platform, and to generate post-pre-processing data: A pre-processing thread group independent of the main calculation thread pool is created, and the data loading task, the format conversion task and the initialization calculation task are executed in parallel through the pre-processing thread group, each pre-processing thread in the pre-processing thread group loads input data from a storage device to a memory buffer through a parallel I / O operation, and the loaded input data is standardized in format and restructured in structure in the memory buffer to generate a post-pre-processing data block.

[0043] In specific implementation, the following steps are taken to achieve the multi-threaded execution of data loading tasks, format conversion tasks and initialization calculation tasks in pre-processing before the start of the main calculation task on the high-performance computing platform, and to generate post-pre-processing data: According to the CPU core topology of the high-performance computing platform and the non-uniform memory access architecture, the total number of open multiprocessing threads is determined, the main calculation thread acquires the post-pre-processing data block through a lock-free synchronization mechanism, and the post-pre-processing data block is decomposed into a plurality of calculation sub-tasks, the total number of open multiprocessing threads is started, the calculation sub-tasks are executed in parallel through a plurality of open multiprocessing threads, and data in the memory is accessed through the nodes of the non-uniform memory access architecture, and the next round of pre-processing tasks is started in parallel while the task of the current main calculation thread is being executed.

[0044] Specifically, the time overlap of data preprocessing and main calculation is achieved through a double buffering mechanism, and the pre-processing thread group and the main calculation thread group form a continuous execution pipeline in the time dimension The embodiment of the application realizes the deep adaptation of the application to the high-performance computing platform in the following way: First, multi-threaded pre-processing is enabled. Many CAE software default to single-threaded in pre-processing tasks such as mesh interpolation and boundary condition setting, and cannot take advantage of the multi-core advantage of the high-performance computing platform. By enabling the pre-processing parallel option, the preparation phase time can be significantly shortened.

[0045] Second, the OpenMP thread configuration is optimized. The number of threads is dynamically set according to the number of physical cores to avoid thread overload or resource idling.

[0046] Application layer adaptation is the final embodiment of the entire optimization chain. RoCE solves the communication bottleneck between nodes, and bind core and THP optimize the node internal calculation and memory subsystem, while application parameter adaptation ensures that the upper layer calculation task can fully press the hardware resources optimized by the bottom layer. The four form a full-stack collaborative optimization system of "communication - scheduling - memory - application", and none of them can be omitted.

[0047] The key technology of the embodiment of the application constitutes an optimization system with clear levels and mutual enhancement: As the communication layer foundation, RoCE high-speed network reduces MPI communication delay from milliseconds to microseconds, eliminates MPI_Wait waiting bottleneck, and ensures efficient cooperation of each node. As the scheduling layer optimization, the binding core strategy improves cache hit rate and guarantees NUMA locality by fixing processes / threads to specific cores, maximizing single-node computing efficiency. As the memory management layer optimization, transparent large pages reduce TLB misses and page table overhead, providing high-bandwidth and low-latency memory access support for RoCE high-speed DMA. As the application layer optimization, application parameter adaptation converts the theoretical peak computing power of Kunpeng hardware into actual application performance through means such as enabling multi-threading, linking optimization libraries, and compiler vectorization.

[0048] There is a clear dependence and enhancement relationship among the four: the efficient operation of RoCE depends on the low-latency and high-bandwidth memory subsystem, and THP is the key to improving memory performance; the merging efficiency of THP depends on the NUMA locality and continuous memory provided by the binding core; the stability of the binding core guarantees the timely processing of RoCE events by MPI communication threads; finally, the effects of all underlying optimizations need to be fully released through application layer adaptation. This closed-loop coordination mechanism enables the system to achieve nearly linear multi-node parallel speedup in complex scenarios such as billion-level grid CAE simulation.

[0049] The embodiment of the application also includes the combination of domestic software and domestic hardware, and provides a performance analysis module based on the performance analysis tool DevKit of the high-performance computing platform server developed by Huawei, and designs and implements a performance collection middleware dedicated to domestic platforms. During software development, the performance collection middleware module is introduced in advance, and during software operation, the performance collection range can be dynamically adjusted through a unified configuration file.

[0050] Online and offline collection and analysis are combined. During online collection, the CPU, memory, network, and disk IO resource occupation during collection is minimized through the combination of the characteristics of domestic hardware and the basic software (math library, compiler, parallel library, solver) developed by domestic self-research. Offline analysis is combined with AI field special large models to give accurate optimization suggestions through professional knowledge.

[0051] In combination with the domestic Linux operating system, the kernel-level performance collection module is deployed to collect the running characteristics of CPU resources, memory access, network communication, and disk IO during program operation. In combination with the unified data collection service, fine-grained collection of performance data during software operation is efficiently realized.

[0052] The main analysis module includes: software hot function, key indicators for OpenMP (Open Multi-Processing) and MPI applications, Parallel Region and Barrier-to-Barrier serial and parallel time, memory access analysis, file IO analysis.

[0053] Program running basic information display, including execution time, CPU utilization, CPI, etc., to help customers correctly evaluate the resource utilization efficiency of the program running state; MPI (Message Passing Interface) Wait Rate is an indicator to evaluate the waiting proportion of the process communication interface, which gives a detailed analysis of the blocking situation of the program communication interface, including communication efficiency, interface proportion, IO situation, etc.

[0054] Hot functions have always been the most effective way to analyze program bottlenecks. In the MPI scenario, there are many processes, and it is difficult to analyze. DevKit collects all process hotspots and summarizes statistics, which is more suitable for hot analysis in MPI scenarios.

[0055] IO analysis: provides basic POSIX read-write interface collection, provides interface call efficiency, call stack, and read-write file data. On a cluster, storage is usually implemented through a network, and analyzing IO problems can find the bottleneck on the network; MPI communication situation: collect the running state of MPI interfaces in the program, analyze the call efficiency and data volume during interface call, etc. Interface problems are the focus of MPI problems, and solving communication blocking, adjusting task balance, and effectively improving running efficiency.

[0056] Fine-grained analysis is mainly to build a graphical analysis of the running program, providing a global perspective for MPI analysis, which can analyze from global to detail and comprehensively: a) Heat map: form a graph of access between ranks, show the communication data volume from the horizontal rank to the vertical rank, color code the access level, form a hot area, focus on analyzing the hot area, and solve the communication imbalance problem; b) Time sequence diagram: spread all rank communication interface calls on the same time axis, use color blocks to mark the communication area and calculation area, from the global you can analyze the communication balance, also can quickly find out the long blocking communication; each communication block marks the interface, communication volume, communication object, call stack, and the micro-architecture indicators and hotspots collected in the calculation block can be more detailed and in-depth analysis of the problem.

[0057] The performance analysis tool of the existing cluster computing is independently run (for example, hpctoolkit), the universality of the tool is good, and the performance data is dynamically collected during the software running. The performance bottleneck identified by this running mode is different from the actual performance bottleneck of the software running, and the performance tool collection process changes the running efficiency of the software, and identifies the wrong performance bottleneck. In the scene of large-scale job, there is no effective performance tool to help performance analysis.

[0058] The method of the present application reduces the occupation of resources such as computing CPU, memory, network and disk IO by combining with domestic hardware, can restore the performance characteristics of the actual software running, find the accurate performance bottleneck, reduce false positives, and support the parallel computing acceleration optimization of E-level cluster.

[0059] Finally, the performance tuning tool Devkit developed by the domestic self-research is used to display in the form of a graphical interface. On the above, we can see the Timeline of all processes, and through different colors, we can see the actual computing time and set communication time of each process.

[0060] In the embodiment, a computer device is provided, which comprises a memory 201, a processor 202, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned any multi-node parallel acceleration method based on a high-performance computing platform and a RoCE network when executing the computer program. Figure 2

[0061] Specifically, the computer device can be a computer terminal, a server or a similar computing device.

[0062] In the embodiment, a computer readable storage medium is provided, which stores a computer program for executing the above-mentioned any multi-node parallel acceleration method based on a high-performance computing platform and a RoCE network.

[0063] ​In particular, computer readable storage media includes permanent and non-permanent, removable and non-removable media implemented by any method or technology for information storage. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer readable storage media does not include transitory media such as modulated data signals and carriers.

[0064] Based on the same inventive concept, the embodiment of the present application also provides a multi-node parallel acceleration device based on a high-performance computing platform and a RoCE network, as described in the following embodiment. Since the multi-node parallel acceleration device based on the high-performance computing platform and the RoCE network solves the problem by the similar principle as the multi-node parallel acceleration method based on the high-performance computing platform and the RoCE network, the implementation of the multi-node parallel acceleration device based on the high-performance computing platform and the RoCE network can refer to the implementation of the multi-node parallel acceleration method based on the high-performance computing platform and the RoCE network, and the repeated parts will not be described. Hereinafter, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiment is preferably implemented in software, hardware or a combination of software and hardware is also possible and is conceived.

[0065] Figure 3 is a structural block diagram of the multi-node parallel acceleration device based on the high-performance computing platform and the RoCE network of the embodiment of the present application, as shown in Figure 3 The structure is described as follows.

[0066] The network deployment module 301 is configured to construct a communication stack based on the RoCE network, and construct an interconnection transmission network from a sending end to a receiving end between cluster nodes based on the communication stack. The binding core strategy configuration module 302 is configured to execute the computing task of the high-performance computing platform through the CPU binding core technology and the non-uniform memory access architecture of the CPU, and execute the communication transmission task between the high-performance computing platforms based on the communication stack of the RoCE network, the non-uniform memory access architecture and the CPU binding core technology; The transparent large page module 303 is configured to start the transparent large page function in the operating system of the high-performance computing platform, and process the memory allocation request initiated by the application program of the operating system and the request of the application program of the operating system to access the memory through the virtual address based on the transparent large page function; The architecture adaptation module 304 is configured to execute the data loading task, the format conversion task and the initialization computing task in the pre-processing of the multi-threading before the main computing task of the high-performance computing platform starts, generate the pre-processed data, and start the thread of the open multi-processing. The main computing task is executed through a plurality of threads of the open multi-processing, and the next round of pre-processing task is started in parallel.

[0067] In one embodiment, the network deployment module comprises: The communication abstraction layer construction unit is configured to construct a high-performance communication abstraction layer, and select a bottom layer transmission protocol through the high-performance communication abstraction layer, wherein the bottom layer transmission protocol is a UD mode. The communication construction stack unit is configured to construct a communication stack comprising an application layer, an RDMA transmission protocol layer, a network layer and a data link layer, and insert the high-performance communication abstraction layer between the application layer and the RDMA transmission protocol layer. The data storage unit is configured to store the to-be-transmitted data of the sending end into the RDMA memory area of the sending end, and construct a UD data packet through the to-be-transmitted data. The first data sending unit is configured to submit the UD data packet to the sending queue of the RoCE network, and encapsulate the UD data packet into a network data frame supporting the RDMA transmission protocol through the RDMA transmission protocol layer of the communication stack, wherein the network data frame comprises a UDP header. The data storage unit is configured to send the network data frame to the receiving end through the RoCE network, verify and extract the UD data packet of the network data frame, and store the UD data packet into the RDMA memory area of the receiving end.

[0068] In one embodiment, the binding core strategy configuration module comprises: The topology analysis unit is configured to analyze the topology of the non-uniform memory access architecture of the CPU, and determine the CPU core set and the local memory area corresponding to each node of the non-uniform memory access architecture. a mapping relationship table establishing unit, configured to establish a mapping relationship table of message passing interface processes and nodes of the non-uniform memory access architecture, the mapping relationship table being configured to correspond each message passing interface process to a node of the non-uniform memory access architecture one by one; a binding unit, configured to start a message passing interface process cluster, and bind each message passing interface process to a specific CPU core of a target node of the non-uniform memory access architecture by using a CPU affinity binding technology according to the mapping relationship table; a memory allocation strategy configuration unit, configured to configure a memory allocation strategy of each message passing interface process, so that the message passing interface process uses local physical memory of the bound node of the non-uniform memory access architecture; a computing task executing unit, configured to execute a computing task of the high-performance computing platform based on the specific CPU core and the memory allocation strategy.

[0069] In one embodiment, the core binding strategy configuration module further includes: a node port binding unit, configured to set the RoCE network to an unreliable datagram transmission mode, and bind each message passing interface process to a local RoCE network card port of a node of the non-uniform memory access architecture one by one; a data obtaining unit, configured to, when communication is performed between the message passing interface processes, obtain, by the message passing interface process of the sending end, to-be-sent data from local memory of the bound node of the non-uniform memory access architecture, encapsulate the to-be-sent data into a RoCE unreliable datagram packet through the local RoCE network card port, and send the to-be-sent data; a second data sending unit, configured to, by the message passing interface process of the receiving end, receive the to-be-sent data through the local RoCE network card port of the bound node of the non-uniform memory access architecture, generate received data, and store the received data in the local memory of the bound node of the non-uniform memory access architecture.

[0070] In one embodiment, the transparent large page enabling module includes: a transparent large page enabling unit, configured to enable a transparent large page function in an operating system of the high-performance computing platform, and set a mode of the transparent large page; a mapping relationship establishing unit, configured to, when an application program of the operating system initiates a memory allocation request, determine a dedicated memory region of a fixed size, take the dedicated memory region as a requested memory region, detect adjacent regular memory pages, merge a plurality of the regular memory pages into a single transparent large page, and establish a direct mapping relationship of an application program virtual address space to the transparent large page; The memory access unit is configured to, when an application of the operating system accesses memory through a virtual address, perform address conversion by the memory management unit, find a physical address through the transparent large page page table buffer table item, and access a continuous physical memory region corresponding to the transparent large page.

[0071] In one embodiment, the architecture adaptation module comprises: The thread group creation unit is configured to create a preprocessing thread group independent of the main computing thread pool. The parallel execution unit is configured to execute data loading tasks, format conversion tasks and initialization computing tasks in parallel through the preprocessing thread group. The data loading unit is configured to load input data from a storage device to a memory buffer through parallel I / O operations by each preprocessing thread in the preprocessing thread group. The structure reorganization unit is configured to perform format standardization and structure reorganization on the loaded input data in the memory buffer to generate preprocessed data blocks.

[0072] In one embodiment, the architecture adaptation module further comprises: The thread total number determination unit is configured to determine the total number of threads of the open multiprocessing according to the CPU core topology of the high-performance computing platform and the non-uniform memory access architecture. The subtask demarcation unit is configured to obtain preprocessed data blocks by the main computing thread through a lock-free synchronization mechanism, and demarcate the preprocessed data blocks into a plurality of computing subtasks. The execution subtask unit is configured to start the total number of threads of the open multiprocessing, execute the computing subtasks in parallel through a plurality of threads of the open multiprocessing, and access data in memory through nodes of the non-uniform memory access architecture. The synchronization start unit is configured to execute the task of the current main computing thread, and start the preprocessing task of the next round in parallel.

[0073] The embodiments of the present application achieve the following technical effects: By implementing RoCE high-speed network, CPU binding core, transparent large page and application parameter adaptation on a high-performance computing platform, a multi-node parallel acceleration framework for CAE simulation of hundreds of millions is constructed; the framework fully utilizes the multi-core, high-bandwidth and NUMA architecture characteristics of the processor of the high-performance computing platform, combines the low-delay communication capability of the RoCE network, realizes the full-stack performance optimization from the hardware layer to the application layer, and has good reproducibility and portability. Through the DevKit performance analysis tool, the optimization effect can be quantitatively evaluated, and further optimization is guided; the acceleration method of the embodiment of the application is not only suitable for the CAE field, but also can be popularized to other large-scale parallel computing scenes such as climate simulation, biological information and financial modeling, and provides a general technical path for efficient operation of domestic HPC software on a self-owned hardware platform.

[0074] Obviously, those skilled in the art should understand that each module or each step of the above-mentioned embodiments of the application can be realized by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into each integrated circuit module, or multiple modules or steps can be manufactured into a single integrated circuit module. Thus, the embodiments of the application are not limited to any specific combination of hardware and software.

[0075] The above only describes the preferred embodiments of the application and is not used to limit the application. For those skilled in the art, the embodiments of the application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application should be included in the protection scope of the application.

Claims

1. A multi-node parallel acceleration method based on a high-performance computing platform and a RoCE network, characterized in that, The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack.

2. The multi-node parallel acceleration method based on high-performance computing platform and RoCE network according to claim 1, wherein, The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack.

3. The multi-node parallel acceleration method based on high-performance computing platform and RoCE network according to claim 1, wherein, The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application relates to a high-performance computing platform and a method for constructing a RoCE network-based communication stack. The application configuring a memory allocation strategy for each of the message passing interface processes, so that the message passing interface processes use local physical memory of nodes of the non-uniform memory access architecture bound to the message passing interface processes; based on the specific CPU core and the memory allocation strategy, performing a computing task of the high-performance computing platform.

4. The multi-node parallel acceleration method based on high-performance computing platform and RoCE network according to claim 1, wherein, based on the communication stack of the RoCE network, the non-uniform memory access architecture and the CPU binding core technology, performing a communication transmission task between the high-performance computing platforms, including: setting the RoCE network to an unreliable datagram transmission mode, and binding each message passing interface process to a local RoCE network card port of a node of the non-uniform memory access architecture; when performing communication between the message passing interface processes, the message passing interface process of the sending end obtains to-be-sent data from the local memory of the node of the non-uniform memory access architecture bound to the message passing interface process, encapsulates the to-be-sent data into a RoCE unreliable datagram packet through the local RoCE network card port, and sends the RoCE unreliable datagram packet; the message passing interface process of the receiving end receives the to-be-sent data through the local RoCE network card port of the node of the non-uniform memory access architecture bound to the message passing interface process, generates received data, and stores the received data in the local memory of the node of the non-uniform memory access architecture bound to the message passing interface process.

5. The multi-node parallel acceleration method based on high-performance computing platform and RoCE network according to claim 1, wherein, enabling a transparent large page function in an operating system of the high-performance computing platform, and based on the transparent large page function, processing a memory allocation request initiated by an application program of the operating system and a request for accessing memory through a virtual address by the application program of the operating system, including: enabling the transparent large page function in the operating system of the high-performance computing platform, and setting a mode of the transparent large page; when the application program of the operating system initiates a memory allocation request, determining a dedicated memory region of a fixed size, taking the dedicated memory region as a requested memory region, detecting adjacent regular memory pages, merging a plurality of the regular memory pages into a single transparent large page, and establishing a direct mapping relationship between an application program virtual address space and the transparent large page; when the application program of the operating system accesses memory through a virtual address, a memory management unit performs address translation, finds a physical address through a page table buffer table item of the transparent large page, and accesses a continuous physical memory region corresponding to the transparent large page.

6. The multi-node parallel acceleration method based on high-performance computing platform and RoCE network according to any one of claims 1 to 5, characterized in that, before a main computing task of the high-performance computing platform starts, using a plurality of threads to perform a data loading task, a format conversion task and an initialization computing task in pre-processing, and generating post-pre-processing data, including: creating a pre-processing thread group independent of a main computing thread pool; performing the data loading task, the format conversion task and the initialization computing task in parallel through the pre-processing thread group; each pre-processing thread in the pre-processing thread group loads input data from a storage device to a memory buffer through a parallel I / O operation; performing format standardization and structure reorganization on the loaded input data in the memory buffer to generate a post-pre-processing data block.

7. The multi-node parallel acceleration method based on high-performance computing platform and RoCE network according to any one of claims 1 to 5, characterized in that, Start the open multi-processing thread, execute the main computing task through multiple open multi-processing threads, and start the next round of preprocessing tasks in parallel, including: According to the CPU core topology of the high-performance computing platform and the non-coherent memory access architecture, determine the total number of open multi-processing threads; The main computing thread obtains the preprocessed data block through a lock-free synchronization mechanism, and decomposes the preprocessed data block into multiple computing sub-tasks; Start the total number of open multi-processing threads, execute the computing sub-tasks in parallel through multiple open multi-processing threads, and access the data in the memory through the node of the non-coherent memory access architecture; When executing the task of the current main computing thread, start the next round of preprocessing tasks in parallel.

8. A multi-node parallel acceleration device based on a high-performance computing platform and a RoCE network, characterized in that, It includes: A network deployment module for building a RoCE network-based communication stack and building a sender-to-receiver interconnection transmission network between cluster nodes based on the communication stack; A core binding strategy configuration module for executing the computing task of the high-performance computing platform through the CPU core binding technology and the non-coherent memory access architecture of the CPU, and executing the communication and transmission task between the high-performance computing platforms based on the communication stack, the non-coherent memory access architecture, and the CPU core binding technology; An enable transparent large page module for enabling the transparent large page function in the operating system of the high-performance computing platform, and processing the memory allocation request initiated by the application program of the operating system and the request for accessing memory through virtual address by the application program of the operating system based on the transparent large page function; An architecture adaptation module for executing the data loading task, format conversion task and initialization computing task in the preprocessing before starting the main computing task of the high-performance computing platform, generating preprocessed data, and starting open multi-processing threads, executing the main computing task through multiple open multi-processing threads, and starting the next round of preprocessing tasks in parallel.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the multi-node parallel acceleration method based on the high-performance computing platform and the RoCE network in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program for executing the multi-node parallel acceleration method based on the high-performance computing platform and the RoCE network in any one of claims 1 to 7.

Images