[0056] Specific embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to illustrate and explain the present disclosure, and are not intended to limit the present disclosure.
[0057] figure 1 It is a data transmission method for a computing cluster shown according to an exemplary embodiment, and is applied to a computing cluster including multiple nodes. The computing cluster can be used as a server, and the method includes:
[0058] S101. Perform unified addressing on the storage spaces of the storage media of the multiple nodes to obtain addressing information, where each address in the addressing information is mapped to a storage medium of any one of the multiple nodes A physical address of storage space.
[0059] In some possible implementation manners, in step S101, the storage space includes internal memory and/or video memory, and the constructing a cache space in the storage space of each node of the plurality of nodes includes:
[0060] A cache space and a local storage space are constructed in the memory and/or video memory of each of the multiple nodes.
[0061] As an example, the computing cluster includes node 1 and node 2. Unified memory addressing will address the heterogeneous memory access space of the two nodes from 0 to 399, where node 1-CPU (0-99), node 1-GPU (100 - 199), node 2-CPU (200-299), node 2-GPU (300-399), node 1 and node 2 maintain the same global address mapping view, and can obtain node information of a certain segment of address, mapping Information, etc., to achieve free access to data.
[0062] Furthermore, there are many ways to implement the unified addressing, such as software implementation, hardware implementation, software and hardware coordinated implementation, and so on. Taking a pure software implementation as an example, unified memory/video memory addressing can use a tree structure/hash structure to implement addressing and indexing, and synchronize between nodes.
[0063] S102. Build a cache space in the storage space of each of the multiple nodes.
[0064] It can be understood that the cache space may be a storage space with a fixed size, or a storage space adjusted in real time according to the size of data required for caching. Specifically, in step S102, a cache management system may be designed to manage the local caches of each node, be responsible for cache application, release, and maintain the data mapping relationship in the cache.
[0065] S103. In response to an application program in any target node among the plurality of nodes initiating a data access request, determine demand data corresponding to the data access request.
[0066] S104. Determine the first node where the demand data is located and the target physical address according to the address of the demand data.
[0067] S105. If the first node is not the target node, query whether the required data is matched in the local cache space of the target node according to the target physical address.
[0068] S106. In a case where the local cache space does not match the demand data, transmit the demand data stored at the target physical address in the first node to the local cache space of the target node.
[0069] In some possible implementation manners, in step S103, transmitting the demand data stored at the target physical address in the first node to the local cache space of the target node includes:
[0070] The required data stored in the target physical address in the first node is transmitted to the local cache space of the target node through direct network copy or remote direct data access or video memory remote direct data access.
[0071] The method of selecting the direct network copy, the remote direct data access, or the video memory remote direct data access may be determined according to the current network conditions, which is not limited in the present disclosure.
[0072] S107. Map the address storing the required data in the local cache space to the target physical address, so that the application obtains the required data from the local cache space of the target node.
[0073] In the embodiment of the present disclosure, by uniformly addressing the storage media in multiple nodes in the computing cluster, the application can directly access data on any node without caring about the actual storage location of the data, and without actively performing data storage. Copying effectively solves the problem of poor flexibility in cross-node data access. In addition, by constructing a cache space and locally constructing a remote data cache, the application can obtain performance similar to that of local node data access when performing cross-node data access, further improving data transmission efficiency.
[0074] In some optional embodiments, the method includes:
[0075] If the first node is the target node, determine that the target physical address points to the local storage space of the target node, and return the demand data stored in the local storage space of the target node to the application.
[0076] With the above solution, if the demand data required by the application is stored in the local storage of the node running the application, the demand data can be directly obtained from the local storage of the node and returned to the application, so that the application Programs can quickly get the data they need.
[0077] In some other optional embodiments, the method also includes:
[0078] According to historical access characteristics, prefetch data and prefetch scale are determined based on the prefetch algorithm;
[0079] The data in the physical address corresponding to the prefetched data is cached in the local cache space according to the prefetched scale.
[0080] Those skilled in the art should know that there are many kinds of prefetching algorithms in related technologies, such as common data prefetching algorithms, interval prefetching algorithms, and some machine learning-based algorithms disclosed in the academic circle this year. The specific algorithms are disclosed in this disclosure. There is no limit to this.
[0081] Further, the prefetching algorithm can be realized by software, or can be solidified into a customized FPGA, which will have higher algorithm performance and reduce the consumption of other system resources at the same time.
[0082] Using the above solution, based on historical access data, prefetch data to the cache space of each node through the prefetch algorithm, so that the application program of each node can obtain the required data from the local cache space more quickly when requesting data. There is no need to retransmit data from other nodes to the local, effectively improving the efficiency of data acquisition.
[0083] Optionally, the method also includes:
[0084] After caching the data in the physical address corresponding to the prefetched data into the local cache space according to the prefetch scale, mapping the address of the storage space where the prefetched data is cached in the local cache space to the The physical address corresponding to the prefetched data.
[0085] It can be understood that the cache management system can be used to update the cache management, add a cache mapping relationship and other processes, so as to realize the above address mapping process.
[0086] With this solution, after caching the data of the remote node to the local cache, the address of the cache is mapped to the physical address corresponding to the prefetch data, so that the required data of the node can be stored at the physical address corresponding to the prefetch data. , accurately find the required data from the local cache without obtaining from the remote node, which reduces the transmission overhead and improves the efficiency of data transmission.
[0087] In yet other optional embodiments, the constructing a cache space in the internal memory and/or video memory of each of the multiple nodes includes:
[0088] Constructing a cache space in the internal memory and video memory of each of the plurality of nodes, the cache space includes a memory cache and a video memory cache, and setting the priorities of the video memory cache and the memory cache;
[0089] The transmitting the demand data stored in the target physical address in the first node to the local cache space of the target node includes:
[0090] According to the priority of the display memory cache and the memory cache, if the cache space with higher priority is not full, cache the required data stored at the target physical address into the cache space with higher priority .
[0091] Those skilled in the art should know that the actual allocation of the cache is constrained by the actual physical capacity and cost. For example, in the current system, the cost of expanding the memory on the server is far less than the cost of expanding the GPU memory, which is also doomed for us not to It is possible to put all the prefetched data into the cache of the video memory. Using the above scheme, by designing a multi-level cache and storing the cache data hierarchically, the storage space of different storage media is effectively used, and the storage efficiency of the cache data is improved.
[0092] Optionally, the setting the priorities of the video memory cache and the memory cache includes:
[0093] Setting the video memory cache as the first priority, and setting the memory cache as the second priority;
[0094] According to the priorities of the video memory cache and the memory cache, if the cache space with a higher priority is not full, cache the required data stored at the target physical address to a cache with a higher priority Cache space includes:
[0095] In the case where the storage space of the video memory cache is full, if the required data stored at the target physical address is not fully cached, the required data stored at the target physical address has not been cached and completed. Memory cache storage space.
[0096] It can be understood that setting the video memory cache to a higher-level cache, or setting the memory cache to a higher-level cache can be selected according to actual needs. The basic principle can be that the higher the access performance of data in the medium, the higher the cache The lower the level, the higher the priority in cache allocation. The above solution only provides a solution for setting the video memory cache to a higher level cache for easy understanding.
[0097] By adopting the above solution, the data is preferentially cached in a storage medium with a higher cache level, and the data can be preferentially stored in a storage medium with higher access performance, so as to ensure better access performance when the application program obtains the data.
[0098] In other optional embodiments, the transmitting the demand data stored at the target physical address in the first node to the local cache space of the target node further includes:
[0099] determining that the storage space pointed to by the target physical address is memory or video memory;
[0100] In the case where the storage space pointed to by the target physical address is a memory, the priority of the memory cache is set to the first priority, and the priority of the video memory cache is set to the second priority;
[0101]In the case where the storage space pointed to by the target physical address is a video memory, the priority of the video memory cache is set to the first priority, and the priority of the memory cache is set to the second priority;
[0102] The priority of the first priority is higher than the priority of the second priority.
[0103] Using the above scheme, according to the storage medium where the data required by the application is located, the priority of the memory cache and the video memory cache is dynamically adjusted. In a multi-level cache system, if the video memory is accessed, the local video memory cache is a first-level cache , when subsequent applications access data, no data transmission is required, which further improves data access efficiency and reduces data access time.
[0104] In order to make it easier for those skilled in the art to understand the technical solutions provided by the embodiments of the present disclosure, the embodiments of the present disclosure provide the following figure 2 The shown is a schematic diagram of data transmission for a computing cluster according to an exemplary embodiment, as shown in figure 2 As shown, the computing cluster includes node 1, node 2 and high-speed network card. The storage space of node 1 and node 2 is divided into local storage and local cache. Local storage also includes local memory and local video memory. Local cache also includes memory cache and Video memory cache. It can be understood that there may be multiple other nodes in the computing cluster except node 1 and node 2, figure 2 It is only shown to facilitate the understanding of those skilled in the art.
[0105] In addition, a unified memory/video memory addressing system and a cache management/prefetch system are also designed. Among them, the unified memory/video memory addressing system can uniformly address each storage medium of each node, for example, address the heterogeneous memory access space of nodes 1 and 2 from 0 to 399, where node 1-CPU (0-99 ), node 1-GPU (100 - 199), node 2-CPU (200-299), node 2-GPU (300-399), node 1 and node 2 maintain the same global address mapping view, and can obtain a certain segment Address node information, mapping information, etc., to achieve free access to data, and each node can also perform local access through the unified memory/video memory addressing system, so that the application program can obtain the data stored in the local storage, and can manage/pre-order through the cache The access system performs remote access and cache reading and writing, so as to cache the data in the remote node to the local cache through the high-speed network card.
[0106] Further, based on figure 2 As shown in the schematic diagram of data transmission for computing clusters, this disclosure also provides image 3 A schematic diagram of a unified memory/video memory addressing system for data transmission shunting process shown, and such as Figure 4 A schematic diagram of a cache management/prefetch system data transmission process is shown.
[0107] like image 3 As shown, the application sends the data access request to the unified memory/video memory addressing system, and the unified memory/video memory addressing system judges whether the required data corresponding to the data access request is stored in the local storage, namely figure 2 The local memory or local video memory shown, if yes, perform local access through unified memory/video memory addressing, so that the application program obtains its required data from the local memory or local video memory, if not, send the data access request to Cache management/prefetch system for e.g. Figure 4 process shown.
[0108] like Figure 4 As shown, the data access request is received by the cache management module, and whether the cache hit is judged by the cache management module, that is, it is determined figure 2 Whether there is required data in the shown memory cache and video memory cache, if the cache hits, the application program will directly access the required data in the local cache; if the cache misses, the data access request will be further sent to the prefetch module to Based on the prefetch algorithm, the prefetch module obtains data from the remote node where the required data is located according to the cache classification and capacity allocation, and caches it in the local cache. After the data transmission is completed, the cache is updated through the cache management, so that the application can Access to demand data transferred to local cache.
[0109] Based on the same inventive concept, Figure 5 It is a block diagram of a data transmission device 50 for a computing cluster according to an exemplary embodiment. The device can be a central processing unit or a part of the central processing unit in the computing cluster, or can be used as any node in the computing cluster A part of, the device is set in a computing cluster including a plurality of nodes, the device includes:
[0110] A unified addressing module 51, configured to uniformly address the storage spaces of the storage media of the plurality of nodes to obtain addressing information, and each address in the addressing information is mapped to any of the plurality of nodes a physical address of a storage space of a node's storage medium; and,
[0111] A cache space construction module 52, configured to construct a cache space in the storage space of each node of the plurality of nodes;
[0112] The first determination module 53 is configured to determine demand data corresponding to the data access request in response to an application program in any target node among the plurality of nodes initiating a data access request;
[0113] The second determination module 54 is configured to determine the first node where the demand data is located and the target physical address according to the address of the demand data;
[0114] A hit query module 55, configured to query whether the required data is hit in the local cache space of the target node according to the target physical address if the first node is not the target node;
[0115] A transmission module 56, configured to transmit the demand data stored in the target physical address in the first node to the local cache space of the target node when the local cache space misses the demand data;
[0116] The mapping module 57 maps the address storing the required data in the local cache space to the target physical address, so that the application obtains the required data from the local cache space of the target node.
[0117] Optionally, the storage space includes internal memory and/or video memory, and the cache space construction module 52 is specifically used for:
[0118] Constructing a cache space and a local storage space in the internal memory and/or video memory of each of the plurality of nodes;
[0119] The device 50 also includes:
[0120] A third determining module, configured to determine that the target physical address points to the local storage space of the target node when the first node is the target node, and store the target physical address in the local storage space of the target node The demand data is returned to the application.
[0121] Optionally, the device 50 also includes:
[0122] The fourth determination module is used to determine the prefetch data and prefetch scale based on the prefetch algorithm according to the historical access characteristics;
[0123] The prefetch module is configured to cache the data in the physical address corresponding to the prefetch data to the local cache space according to the prefetch scale.
[0124] Optionally, the device 50 also includes:
[0125] a remapping module, configured to, after caching the data in the physical address corresponding to the prefetched data into the local cache space according to the prefetch scale, cache the storage of the prefetched data in the local cache space The address mapping of the space is the physical address corresponding to the prefetched data.
[0126] Optionally, the cache space construction module 52 is specifically used for:
[0127] Constructing a cache space in the internal memory and video memory of each of the plurality of nodes, the cache space includes a memory cache and a video memory cache, and setting the priorities of the video memory cache and the memory cache;
[0128] The transmission module 56 is used for:
[0129] According to the priority of the display memory cache and the memory cache, if the cache space with higher priority is not full, cache the required data stored at the target physical address into the cache space with higher priority .
[0130] Optionally, the cache space construction module 52 is also used for:
[0131] Setting the video memory cache as the first priority, and setting the memory cache as the second priority;
[0132] The transmission module 56 is also used for:
[0133] When the storage space of the video memory cache is full, if the required data stored at the target physical address is not fully cached, then the required data stored at the target physical address is not cached for completed data Cache to memory cache storage space.
[0134] Optionally, the cache space construction module 52 is also used for:
[0135] determining that the storage space pointed to by the target physical address is memory or video memory;
[0136] In the case where the storage space pointed to by the target physical address is a memory, the priority of the memory cache is set to the first priority, and the priority of the video memory cache is set to the second priority;
[0137] In the case where the storage space pointed to by the target physical address is a video memory, the priority of the video memory cache is set to the first priority, and the priority of the memory cache is set to the second priority;
[0138] The priority of the first priority is higher than the priority of the second priority.
[0139] Optionally, the transmission module 56 is specifically used for:
[0140] The required data stored in the target physical address in the first node is transmitted to the local cache space of the target node through direct network copy or remote direct data access or video memory remote direct data access.
[0141] Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.
[0142] Image 6 It is a block diagram of an electronic device 600 shown according to an exemplary embodiment. like Image 6 As shown, the electronic device 600 may include: a processor 601 and a memory 602 . The electronic device 600 may also include one or more of a multimedia component 603 , an input/output (I/O) interface 604 , and a communication component 605 .
[0143]Wherein, the processor 601 is used to control the overall operation of the electronic device 600, so as to complete all or part of the steps in the above-mentioned data transmission method for computing clusters. The memory 602 is used to store various types of data to support the operation of the electronic device 600, for example, these data may include instructions for any application or method operating on the electronic device 600, and application-related data, For example, demand data, messages sent and received, pictures, audio, video, etc. The memory 602 can be implemented by any type of volatile or non-volatile memory device or their combination, such as Static Random Access Memory (Static Random Access Memory, referred to as SRAM), Electrically Erasable Programmable Read-Only Memory (EPROM) Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), Erasable Programmable Read-Only Memory (Erasable Programmable Read-Only Memory, referred to as EPROM), Programmable Read-Only Memory (Programmable Read-Only Memory, referred to as PROM), read-only Memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. Multimedia components 603 may include screen and audio components. The screen can be, for example, a touch screen, and the audio component is used for outputting and/or inputting audio signals. For example, an audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 602 or sent through the communication component 605 . The audio component also includes at least one speaker for outputting audio signals. The I/O interface 604 provides an interface between the processor 601 and other interface modules, which may be a keyboard, a mouse, buttons, and the like. These buttons can be virtual buttons or physical buttons. The communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC for short), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or one or a combination of them , is not limited here. Therefore, the corresponding communication component 605 may include: a Wi-Fi module, a Bluetooth module, an NFC module and the like.
[0144] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), digital signal processors (Digital Signal Processor, DSP for short), digital signal processing equipment (Digital Signal Processing Device, referred to as DSPD), programmable logic device (Programmable Logic Device, referred to as PLD), field programmable gate array (Field Programmable Gate Array, referred to as FPGA), controller, microcontroller, microprocessor or other electronic components to achieve , for executing the above-mentioned data transmission method for computing clusters.
[0145] In another exemplary embodiment, there is also provided a computer-readable storage medium including program instructions. When the program instructions are executed by a processor, the steps of the above-mentioned data transmission method for a computing cluster are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 602 including program instructions, and the above-mentioned program instructions can be executed by the processor 601 of the electronic device 600 to complete the above-mentioned data transmission method for computing clusters.
[0146] Figure 7 It is a block diagram of an electronic device 700 shown according to an exemplary embodiment. For example, the electronic device 700 may be provided as a server. refer to Figure 7 , the electronic device 700 includes a processor 722 , the number of which may be one or more, and a memory 732 for storing computer programs executable by the processor 722 . The computer program stored in memory 732 may include one or more modules each corresponding to a set of instructions. In addition, the processor 722 may be configured to execute the computer program, so as to execute the above-mentioned data transmission method for a computing cluster.
[0147] In addition, the electronic device 700 may further include a power supply component 726 and a communication component 750, the power supply component 726 may be configured to perform power management of the electronic device 700, and the communication component 750 may be configured to implement communication of the electronic device 700, for example, wired or wireless communication. In addition, the electronic device 700 may further include an input/output (I/O) interface 758 . The electronic device 700 can operate based on an operating system stored in the memory 732, such as Windows Server TM , Mac OSX TM , Unix TM , Linux TM and many more.
[0148] In another exemplary embodiment, there is also provided a computer-readable storage medium including program instructions. When the program instructions are executed by a processor, the steps of the above-mentioned data transmission method for a computing cluster are implemented. For example, the non-transitory computer-readable storage medium can be the above-mentioned memory 732 including program instructions, and the above-mentioned program instructions can be executed by the processor 722 of the electronic device 700 to complete the above-mentioned data transmission method for computing clusters.
[0149] In another exemplary embodiment, there is also provided a computer program product comprising a computer program executable by a programmable device, the computer program having a function for performing the above-mentioned The code portion of the data transfer method for the compute cluster.
[0150] The preferred embodiments of the present disclosure have been described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure. These simple modifications all belong to the protection scope of the present disclosure.
[0151] In addition, it should be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner if there is no contradiction. The combination method will not be described separately.
[0152] In addition, various implementations of the present disclosure can be combined arbitrarily, as long as they do not violate the idea of the present disclosure, they should also be regarded as the content disclosed in the present disclosure.
