Cooperative computing system and method based on lightweight messaging and shared local memory

Abstract

The application belongs to the field of high-performance microprocessors, and relates to a collaborative computing system and method based on lightweight messages and shared local memories. S1: a main core executes a core algorithm; S2: it is judged whether to execute to an accelerated computing part, if not, return to step S1, if yes, execute step S3; S3: the main core continues to execute the core algorithm in the program, and simultaneously sends a lightweight message to a data transfer processor; S4: the data transfer processor extracts original data at a corresponding position in a local data memory and forwards the original data to a heterogeneous core; S5: the heterogeneous core performs corresponding calculation to obtain corresponding result data, and returns the result data to the data transfer processor; S6: the data transfer processor sends the result data to the local data memory, and the local data memory stores the result data to the corresponding position for the main core to extract. The application provides a collaborative computing system and method based on lightweight messages and shared local memories, and the system structure is simple and the interactive performance is high.

Description

Technical Field

[0001] This invention belongs to the field of high-performance microprocessors, specifically relating to a collaborative computing system and method based on lightweight messaging and shared local memory. Background Technology

[0002] Asynchronous concurrent processing is often used as an effective method to accelerate processing in some specialized fields. A common asynchronous processing structure in the field of artificial intelligence is CPU+TPU. The CPU and TPU are asynchronous concurrent processing structures that share main memory. The CPU is responsible for the serial and control flow processing, while the TPU accelerates the special operations in the field of artificial intelligence, especially convolution operations. The two mainly exchange data through main memory.

[0003] In the fields of artificial intelligence and scientific computing, there are some core computations with distinct characteristics. For example, convolution operations in the field of intelligence are a very typical type of computation. For these domain-specific core computations, conventional processors cannot handle them efficiently. Some accelerators have achieved high processing performance by supporting these specialized computations. However, the interaction cost between the main processor and the accelerator is relatively high because they perform collaborative computations through main memory.

[0004] Furthermore, since heterogeneous chips primarily interact with each other via main memory, the interaction latency is relatively long, and frequent interactions significantly reduce the performance gains. On-chip heterogeneous core collaboration can reduce the interaction latency between the acceleration core and the main processing core; however, the relatively complete implementation of the acceleration core results in significant implementation overhead. Therefore, proposing an interaction mechanism for heterogeneous core collaborative computing with low interaction cost and low implementation overhead to address the diverse data processing operation requirements in scientific computing and artificial intelligence fields is of significant practical importance. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a collaborative computing system and method based on lightweight messaging and shared local memory, which features a simple system structure and high interactive performance.

[0006] The present invention adopts the following technical solution:

[0007] A collaborative computing system based on lightweight messaging and shared local memory includes a main core module, a data transfer and processing module, a local data storage module, and a heterogeneous core module. The main core module, the data transfer and processing module, and the local data storage module are connected in sequence. The data transfer and processing module is also connected to the heterogeneous core module, and the main core module is also connected to the local data storage module.

[0008] The local data storage module is used to store raw data and result data;

[0009] The main core module is used to continuously execute the core algorithms in the program, and when it reaches the accelerated calculation part, it sends a lightweight message containing the position and length information of the original data area and the result data area to the data relay processing module.

[0010] The data relay processing module is used to extract the original data at the corresponding location in the local data storage module according to the lightweight message, and forward it to the heterogeneous core module;

[0011] The heterogeneous core module is used to perform corresponding calculations based on the received raw data to obtain the corresponding result data, and then return the result data to the data transfer and processing module.

[0012] The data transfer and processing module is also used to send the result data to the local data storage module, which stores the result data in the corresponding location for retrieval by the main core module.

[0013] As a preferred solution, the system includes multiple heterogeneous core modules, and the data transfer and processing module is connected to each of the multiple heterogeneous core modules.

[0014] The lightweight message sent from the main core module to the data relay processing module also includes algorithm type information;

[0015] The data relay processing module is used to forward the extracted raw data to the heterogeneous core module that executes the corresponding algorithm type based on the algorithm type information in the lightweight message.

[0016] As a preferred embodiment, the plurality of heterogeneous core modules include a heterogeneous core module for convolution operations and a heterogeneous core module for matrix transpose operations.

[0017] As a preferred option, the heterogeneous core module includes interconnected computing units and data storage units, both of which are connected to the data transfer and processing module.

[0018] The calculation unit is used to perform corresponding calculations based on the received raw data to obtain the corresponding result data;

[0019] The data temporary storage unit is used to temporarily store the result data during the calculation process of the calculation unit, and to return all the result data to the data transfer and processing module after the calculation unit finishes its calculation.

[0020] As a preferred embodiment, the data relay processing module includes a message forwarding submodule and a data relay module. The data relay module includes a descriptor management and parsing unit, a data loading unit, a data write-back unit, and a local storage access interface unit. The message forwarding submodule is connected to the main core module and the descriptor management and parsing unit, respectively. The descriptor management and parsing unit is also connected to the data loading unit and the data write-back unit, respectively. The local storage access interface unit is connected to the data loading unit, the data write-back unit, and the local data storage module, respectively.

[0021] As a preferred embodiment, the computing unit is connected to the data loading unit, and the data temporary storage unit is connected to the data write-back unit.

[0022] A collaborative computing method based on lightweight messaging and shared local memory is also disclosed. Based on the aforementioned collaborative computing system based on lightweight messaging and shared local memory, the method includes the following steps:

[0023] S1, the core algorithm in the main core execution program;

[0024] S2. Determine whether the main core has executed the accelerated computing part. If not, return to step S1; if so, execute step S3.

[0025] S3, the main core continues to execute the core algorithm in the program, and at the same time sends a lightweight message containing the position and length information of the original data area and the result data area to the data relay processor;

[0026] S4. The data relay processor extracts the original data at the corresponding location from the local data memory according to the lightweight message and forwards it to the heterogeneous core.

[0027] S5. The heterogeneous core performs corresponding calculations based on the received raw data to obtain the corresponding result data, and returns the result data to the data transfer processor.

[0028] S6. The data relay processor sends the result data to the local data memory, which stores the result data in the corresponding location for retrieval by the main core.

[0029] As a preferred embodiment, in step S3, the lightweight message also includes algorithm type information;

[0030] In step S4, the data relay processor also forwards the extracted raw data to the heterogeneous core that executes the corresponding algorithm type based on the algorithm type information in the lightweight message.

[0031] As preferred options, algorithm types include convolution operations and matrix transpose operations.

[0032] As a preferred option, heterogeneous cores include heterogeneous cores for convolution operations and heterogeneous cores for matrix transpose operations.

[0033] The beneficial effects of this invention are:

[0034] The heterogeneous core collaborative computing and interaction mechanism implemented in this invention can accelerate computing and data processing, enable asynchronous execution between heterogeneous cores, and facilitate collaborative computing and fine-grained interaction between cores.

[0035] The interactive mechanism implemented in this invention has a certain granularity (instruction) control method, is flexible in programming, and has good versatility.

[0036] This invention accesses and exchanges data through local storage, and can exchange data from a few bytes to a few kilobytes. Therefore, the access latency is only a few ticks, and the access process is smooth and bubble-free (usually there are few access sources and few conflicts). Local storage interaction is relatively lightweight.

[0037] In this invention, the messages sent by the main core only include algorithm type information, the location and length information of the original data area and the result data area, and only carry necessary control information, i.e., they are forwarded via lightweight messages.

[0038] This invention sets up multiple heterogeneous core modules that can execute different algorithms, making the collaborative computing system more adaptable on the basis of the above advantages. It can adapt to programs with multiple algorithms, thereby meeting the diverse data processing operation needs in the fields of scientific computing and artificial intelligence. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of the structure of the collaborative computing system based on lightweight messaging and shared local memory described in this invention;

[0041] Figure 2 This is a diagram illustrating sequential processing and asynchronous concurrent processing;

[0042] Figure 3 This is a flowchart of the collaborative computing method based on lightweight messages and shared local memory described in this invention. Detailed Implementation

[0043] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0044] Example 1:

[0045] Reference Figure 1 This embodiment provides a collaborative computing system based on lightweight messaging and shared local memory, including a main core module, a data transfer and processing module, a local data storage module, and a heterogeneous core module. The main core module, the data transfer and processing module, and the local data storage module are connected in sequence. The data transfer and processing module is also connected to the heterogeneous core module, and the main core module is also connected to the local data storage module.

[0046] The local data storage module is used to store raw data and result data;

[0047] The main core module is used to continuously execute the core algorithms in the program, and when it reaches the accelerated calculation part, it sends a lightweight message containing the position and length information of the original data area and the result data area to the data relay processing module.

[0048] The data relay processing module is used to extract the original data at the corresponding location in the local data storage module according to the lightweight message, and forward it to the heterogeneous core module;

[0049] The heterogeneous core module is used to perform corresponding calculations based on the received raw data to obtain the corresponding result data, and then return the result data to the data transfer and processing module.

[0050] The data transfer and processing module is also used to send the result data to the local data storage module, which stores the result data in the corresponding location for retrieval by the main core module.

[0051] Specifically:

[0052] The system includes multiple heterogeneous core modules, and the data transfer and processing module is connected to multiple heterogeneous core modules respectively;

[0053] The lightweight message sent from the main core module to the data relay processing module also includes algorithm type information;

[0054] The data relay processing module is used to forward the extracted raw data to the heterogeneous core module that executes the corresponding algorithm type based on the algorithm type information in the lightweight message.

[0055] Scientific computing and artificial intelligence have various needs for accelerated computing and data processing. Collaborative processing among heterogeneous cores can hide latency and improve overall processing performance. Analysis of the computing and data processing requirements of typical applications in scientific computing and artificial intelligence reveals that to maximize the efficiency of batch computing and data processing, it is necessary to provide batch-sized datasets for processing, and then perform accelerated computing on these datasets. Obtaining batch datasets typically requires data format transformation, data content transformation, and efficient data access of the original data. Accelerated computing is then selected and designed based on the characteristics of typical applications. (Refer to...) Figure 2 As shown, taking asynchronous concurrent processing of data processing and computation as an example, this invention demonstrates that asynchronous execution between heterogeneous cores can achieve better processing performance compared to sequential execution in traditional core processing. The solution provided by this invention primarily alleviates the interaction costs of collaborative computing between heterogeneous cores by simplifying the complexity of the structure to improve feasibility.

[0056] The heterogeneous core collaborative computing and interaction mechanism implemented in this invention can accelerate computing and data processing, enable asynchronous execution between heterogeneous cores, and facilitate collaborative computing and fine-grained interaction between cores.

[0057] The interactive mechanism implemented in this invention has a certain granularity (instruction) control method, is flexible in programming, and has good versatility.

[0058] This invention accesses and exchanges data through local storage, and can exchange data from a few bytes to a few kilobytes. Therefore, the access latency is only a few ticks, and the access process is smooth and bubble-free (usually there are few access sources and few conflicts). Local storage interaction is relatively lightweight.

[0059] In this invention, the messages sent by the main core only include algorithm type information, the location and length information of the original data area and the result data area, and only carry necessary control information, i.e., they are forwarded via lightweight messages.

[0060] This invention employs multiple heterogeneous core modules capable of executing different algorithms. This enhances the adaptability of the collaborative computing system beyond its aforementioned advantages, enabling it to accommodate programs with various algorithms and thus meet the diverse data processing requirements in scientific computing and artificial intelligence fields.

[0061] This invention achieves asynchronous collaborative computing and interaction between two heterogeneous cores through lightweight message forwarding and shared local memory. The implemented heterogeneous core collaborative computing instruction ASP (Asynchronous Synergistic Processing) is shown below:

[0062] ASP Rb, #type

[0063] The `#type` parameter indicates the processing algorithm type. Depending on the application domain the processor is targeting, it can support multiple cooperative algorithm types. It can be a convolution operation or a matrix transpose operation. Taking the matrix transpose operation as an example, `Rb` is used to specify the specific operation and indicate relevant information about the data area, including the algorithm type, the address of the original data area, the address of the result area, the number of transpose rows, the data processing precision, and the handshake signal address, etc.

[0064] When the program runs, the main core primarily executes the core algorithm, handling data processing and serial program flow control. When it reaches the accelerated computation section, it sends an asynchronous computation request to the message forwarding component via the ASP instruction. This request is lightweight; as described earlier, it only carries essential algorithm type information and information indicating the location and length of the raw and result data areas. The message forwarding component forwards the request to the target computation core (usually the computation acceleration engine), driving the engine to execute. Upon receiving the request, the target computation core needs to parse and manage the request, determine the appropriate processing based on the request's content, and retrieve initial data from the shared local data storage according to the location and length of the raw data area indicated in the request for corresponding computation acceleration processing. During processing, the computation result is stored in the result area location indicated in the request within the local data storage. After all data processing is complete, a response word (placed in the shared local data storage) or a synchronization signal (via a fast synchronization mechanism connected to hardware) is sent to notify the main core that the processing result can be read.

[0065] This lightweight interaction mechanism, which supports collaborative computing between different cores, is characterized by its simple structure, good interaction performance, and good versatility.

[0066] Specifically, this is achieved through the following structure:

[0067] Reference Figure 1 As shown, the multiple heterogeneous core modules include a convolution operation heterogeneous core module, a matrix transpose operation heterogeneous core module, and so on.

[0068] The heterogeneous core module includes interconnected computing units and data storage units, both of which are connected to the data transfer and processing module.

[0069] The calculation unit is used to perform corresponding calculations based on the received raw data to obtain the corresponding result data;

[0070] The data temporary storage unit is used to temporarily store the result data during the calculation process of the calculation unit, and to return all the result data to the data transfer and processing module after the calculation unit finishes its calculation.

[0071] It should be noted that, Figure 1 Only one heterogeneous core module is shown in the image.

[0072] The data relay processing module includes a message forwarding submodule and a data relay module. The data relay module includes a descriptor management and parsing unit, a data loading unit, a data write-back unit, and a local storage access interface unit. The message forwarding submodule is connected to the main core module and the descriptor management and parsing unit, respectively. The descriptor management and parsing unit is also connected to the data loading unit and the data write-back unit, respectively. The local storage access interface unit is connected to the data loading unit, the data write-back unit, and the local data storage module, respectively.

[0073] The computing unit is connected to the data loading unit, and the data temporary storage unit is connected to the data write-back unit.

[0074] In this invention, the main core module and the heterogeneous core module constitute a heterogeneous computing device. The main core module drives the heterogeneous core module to perform processing via messages. The heterogeneous core module is responsible for computation and data processing, and shares local data storage with the main core module. The heterogeneous core module receives asynchronous processing requests from the main core module, accesses data in the local data storage for computation, and after processing, interacts with the main core module through a handshake mechanism (response word or synchronization). The descriptor management and parsing unit mainly manages and parses the request descriptors sent by the ASP, obtaining the required algorithm type information, the starting address and length of the original / result data area, and the handshake method after processing. The data loading module and the data write-back module complete the reading of the initial data and the writing back of the result data.

[0075] Example 2:

[0076] Reference Figure 3 As shown, this embodiment provides a collaborative computing method based on lightweight messaging and shared local memory, including the following steps:

[0077] S1, the core algorithm in the main core execution program;

[0078] S2. Determine whether the main core has executed the accelerated computing part. If not, return to step S1; if so, execute step S3.

[0079] S3, the main core continues to execute the core algorithm in the program, and at the same time sends a lightweight message containing the position and length information of the original data area and the result data area to the data relay processor;

[0080] S4. The data relay processor extracts the original data at the corresponding location from the local data memory according to the lightweight message and forwards it to the heterogeneous core.

[0081] S5. The heterogeneous core performs corresponding calculations based on the received raw data to obtain the corresponding result data, and returns the result data to the data transfer processor.

[0082] S6. The data relay processor sends the result data to the local data memory, which stores the result data in the corresponding location for retrieval by the main core.

[0083] It should be noted that the collaborative computing method based on lightweight messages and shared local memory provided in this embodiment is similar to that in Embodiment 1, and will not be described in detail here.

[0084] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope of the present invention.

Claims

1. A collaborative computing system based on lightweight messaging and shared local memory, characterized in that, It includes a main core module, a data transfer and processing module, a local data storage module, and a heterogeneous core module. The main core module, the data transfer and processing module, and the local data storage module are connected in sequence. The data transfer and processing module is also connected to the heterogeneous core module, and the main core module is also connected to the local data storage module. The local data storage module is used to store raw data and result data; The main core module is used to continuously execute the core algorithms in the program, and when it reaches the accelerated calculation part, it sends a lightweight message containing the position and length information of the original data area and the result data area to the data relay processing module. The data relay processing module is used to extract the original data at the corresponding location in the local data storage module according to the lightweight message, and forward it to the heterogeneous core module; The heterogeneous core module is used to perform corresponding calculations based on the received raw data to obtain the corresponding result data, and then return the result data to the data transfer and processing module. The data transfer and processing module is also used to send the result data to the local data storage module, which stores the result data in the corresponding location for retrieval by the main core module.

2. The collaborative computing system based on lightweight messaging and shared local memory according to claim 1, characterized in that, The system includes multiple heterogeneous core modules, and the data transfer and processing module is connected to multiple heterogeneous core modules respectively; The lightweight message sent from the main core module to the data relay processing module also includes algorithm type information; The data relay processing module is used to forward the extracted raw data to the heterogeneous core module that executes the corresponding algorithm type based on the algorithm type information in the lightweight message.

3. The collaborative computing system based on lightweight messaging and shared local memory according to claim 2, characterized in that, The multiple heterogeneous core modules include a convolution operation heterogeneous core module and a matrix transpose operation heterogeneous core module.

4. The collaborative computing system based on lightweight messaging and shared local memory according to claim 1, characterized in that, The heterogeneous core module includes interconnected computing units and data storage units, both of which are connected to the data transfer and processing module. The calculation unit is used to perform corresponding calculations based on the received raw data to obtain the corresponding result data; The data temporary storage unit is used to temporarily store the result data during the calculation process of the calculation unit, and to return all the result data to the data transfer and processing module after the calculation unit finishes its calculation.

5. The collaborative computing system based on lightweight messaging and shared local memory according to claim 4, characterized in that, The data relay processing module includes a message forwarding submodule and a data relay module. The data relay module includes a descriptor management and parsing unit, a data loading unit, a data write-back unit, and a local storage access interface unit. The message forwarding submodule is connected to the main core module and the descriptor management and parsing unit, respectively. The descriptor management and parsing unit is also connected to the data loading unit and the data write-back unit, respectively. The local storage access interface unit is connected to the data loading unit, the data write-back unit, and the local data storage module, respectively.

6. The collaborative computing system based on lightweight messaging and shared local memory according to claim 5, characterized in that, The computing unit is connected to the data loading unit, and the data temporary storage unit is connected to the data write-back unit.

7. A collaborative computing method based on lightweight messaging and shared local memory, based on the collaborative computing system based on lightweight messaging and shared local memory as described in any one of claims 1-6, characterized in that, Including the following steps: S1, the core algorithm in the main core execution program; S2. Determine whether the main core has executed the accelerated computing part. If not, return to step S1; if so, execute step S3. S3, the main core continues to execute the core algorithm in the program, and at the same time sends a lightweight message containing the position and length information of the original data area and the result data area to the data relay processor; S4. The data relay processor extracts the original data at the corresponding location from the local data memory according to the lightweight message and forwards it to the heterogeneous core. S5. The heterogeneous core performs corresponding calculations based on the received raw data to obtain the corresponding result data, and returns the result data to the data transfer processor. S6. The data relay processor sends the result data to the local data memory, which stores the result data in the corresponding location for retrieval by the main core.

8. The collaborative computing method based on lightweight messaging and shared local memory according to claim 7, characterized in that: In step S3, the lightweight message also includes algorithm type information; In step S4, the data relay processor also forwards the extracted raw data to the heterogeneous core that executes the corresponding algorithm type based on the algorithm type information in the lightweight message.

9. The collaborative computing method based on lightweight messaging and shared local memory according to claim 8, characterized in that, Algorithm types include convolution operations and matrix transpose operations.

10. The collaborative computing method based on lightweight messaging and shared local memory according to claim 9, characterized in that, Heterogeneous kernels include heterogeneous kernels for convolution operations and heterogeneous kernels for matrix transpose operations.

Images