A network design method for a multi-core particle integrated system based on network calculation

Abstract

The application discloses a network algorithm integrated multi-core particle integrated system network design method, which designs the network layer in the multi-core particle integrated system by integrating an operation unit in a network unit. The method mainly comprises router micro-architecture design, task mapping and software and hardware collaborative design; the router micro-architecture design is the hardware implementation, flow control, pipeline design and arbitration design of the router, and part of the calculation amount that needs to be aggregated is transplanted from the processor to the network for processing; the task mapping is the mapping of the tasks in the application system flow diagram to the multi-core particle integrated system; and the software and hardware collaborative design is the optimal software and hardware configuration designed in the mode of software and hardware combination with the minimum application running time as the target. The application can realize calculation in the data transmission process, reduces the data amount of network transmission and communication delay, thereby accelerating the running speed of the application on the multi-core particle integrated system.

Description

Technical Field

[0001] This invention relates to the field of integrated circuits, and in particular to a network design method for a multi-chip integrated system based on network computing. Background Technology

[0002] Multi-die integration systems integrate multiple dies (bare chips) that perform specific functions into a system-on-a-chip (SoC) using advanced packaging technology. Compared to traditional single-chip integration, multi-die integration technology offers advantages and potential in chip performance and power consumption optimization, cost, and business models. Deep neural networks (DNNs) are deployed to multi-die integration systems by splitting the training dataset / model. Each die processes different subsets of the training data in parallel to train a local copy of the model. After each die completes its processing, the training results are aggregated, the parameters are aggregated, and then broadcast to all nodes to update their local models before proceeding to the next iteration.

[0003] In multi-core integrated systems, the communication bandwidth between cores is limited, and cross-core network communication takes longer than intra-core communication. In the parameter aggregation stage of deep neural networks, if the network transmission rate is slow, it is more likely to cause network congestion, increase network latency, and thus lead to a decline in overall performance. Therefore, the network communication performance has become a performance bottleneck during training.

[0004] Existing methods, such as RingAllReduce, hierarchical All Reduce, and Double Binary Tree, address the communication bottlenecks caused by distributed training of deep neural networks. However, these methods do not directly reduce the total communication volume. Furthermore, most existing network-computing accelerators are integrated into switches. Switches handle data communication between multiple ports, and by integrating a processor into the switch to aggregate data before forwarding, the amount of data transmitted during forwarding is reduced. However, switches may connect to multiple local area networks (LANs) simultaneously. When multiple LANs are connected simultaneously, the network traffic granularity is coarse, resulting in low data aggregation and poor application performance improvement. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a network design method based on network-computing integration for multi-core integrated systems.

[0006] The objective of this invention is achieved through the following technical solution: a network design method for a multi-core integrated system based on network computing, comprising the following steps:

[0007] (1) Router micro-architecture design: Implement the hardware of the router, design the pipeline, flow control and arbitration, and add a parser, arithmetic unit and FIFO buffer queue to the router;

[0008] (2) Task mapping: Before the distributed training application of the neural network is run, the corresponding system flow graph is generated, the allocation strategy is generated based on the system flow graph, and the tasks in the system flow graph are mapped to the processor resources of the multi-core integrated system according to the allocation strategy;

[0009] (3) Software and hardware co-design: Based on the task mapping scheme in step (2), construct error model and performance model, minimize the running time of the distributed training application of neural network, and obtain the optimal software and hardware configuration with the minimum running time.

[0010] Further, the router hardware implementation in step (1) specifically involves adding a parser, an arithmetic unit, and a FIFO buffer queue to the router; the parser is a comparator used to determine the network calculation flag bit in the packet header; the arithmetic unit includes an arithmetic logic unit, a buffer, and a register; the arithmetic logic unit consists of multiple parallel single-precision / double-precision floating-point adders used to calculate the exponent and mantissa in parallel; the buffer is used to store the input data of the arithmetic logic unit; the register is used to store the source address, destination address, and network calculation flag bit in the packet header; and the FIFO buffer queue is used to store the data packets output by the arithmetic unit.

[0011] Furthermore, the router pipeline design in step (1) is specifically as follows: an additional network computing pipeline is added before routing selection; when the data packet enters the router, it first enters the network computing pipeline. If the data packet needs network computing, it is transmitted to the computing unit for calculation. If the data packet does not need network computing, it enters the next pipeline for routing selection.

[0012] Further, the router flow control in step (1) specifically involves the following: when a data packet is transmitted to the router, the parser first compares the network calculation flag bit in the packet header of the input port with the preset network calculation flag bit. If they are equal, the entire data packet is split, and the packet header is stored in the register of the arithmetic unit, while the data portion is stored in the buffer inside the arithmetic unit. After the data portion enters the buffer, a judgment is triggered. When the amount of data in the buffer is greater than 1, the data in the buffer is transmitted to the arithmetic logic unit for floating-point addition. When the amount of data in the buffer is 1, the system waits for the next data that needs to be calculated in the network to enter the buffer.

[0013] Furthermore, the router arbitration design in step (1) is as follows: the number of input ports of the cross switch in the router is increased from the original five ports to six ports; the arbitration part of the router is priority arbitration, which polls each input port according to the priority arbitration algorithm to determine the weight value of the port with a packet forwarding request, and after the polling is completed, the port with the largest weight value is determined by the comparator as the input of the cross switch and output after arbitration.

[0014] Furthermore, step (2) includes the following sub-steps:

[0015] (2.1) Set one of the processors in the multi-core integrated system as the resource manager to calculate the optimal allocation strategy and allocate tasks;

[0016] (2.2) Initialize the data flow graph G = (V, E, w) according to the task partitioning of the distributed training application of the neural network; where V is a graph node, E is a directed edge, and w is the weight; the graph node V is used to represent the task, and for tasks with computational dependencies V i and V j , construct V i To V j The directed edge E ij E ij weight w ij For V i To V j The transmitted communication volume; the data flow graph G has two virtual vertices r and d, r as the starting point and connected to the first-layer task, and d as the ending point and connected to the last-layer task; define a resource set S, and sequentially traverse the processor resources s contained in each chip in the multi-chip integrated system and add them to the resource set S, where s i ∈S, where S contains the location information of processor resources in the multi-chip integrated system;

[0017] (2.3) Based on the data flow graph G obtained in step (2.2), G is traversed in the order of breadth-first search, and the traversed nodes are added to the task queue L in sequence; the first task r of the task queue L is assigned to the resource manager.

[0018] (2.4) Based on the task queue L obtained in step (2.3), traverse the task queue L sequentially; for the current task l i ∈L, select the previous task l from the resource set S obtained in step (2.2). i-1 The allocated processor resources s i-1 The processor resource with the shortest distance and currently idle is selected as the task. iThe allocated object is marked as already occupied. Since the multi-core integrated system has a mesh structure both within and between cores, there may be multiple processor resources with equal and shortest distances. These equal and shortest-distance processor resources are stored as a resource set P, and processor resources in resource set P are selected sequentially as task l. i The object to be allocated;

[0019] (2.5) Iterate through the task queue L sequentially, repeating step (2.4) until all tasks l in the task queue L are completed. i Allocation complete; during the traversal, a task allocation strategy F(G, S) is generated;

[0020] (2.6) Based on the allocation strategy F(G, S) obtained in step (2.5), minimize the application runtime T(F(G, S), h ALU ), can be expressed as minT(f(G,S), h ALU ), which is the combination with the shortest running time among multiple combinations; where f(G, S) ∈ F(G, S), h ALU For the configuration of the computing units in the router, T can be represented as the running time from r to d, including the execution time of the task in the processor and the communication time in the network; a linear regression model is established based on the task allocation strategy f(G, S), and a prediction module is established for the execution time and communication time. The prediction model is obtained by linear fitting through historical running data; the minimum application running time and the optimal allocation strategy are obtained by traversing F(G, S).

[0021] Furthermore, step (3) includes the following sub-steps:

[0022] (3.1) Based on the task allocation strategy obtained in step (2.6), obtain the summary gradient value of the neural network during distributed training on the multi-core integrated system, based on the current neural network model structure M. nn Using a set of hardware configurations H that include different types and numbers of arithmetic logic units ALU And establish an error model E(h) based on the accuracy loss rate α and the quality loss of the neural network. ALU M nn The relationship between the accuracy loss and the model quality loss was linearly fitted using an offline-trained error model.

[0023] (3.2) Based on the error model E(h) obtained in step (3.1) ALU M nn A performance model is built using the pruned neural network N and the truncated neural network R, so that the training time T(H) is reduced. ALU M nn The minimum value of N is required, and the quality loss value must be less than the threshold ΔE. Additionally, the area increase of the router after adding integrated network computing functionality must be within the specified range.aggr The area of ​​the original router is less than 10% of the original router's area. router ;

[0024] (3.3) Select the different hardware configurations and the pruned neural network models obtained in step (3.1) and substitute them into the running time model obtained in step (3.2). Use the branch and bound method to obtain the optimal hardware configuration and the corresponding neural network model with the minimum running time.

[0025] Further, step (3.1) includes the following sub-steps:

[0026] (3.1.1) Label the gradients of the data that need to be summarized and updated before each round of gradient summarization; set an initial precision reduction rate α. During training, the neural network will sort the gradients according to the absolute value of the gradients, and gradients of α% will have their precision reduced.

[0027] (3.1.2) Select a small batch of gradients from the gradients marked in step (3.1.1) and sort them from smallest to largest. Find the absolute value corresponding to the gradient sorted into α%, and set this absolute value as the threshold of the whole.

[0028] (3.1.3) The absolute value of the residual gradient is compared with the threshold obtained in step (3.1.2). When the absolute value of the gradient is less than the threshold, the precision of the gradient will be reduced.

[0029] (3.1.4) The difference between the training result obtained by the same number of iterations and the training result with the original precision is taken as the quality loss value. After each iteration, the corresponding quality loss will be obtained. α will gradually increase until the quality loss is greater than the acceptable range. Then, the current model is taken as the optimal software model within the acceptable error range.

[0030] Furthermore, the multi-chip integrated system is a hybrid system that packages the CPU, GPU, and LLC as chips; the LLC is used for information transmission between the CPU and GPU; when the CPU writes data, it updates the data on the LLC; when the GPU accesses data, it directly accesses the data previously written by the CPU.

[0031] Furthermore, the neural network distributed training application is used to implement distributed computing using multiple machines when training neural networks on large datasets, and to split the model and training dataset in a model parallel or data parallel manner.

[0032] Compared with existing technologies, the beneficial effects of this invention are: by adding computing units to the router, some computational workload is transferred to the network, thereby reducing the amount of data transmitted in the network, alleviating the performance bottleneck caused by the AllReduce communication mode during training and the slow data transmission between cores, and thus accelerating model training. This invention is the first to propose a network design method for a multi-core integrated system based on network-computing integration. This method is designed for networks with small traffic granularity and high data aggregation, and by constructing a performance model to configure the hardware and software, it can significantly improve the running speed of applications. Attached Figure Description

[0033] Figure 1 This is a structural diagram of the arithmetic unit in the router of this invention;

[0034] Figure 2 This is a microarchitecture diagram of the router of this invention;

[0035] Figure 3 This is an offline analysis flowchart of the error model in this invention;

[0036] Figure 4 This is a diagram illustrating the packet assembly process of the data packets of this invention in a router. Detailed Implementation

[0037] The core technology of this invention is to add a computing unit to the router and build an error model and a performance model based on it to improve the running speed of the application in a multi-core integrated system.

[0038] The proposed network design method for multi-core integrated systems based on network computing is specifically designed for CPU-GPU heterogeneous multi-core integrated systems. In particular, it encapsulates multiple CPUs, GPUs, and shared memory (LLC) as cores into a hybrid system. The LLC is used for information transmission between the CPU and GPU. When the CPU writes data, it updates the data in the LLC, allowing the GPU to directly access the data previously written by the CPU.

[0039] The multi-chip integrated system employs a two-layer on-chip network interconnection. The on-chip network within the GPU chip uses a crossbar structure, while the on-chip network within the CPU chip uses an XY-routing mesh structure. The on-chip networks between chips also use an XY-routing mesh structure. Multiple on-chip networks interconnect to form a hierarchical integrated intra-chip and inter-chip network. The inter-chip network routers in the multi-chip integrated system support integrated network and computing functions, while the intra-chip (CPU chip) network routers do not support network and computing functions.

[0040] This invention proposes a network design method for a multi-core integrated system based on network computing, comprising the following steps:

[0041] (1) Router Microarchitecture Design: Design the router's hardware implementation, pipeline, flow control, and arbitration; add a resolver, processing unit, and FIFO buffer queue to a conventional router; see [link to relevant documentation]. Figure 2 The router microarchitecture diagram includes the following sub-parts:

[0042] (1.1) Hardware Implementation: Compared to conventional routers, a parser, a processing unit, and a FIFO buffer queue are added. The network processing flag is used to distinguish whether a data packet needs to be processed in the network. The parser is a comparator used to determine the network processing flag in the data packet header. For example... Figure 1 As shown, the arithmetic unit includes: an arithmetic logic unit (ALU), a buffer, and registers. The ALU consists of multiple parallel single-precision / double-precision floating-point adders, allowing for parallel calculation of the exponent and mantissa. The buffer stores the input data to the ALU. Registers store packet headers (source address, destination address, network flags, etc.). A FIFO buffer queue stores the data packets output by the arithmetic unit.

[0043] The parser pre-defines a network computing flag, which sets a bit in the packet header to identify whether the current packet requires network computing. See also... Figure 4 The diagram illustrates the packet assembly process in a router. When a data packet arrives at the router, the parser first compares the network computing flag in the packet header of the input port with a preset network computing flag. If they are equal, the entire data packet is split, with the packet header stored in the arithmetic unit's register and the data portion stored in the arithmetic unit's internal buffer. After the data portion is processed by the ALU, it is concatenated with the packet header in the register to form a new data packet, which is temporarily stored in an added FIFO. If the data packet does not match the preset network computing flag, it directly enters the routing pipeline. The concatenated data packet is set to the highest priority, and during arbitration, high-priority packets are processed and forwarded first.

[0044] Within the buffer, a check is triggered upon data entering the buffer. If the amount of data in the buffer is greater than 1, the data in the buffer is sent to the ALU for floating-point addition. When the amount of data in the buffer is 1, it needs to wait for the next data to be calculated on the network to enter the buffer. A timer is set, starting when the amount of data in the buffer is 1. If no new data is transmitted to the buffer within the specified time, the corresponding timer is cleared, and the data portion and the header of the data packet stored in the register are concatenated to form the original data packet, which is set to low priority and temporarily stored in an added FIFO.

[0045] (1.2) Pipeline Design: To support the integrated network computing function, a network computing pipeline is added before routing selection based on the routing table. After a data packet enters the router, it first enters the network computing pipeline. Data packets that require network computing are sent to the computing unit for processing, while data packets that do not require network computing enter the next pipeline for routing selection.

[0046] (1.3) Flow Control: When a data packet is transmitted to the router, the parser first compares the network calculation flag bit in the packet header of the input port with the preset network calculation flag bit. If they are equal, the entire data packet is split, and the packet header is stored in the register of the arithmetic unit, while the data part is stored in the buffer inside the arithmetic unit. After the data part enters the buffer, a judgment is triggered. When the amount of data in the buffer is greater than 1, the data in the buffer is transmitted to the arithmetic logic unit (ALU) for floating-point addition. When the amount of data in the buffer is 1, it waits for the next data that needs to be calculated in the network to enter the buffer. A timer is set, and the timer starts when the amount of data in the buffer is 1. If no new data is transmitted to the buffer within the specified time, the corresponding timer is cleared, and the data part and the packet header stored in the register are concatenated to form the original data packet, which is set to low priority and temporarily stored in the added FIFO. When the amount of data in the buffer is greater than 1, the data part is calculated by the arithmetic logic unit (ALU) and concatenated with the packet header in the register to form a new data packet, which is temporarily stored in the added FIFO buffer queue. If the data packet does not match the preset network computing flag, it will directly enter the routing selection pipeline.

[0047] (4) Arbitration Design: The added processing unit in the router concatenates data to form new data packets. These new data packets are then output to the crossbar switch for arbitration. Therefore, the number of input ports on the crossbar switch increases from 5 to 6. The router's arbitration section uses priority arbitration. The priority arbitration algorithm is as follows: Each time, the signal with the highest priority is selected for processing; high-priority data packets are prioritized for arbitration and output by the arbitrator. Each input port is polled according to the priority arbitration algorithm to determine the weight value of each port with a data packet forwarding request. After polling, a comparator determines the port with the highest weight value as the input for crossbar switch processing, which is then output after arbitration. The weight value of high-priority data packets is higher than that of low-priority data packets.

[0048] Data packets requiring network computation are stored on different input ports of the router. The parser compares the network computation flag in the packet header with a preset flag. If they are equal, the packet header is split and stored in a register, the other packet header is discarded, and the data portion of the packet is stored in a buffer. At this point, the amount of data in the buffer is greater than 1, so the data portion is sent to the ALU for computation. The computation result is concatenated with the packet header in the register to form a new data packet, which is stored in a newly added FIFO buffer queue as input to the crossbar switch and arbitrated by the crossbar switch allocator.

[0049] (2) Task Mapping: Before the distributed training application of the neural network runs, a corresponding system flow graph is generated, where the system flow graph refers to the calling relationship, hierarchical relationship and sequential relationship of the modules in the application; the tasks in the graph are mapped to the processor resources of the multi-core integrated system according to a given allocation strategy to minimize the application runtime; specifically including the following sub-steps:

[0050] (2.1) In the multi-core integrated system, one processor is designated as the resource manager for calculating the optimal allocation strategy and allocating tasks. The data flow graph G = (V, E, w) is initialized based on the application-defined tasks; where the data flow graph refers to the flow of data generated from the system flow graph between various modules. Graph nodes V represent tasks; for tasks with computational dependencies, V... i and V j , construct V i To V j The directed edge E ij E ij weight w ij For V i To V j The transmitted communication volume G has two virtual vertices r and d. r serves as the starting point and is connected to the first-layer task, while d serves as the ending point and is connected to the last-layer task. A resource set S is defined. The processor resources s contained in each chip of the multi-chip integrated system are sequentially traversed and added to set S, where s... i ∈S contains the location information of the processor resource in the multi-chip integrated system.

[0051] (2.2) Based on the data flow graph G obtained in step (2.1), traverse G in breadth-first search order and add the traversed nodes to the task queue L in sequence. The first task r of L is assigned to the resource manager.

[0052] (2.3) Based on the task queue L obtained in step (2.2), for each task l in it i ∈L, select the previous task l from S i-1 The allocated s i-1 The processor resource with the shortest distance and currently idle is used as l iThe allocated object is marked as occupied. Since multi-core integrated systems have a mesh structure both within and between cores, there may be multiple processor resources with equal and shortest distances, stored as a resource set P. Processor resources in P are selected sequentially as l. i The object to be allocated.

[0053] (2.4) Traverse the task queue L, repeating step three until all tasks in L are assigned. During the traversal, generate multiple task assignment strategies F(G, S). Among them, the strategy that minimizes the application runtime T(F(G, S), h) is chosen. ALU ) can be represented as

[0054] T(F(G,S),h ALU )=minT(f(G,S),h ALU f(G,S)∈F(G,S)

[0055] Where h ALU This describes the configuration of the computing units in the router. For some tasks, the transmitted data is calculated by the router during network transmission, thus shortening the execution time of the tasks on the processor. T can be represented as the running time from r to d, including the execution time of the task on the processor (the original computational load minus the execution time of the computational load reduced by network computation) and the communication time. A linear regression model is established based on the task allocation strategy f(G, S). A prediction module is built for the execution time and communication time, and the prediction model is obtained by linear fitting through historical running data. Traversing F(G, S) yields the minimum application running time and the optimal allocation strategy.

[0056] (3) Hardware-Software Co-design: Based on the previously obtained mapping scheme, a performance model is established, with the optimization objective of minimizing the application's runtime, to obtain the optimal integrated hardware-software design. The hardware-software co-design involves optimizing the neural network structure and selecting the optimal computing unit configuration. Due to the limited area of ​​the router, the number of different types of ALUs (16-bit, 32-bit, and 64-bit floating-point adders) that can be configured in the computing unit is limited without affecting the router's normal routing and forwarding functions. Optimizing the neural network involves representing some gradients in the neural network with low-precision values, which are then adapted to the ALUs in the computing unit. For example, if the computing unit integrates multiple 16-bit ALUs, some of the original 32-bit floating-point gradients are converted to 16-bit floating-point values; specifically, this includes the following sub-steps:

[0057] (3.1) Constructing an Error Model: Optimizing a neural network requires ensuring model performance. An error model is established based on the relationship between accuracy loss rate and neural network quality loss. This error model measures the impact of different degrees of accuracy degradation on the error of the neural network output. This model is trained offline. For example... Figure 3As shown, the specific steps include the following:

[0058] (3.1.1) Label the data (gradients) that need to be summarized and updated. This part of the data will be computed on the network. Set an initial precision reduction rate α. During training, the neural network will sort the gradients according to their absolute values. Gradients with an initial precision reduction of α% will have their precision reduced.

[0059] (3.1.2) Before each round of gradient aggregation, select a small portion of gradients and sort them from smallest to largest. Find the absolute value corresponding to the gradient sorted into α%, and set this absolute value as the threshold of the whole.

[0060] (3.1.3) The remaining gradients will be compared with the threshold obtained in step (3.1.2). When the absolute value of the gradient is less than the threshold, the precision will be reduced.

[0061] (3.1.4) The difference between the training result obtained after the same number of iterations and the training result with the original precision is used as the quality loss value. After each iteration, a corresponding quality loss is obtained, and α will gradually increase until the quality loss is greater than the acceptable range, thus obtaining the optimal neural network model within the acceptable error range.

[0062] (3.2) Constructing a performance model: There are multiple combinations of the type and number of ALUs in the arithmetic unit. ALU According to the current combination h ALU ∈H ALU Establish the error model E(h) ALU M nn (R). Among them, M nn This is the DNN model structure, where R is the percentage of data with reduced precision. The goal is to build a performance model that minimizes the DNN training time T(H). ALU M nn With N) as the optimization objective, h is configured with computing units. ALU For decision variables, increase the model's quality loss E(h) ALU M nn (R) and the increased area of ​​the modified router aggr For constraints:

[0063] T(H ALU M nn N) = minT(h ALU M nn ,N),h ALU ∈H ALU

[0064] st E(h ALU M nn ,R)≤ΔE

[0065] Area(h ALU )≤β*Area router

[0066] The overall training time of the neural network is T(h) ALU M nn The value of E(h) can be obtained by performing performance models on the CPU and GPU. ALU M nn R) needs to be less than a given threshold ΔE, Area aggr The area of ​​the original router needs to be less than a certain proportion β. router Area aggr This can be achieved by increasing the buffer capacity, h ALU An approximate value is obtained by summing the areas occupied by registers and FIFO queues. E(h) ALU M nn The relationship between the accuracy loss and the model quality loss can be linearly fitted by the error model trained offline.

[0067] The CPU performance formula is:

[0068] T cpu =IC×CPI

[0069] Where IC is the number of instructions processed during the program, and CPI is the average number of clock cycles per instruction. IC can be obtained by simulating the execution of the code using an instruction set simulator, while CPI is determined by the CPU configuration.

[0070] The performance of a GPU is related to the computing power of the GPU processor and the memory access speed (accessing LLC). When the computation time required for a single GPU core to run is less than the kernel's memory access wait time, the GPU's performance mainly depends on the memory access speed; when the computation time is greater than the kernel's memory access wait time, the GPU's performance mainly depends on the GPU processor's computing power.

[0071] The performance formula for the GPU is:

[0072]

[0073] Mem_cycle represents the latency of a single kernel memory access, including network transfer time and LLC access time. Network transfer time is obtained by linearly fitting the distance between the source and destination addresses and the time of a single network computation. LLC access time depends on the physical characteristics of the storage medium. Comp_cycle is the computation time for a single kernel execution of a single task; it is a constant in the performance model and can be approximated by running benchmark packages. M_handel is the number of GPU kernel memory accesses that the LLC can handle simultaneously, calculated based on the LLC's resource configuration. n is the number of tasks the kernel needs to execute, specified by the program.

[0074] Different parameter values ​​were obtained based on different CPU, GPU, and LLC chip configurations. Linear least squares was used to input these parameter values ​​into the GPU performance model for polynomial fitting. The optimization objective and constraints are both decision variables h. ALU For linear polynomials, use the branch and bound method for all feasible h ALU Perform a search to obtain the optimal computing unit configuration with the minimum runtime. ALU Including the arithmetic unit type h type and quantity h num Specifically, it includes the following sub-steps:

[0075] (3.2.1) The search tree is initialized to an empty set, and the minimum running time T is... * Initialize to +∞. The search tree has 2 levels, and the first-level nodes store h. type The second-level node stores h num .

[0076] (3.2.2) Generate a tree node Node, and select an h type As an attribute of this node, treat the Node as a branch and fix h. type Calculate the lower bound of Node. If the lower bound is greater than T... * If so, then delete this branch.

[0077] (3.2.3) If the lower limit is less than or equal to T * A second-level node is generated based on the Node, storing different h values. num Traverse the second-level tree nodes in breadth-first search order, and store the h values ​​of the parent nodes. type and the h stored in the current node num Substitute the values ​​into the performance formula and solve. If the calculated application runtime is less than T... * Update T * And store the {h} contained in the current node type h num} is the optimal solution.

[0078] (3.2.4) Repeat step (3.2.2) until step (3.2.3) has been traversed, then end.

[0079] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

[0080] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A network design method for a multi-core integrated system based on network computing, characterized in that, The method includes the following steps: (1) Router microarchitecture design: Implement the hardware of the router, design the pipeline, flow control and arbitration, and add a parser, arithmetic unit and FIFO buffer queue to the router; (2) Task mapping: Before the distributed training application of the neural network is run, the corresponding system flow graph is generated, the allocation strategy is generated based on the system flow graph, and the tasks in the system flow graph are mapped to the processor resources of the multi-core integrated system according to the allocation strategy; (2.1) Designate one processor in the multi-core integrated system as the resource manager to calculate the optimal allocation strategy and allocate tasks; (2.2) Data flow diagram for task initialization based on distributed training application of neural network ;in, For graph nodes, For directed edges, The weights; the graph nodes Used to represent tasks, especially computationally dependent tasks. and , build arrive directed edges , weights for Towards Transmitted communication volume; the data flow graph There are two virtual vertices. and , This serves as the starting point and is connected to the first-level task. As the endpoint and connected to the last level of tasks; define the resource set. Sequentially traverse the processor resources contained in each chip in the multi-chip integrated system. Add to resource collection ,in , Includes information on the location of processor resources in a multi-chip integrated system; (2.3) Data flow diagram obtained in step (2.2) Traverse in breadth-first search order And add the visited nodes to the task queue in turn. The task queue The first task Assign to the resource manager; (2.4) Based on the task queue obtained in step (2.3) traverse the task queue sequentially For the current task The resource set obtained from step (2.2) Select the previous task Allocated processor resources The processor resource with the shortest distance and currently idle is selected as the task. The allocated object is marked as occupied; since multi-core integrated systems have a mesh structure within and between cores, there may be multiple processor resources that are equidistant and have the shortest distance. These equidistant processor resources are stored as a resource set. and select resource sets in sequence. Processor resources in the task The object to be allocated; (2.5) Sequential traversal of the task queue Repeat step (2.4) until the task queue is full. All tasks Allocation complete; task allocation strategy generated during traversal. ; (2.6) Allocation strategy based on step (2.5) Minimize application runtime , can be represented as This is the combination with the shortest running time among various combinations; among which , This refers to the configuration of the processing unit in the router. It can be represented as arrive The runtime includes the time a task spends executing on the processor and communicating over the network; based on the task allocation strategy. A linear regression model is established, and a prediction module is built for execution time and communication time. The prediction model is obtained by linear fitting through historical running data; traversal To obtain the minimum application runtime and the optimal allocation strategy; (3) Software and hardware co-design: Based on the task mapping scheme in step (2), construct error model and performance model, minimize the running time of the distributed training application of neural network, and obtain the optimal software and hardware configuration with the minimum running time.

2. The network design method for a multi-core integrated system based on network computing as described in claim 1, characterized in that, The router hardware implementation in step (1) specifically involves adding a parser, an arithmetic unit, and a FIFO buffer queue to the router. The parser is a comparator used to determine the network calculation flag in the packet header. The arithmetic unit includes an arithmetic logic unit, a buffer, and a register. The arithmetic logic unit consists of multiple parallel single-precision / double-precision floating-point adders used to calculate the exponent and mantissa in parallel. The buffer is used to store the input data of the arithmetic logic unit. The register is used to store the source address, destination address, and network calculation flag in the packet header. The FIFO buffer queue is used to store the data packets output by the arithmetic unit.

3. The network design method for a multi-core integrated system based on network computing as described in claim 2, characterized in that, The router pipeline design in step (1) is as follows: before performing route selection, a network computing pipeline is added; when the data packet enters the router, it first enters the network computing pipeline. If the data packet needs network computing, it is transmitted to the computing unit for calculation. If the data packet does not need network computing, it enters the next pipeline for route selection.

4. The network design method for a multi-core integrated system based on network computing as described in claim 3, characterized in that, The router flow control in step (1) is as follows: When a data packet is transmitted to the router, the parser first compares the network calculation flag bit in the packet header of the input port with the preset network calculation flag bit. If they are equal, the entire data packet is split and the packet header is stored in the register of the arithmetic unit, and the data part is stored in the buffer inside the arithmetic unit. After the data part enters the buffer, a judgment is triggered. When the amount of data in the buffer is greater than 1, the data in the buffer is transmitted to the arithmetic logic unit for floating-point addition. When the amount of data in the buffer is 1, the next data that needs to be calculated on the network enters the buffer.

5. The network design method for a multi-core integrated system based on network computing as described in claim 2, characterized in that, The router arbitration design in step (1) is as follows: the number of input ports of the cross switch in the router is increased from the original five ports to six ports; the arbitration part of the router is priority arbitration, which polls each input port according to the priority arbitration algorithm to determine the weight value of the port with a packet forwarding request. After the polling is completed, the port with the largest weight value is determined by the comparator as the input of the cross switch and output after arbitration.

6. The network design method for a multi-core integrated system based on network computing as described in claim 1, characterized in that, Step (3) includes the following sub-steps: (3.1) Based on the task allocation strategy obtained in step (2.6), obtain the summary gradient value of the neural network during distributed training on the multi-core integrated system, based on the current neural network model structure. It uses a combination of hardware configurations containing different types and numbers of arithmetic logic units. and accuracy loss rate Establish an error model with the quality loss of the neural network. The relationship between accuracy loss and model quality loss was linearly fitted using an error model trained offline. (3.2) Error model based on step (3.1) and the pruned neural network Build a performance model to reduce training time Minimum, and satisfying that the quality loss value is less than the threshold. And the increased area of ​​the router after adding network computing functionality. Less than 10% The area of ​​the original router ; (3.3) Select the different hardware configurations and the pruned neural network models obtained in step (3.1) and substitute them into the running time model obtained in step (3.2). Use the branch and bound method to obtain the optimal hardware configuration and the corresponding neural network model with the minimum running time.

7. The network design method for a multi-core integrated system based on network computing as described in claim 6, characterized in that, Step (3.1) includes the following sub-steps: (3.1.1) Before each round of gradient aggregation, label the gradients of the data that need to be aggregated and updated; set the initial precision reduction rate. During training, neural networks sort the data according to the magnitude of the absolute value of the gradient. A gradient of % will reduce precision; (3.1.2) Select a mini-batch of gradients from the gradients marked in step (3.1.1), sort them from smallest to largest, and find the sorted gradients. The absolute value corresponding to the gradient of %, and set this absolute value as the threshold for the whole; (3.1.3) The absolute value of the residual gradient is compared with the threshold obtained in step (3.1.2). When the absolute value of the gradient is less than the threshold, the precision of the gradient will be reduced. (3.1.4) The difference between the training result obtained after the same number of iterations and the original precision training result is used as the quality loss value; after each iteration, the corresponding quality loss will be obtained. The error will gradually increase until the quality loss exceeds the acceptable range, at which point the current model will be selected as the optimal software model within the acceptable error range.

8. The network design method for a multi-core integrated system based on network computing as described in claim 1, characterized in that, The multi-core integrated system is a hybrid system that packages a CPU, GPU, and LLC as cores; the LLC is used for information transmission between the CPU and GPU; when the CPU writes data, it updates the data on the LLC; when the GPU accesses data, it directly accesses the data previously written by the CPU.

9. The network design method for a multi-core integrated system based on network computing as described in claim 1, characterized in that, The neural network distributed training application is used to implement distributed computing using multiple machines when training neural networks on large datasets, and to split the model and training dataset in a model parallel or data parallel manner.

Images