A data processing device and a data processing system

Abstract

The application discloses a data processing device and a data processing system in the computer technical field. In the application, the data processing device including a memory, an operator, a detector, a trigger and a dispatcher can detect whether the corresponding layer in the model is frozen or not at the hardware level and complete the freezing mark. After receiving the freezing signal, the operator can skip the operation process corresponding to the target layer which is frozen and no longer execute the calculation task related to the layer. The dispatcher synchronously identifies various hardware resources occupied by the target layer according to the freezing signal, judges the hardware resources as the resources which can be rescheduled, thereby realizing the release and redistribution of the hardware resources related to the frozen layer, reducing the invalid occupation of the hardware resources, improving the hardware resource utilization and the running performance of the data processing device when executing the model training task.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing device and a data processing system. Background Technology

[0002] Currently, in model training processes, the weights of some network layers are often fixed at the algorithm level to constrain the corresponding layer parameters from being updated. When using hardware devices for model training, even if some network layer weights are frozen at the algorithm level, the hardware device will still allocate relevant hardware resources for those frozen network layers, resulting in unnecessary occupation of hardware resources.

[0003] Therefore, how to reduce the unnecessary occupation of hardware resources by the frozen network layers in the model is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a data processing device and a data processing system to reduce the unnecessary occupation of hardware resources by the frozen network layer in the model.

[0005] In a first aspect, this application provides a data processing device, including a memory, an arithmetic unit, a detector, a trigger, and a tuner; The memory is used to store the model input data and the weight parameters of each layer in the target model; it is also used to store the updated weight parameters of each layer in the target model obtained from model training and intermediate training results; The arithmetic unit is used to train the target model using the model input data in the memory, and to transfer the updated weight parameters and intermediate training results to the memory; it is also used to skip the target layer's target operations during the model training process based on the frozen signal of the target layer in the target model. The detector is used to confirm, using the updated weight parameters in memory, that the current weight parameters of the target layer meet the convergence condition, and then outputs the target layer's freeze signal to the trigger. The trigger is used to mark the target layer as frozen based on the signal to be frozen, and outputs the frozen signal to the arithmetic unit and the tuner; The allocator is used to determine, based on the frozen signals, the corresponding hardware resources occupied by the target layer in memory, arithmetic unit, and detector as allocable resources.

[0006] Secondly, this application provides a data processing system, including a load balancing device, an interconnection device connected to the load balancing device, and a plurality of data processing devices connected to the interconnection device; Interconnect devices are used to collect resource usage information from various data processing devices; Load balancing devices are used to allocate tasks to each data processing device according to resource usage information and in a preset load balancing manner.

[0007] Thirdly, this application provides a data processing system, including a switching device and multiple data processing devices; each data processing device is connected to the switching device via a cache coherence protocol.

[0008] As can be seen, the beneficial effects of this application are as follows: the data processing device, including a memory, arithmetic unit, detector, trigger, and allocator, can detect whether a corresponding layer in the model is frozen at the hardware level and complete the freeze marking. After receiving the frozen signal, the arithmetic unit can skip the computation process corresponding to the frozen target layer and no longer execute the related computation tasks of that layer. The allocator synchronously identifies various hardware resources occupied by the target layer based on the freeze signal, and determines that part of the hardware resources are reschedulable, thereby realizing the release and reallocation of the hardware resources related to the frozen layer. In other words, this scheme can avoid the data processing device continuously reserving computation and storage-related hardware resources for the frozen layer in the target model, thereby reducing the invalid occupation of hardware resources, scheduling the released hardware resources to the training tasks of other network layers of the model, improving the utilization rate of hardware resources and the running performance of the data processing device when performing model training tasks.

[0009] Correspondingly, the data processing system provided in this application also has the above-mentioned technical effects. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the structure of a data processing device disclosed in this application; Figure 2 This is a schematic diagram of a single FPGA accelerator card structure disclosed in this application; Figure 3 This is a schematic diagram of an on-chip high-speed memory layer structure disclosed in this application; Figure 4 This is a schematic diagram of a hardware detection module for layer convergence state disclosed in this application. Figure 5 This is a schematic diagram of an automatic freeze hardware trigger module disclosed in this application; Figure 6 This is a schematic diagram of the structure of an on-chip resource dynamic allocation module disclosed in this application; Figure 7This is a schematic diagram of a customized on-chip pipeline parallel module structure disclosed in this application; Figure 8 This is a schematic diagram illustrating the process of model training using a single FPGA accelerator card disclosed in this application. Figure 9 This is a schematic diagram of an FPGA cluster distributed system disclosed in this application; Figure 10 This is a schematic diagram of a distributed system based on the CXL protocol disclosed in this application; Figure 11 A server architecture diagram provided for this application; Figure 12 A terminal structure diagram provided for this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0013] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] Currently, model training often involves fixing the weights of some network layers at the algorithm level to prevent the corresponding layer parameters from being updated. However, when using hardware for model training, even if the algorithm has frozen some network layer weights, the hardware still allocates resources to those frozen layers, resulting in unnecessary hardware resource consumption. Therefore, this application provides a data processing device that can reduce the unnecessary hardware resource consumption caused by frozen network layers in the model.

[0016] See Figure 1 As shown in the figure, this application discloses a data processing device, including a memory, an arithmetic unit, a detector, a trigger, and a tuner.

[0017] The memory is used to store the model input data and the weight parameters of each layer in the target model; it is also used to store the updated weight parameters of each layer in the target model obtained from model training and intermediate training results.

[0018] The arithmetic unit is used to train the target model using the model input data in the memory, and to transfer the updated weight parameters and intermediate training results to the memory; it is also used to skip the target layer and perform target operations during the model training process based on the frozen signal of the target layer in the target model.

[0019] The detector is used to confirm, using the updated weight parameters in memory, that the current weight parameters of the target layer meet the convergence condition, and then outputs the target layer's freeze signal to the trigger.

[0020] The trigger is used to mark the target layer as frozen based on the signal to be frozen, and outputs the frozen signal to the arithmetic unit and the tuner.

[0021] The allocator is used to determine, based on the frozen signals, the corresponding hardware resources occupied by the target layer in memory, arithmetic unit, and detector as allocable resources.

[0022] The data processing device provided in this embodiment can achieve the following at the hardware level: automatically identify the target layer whose weights have reached convergence and complete the freezing mark during model training; control the arithmetic unit to skip the computational tasks corresponding to the frozen layer; and release the computational resources, storage resources, and detection-related hardware resources occupied by the frozen layer. The released hardware resources can be used by other unfrozen network layers, thereby avoiding the continuous invalid allocation of hardware resources to the frozen network layer, reducing unnecessary consumption of hardware computing power, and improving the overall utilization efficiency of hardware resources during model training.

[0023] In one implementation, the memory includes a weight area, an intermediate result area, a training data area, and a status register area. The weight area stores weight parameters and updated weight parameters; the intermediate result area stores intermediate training results; the training data area stores model input data; and the status register area records whether each layer in the target model has been frozen. This partitioning of the memory enables the categorized management of different types of data, facilitating rapid location and retrieval of required data by other modules within the device. Simultaneously, centralized storage of the frozen state information of each network layer reduces the overhead of status information queries.

[0024] The memory also includes a ping-pong switching control unit, which controls the ping-pong buffers in the weight area, intermediate result area, training data area, and status register area. Specifically, each of these areas has two sub-regions for implementing ping-pong buffering. For example, the weight area includes a first weight sub-region and a second weight sub-region, which are used for ping-pong buffering; the intermediate result area includes a first intermediate result sub-region and a second intermediate result sub-region, which are used for ping-pong buffering; the training data area includes a first training data sub-region and a second training data sub-region, which are used for ping-pong buffering; and the status register area includes a first status register sub-region and a second status register sub-region, which are used for ping-pong buffering. Ping-pong buffering enables parallel data reading and writing, alleviating data access latency during continuous model training and reducing idle time caused by waiting for data input.

[0025] In one embodiment, the arithmetic unit includes a layer state determination unit, a data reading unit, a ping-pong buffer unit, a forward computation unit, a backpropagation unit, a weight update unit, a data pass-through unit, and a pipeline output unit. The layer state determination unit is used to determine whether each layer in the target model has been frozen. The data reading unit is used to read the data required for model training from the memory. The ping-pong buffer unit is used to perform ping-pong buffering on the computation process data of each layer in the target model. The forward computation unit is used to execute the forward computation process of each layer in the target model to obtain intermediate training results. The backpropagation unit is used to execute the backpropagation process of each layer in the target model. The weight update unit is used to update the weight parameters of each layer in the target model to obtain updated weight parameters. The data pass-through unit is used to skip the target computation performed by the target layer during model training based on the frozen signal. The pipeline output unit is used to transmit the updated weight parameters and intermediate training results to the memory. Thus, the arithmetic unit adopts a multi-unit modular design structure, which can decouple different computation logics during model training, independently control the computation jump process corresponding to the frozen layer, facilitate the rapid switching of computation paths in the frozen state, and ensure the continuous and stable operation of the training pipeline.

[0026] In one embodiment, the detector includes a weight reading unit, a multiplier array, an adder array, a similarity comparison unit, and a threshold comparison unit. The weight reading unit reads updated weight parameters and reference weight parameters for each layer in the target model from memory. The multiplier array performs element-wise multiplication on the updated weight parameters to obtain a first result; it also performs element-wise multiplication on the reference weight parameters to obtain a second result. The adder array accumulates the first result to obtain a first accumulated result; it then accumulates the second result to obtain a second accumulated result. The similarity comparison unit compares the first and second accumulated results to obtain a weight similarity. The threshold comparison unit determines whether the current weight parameters of each layer in the target model meet the convergence condition based on the comparison result of the weight similarity and a preset threshold. When the current weight parameters of the target layer meet the convergence condition, a freeze signal is output to the trigger. Thus, by utilizing the array operation structure, the detector can perform weight similarity calculations in parallel, quickly identify whether a network layer has reached convergence, and promptly initiate the freeze-related process, shortening the time required for convergence state identification.

[0027] In one implementation, the trigger includes a trigger signal receiving unit, a freeze state register, and an incremental bitstream reloading unit. The trigger signal receiving unit receives a signal to be frozen. The freeze state register sets the state flag bit of the target layer according to the signal to be frozen, marking the target layer as frozen. The incremental bitstream reloading unit disables the target operation of the target layer. This structure can quickly write the network layer freeze flag and simultaneously cut off the corresponding operation path of the frozen layer, enabling a rapid conversion from freeze signal to operation control. More specifically, the incremental bitstream reloading unit can address and read the frozen incremental configuration bitstream segment corresponding to the target layer from the configuration bitstream storage area. Through the configuration access port, it performs partial rewriting of the operation mode and output enable configuration bits, configuration lookup table and clock enable configuration bits, and memory read / write enable configuration bits of the forward computation, backward propagation, and weight update units corresponding to that layer in the operation module, thereby shielding the hardware computation and storage logic of that layer.

[0028] In one implementation, the detector is further configured to output a thawing signal to a trigger if the target layer needs to be thawed; the trigger is configured to mark the target layer as thawed based on the thawing signal and output a thawed signal to the arithmetic unit and the tuner; the tuner is configured to allocate corresponding hardware resources in the memory, arithmetic unit, and detector to the target layer based on the thawed signal; the arithmetic unit is configured to perform target operations on the target layer without skipping them during model training based on the thawed signal. This configuration supports dynamic switching of network layers between frozen and thawed states, adapting to application scenarios where model training is performed in stages. Correspondingly, the unfreeze signal is input to the trigger from the external training control terminal through the configuration interface. Then, the frozen state register in the trigger performs a reset operation on the state flag bit of the target layer, marking it as an unfrozen active state. At the same time, the incremental bit stream reloading unit is triggered to read the active state incremental configuration bit stream corresponding to the target layer, and performs reverse rewriting on the operation mode and output enable configuration bit, configuration lookup table and clock enable configuration bit, and memory read / write enable configuration bit of the forward calculation, backward propagation, and weight update units corresponding to this layer in the operation module, so as to restore the hardware calculation and storage logic functions required by the target layer.

[0029] In one implementation, the allocator includes a state identification unit, a computing resource allocation unit, a configuration resource allocation unit, and a storage resource allocation unit. The state identification unit confirms that the target layer is in a frozen state based on a frozen signal and sends disable commands to the computing resource (e.g., DSP) allocation unit, the configuration resource (e.g., CLB) allocation unit, and the storage resource (e.g., BRAM) allocation unit, respectively. The computing resource allocation unit, based on the received disable commands, causes the target layer to release the computing resource blocks (DSP blocks) occupied in the computing unit. The configuration resource allocation unit, based on the received disable commands, causes the target layer to release the configuration blocks (CLB resources) occupied in the detector. The storage resource allocation unit, based on the received disable commands, causes the target layer to release the cache blocks (BRAM blocks) occupied in the memory. By setting up independent allocation units according to resource categories, the release process of various hardware resources can be classified and managed, achieving coordinated reclamation of computing resources, detection and configuration resources, and storage resources. More specifically, the state recognition unit can read the layer status flag bit of the frozen state register in the trigger in real time. When the target layer is identified as being in a frozen state, it sends a disable command to the three types of resource allocation units: DSP, CLB, and BRAM. The DSP resource allocation unit writes the clock enable register and output enable register disable values ​​to all DSP blocks occupied by the frozen layer in the arithmetic unit module, turns off the working clock, and sets the output port as a bypass. The CLB resource allocation unit writes the global clock enable configuration register disable value to the CLB resources occupied by the frozen layer in the arithmetic unit module and the layer convergence state hardware detection module, and rewrites the lookup table to configure the pass-through state, stopping the logic flip. The BRAM resource allocation unit writes the read enable register and write enable register invalid values ​​to the BRAM blocks occupied by the dedicated cache partition of the frozen layer in the memory, closes the read and write access paths, and thus releases the hardware resources occupied by the frozen layer.

[0030] In one implementation, the allocator is also used to allocate allocable resources to non-frozen layers in the target model, excluding the target layer. This processing method can reinvest the hardware resources released from the frozen layers into effective model training tasks, fully utilizing the value of idle hardware resources. More specifically, the three types of resource allocation units—DSP, CLB, and BRAM—connect idle DSP, CLB, and BRAM resources to the operational pipeline, control logic, and cache partitions of the active layers, expanding the operational parallelism and storage capacity of the active layers. When a defreezing signal is detected for any layer, the three types of resource allocation units first reclaim the idle resources allocated to other active layers, and then sequentially restore the enable register and configuration register values ​​of the hardware resources corresponding to the original frozen layer, opening up the complete hardware path of that layer and restoring its normal function.

[0031] As can be seen, the data processing device provided in this embodiment, including a memory, an arithmetic unit, a detector, a trigger, and a tuner, can detect whether a corresponding layer in the model is frozen at the hardware level and complete the freeze marking. After receiving the frozen signal, the arithmetic unit can skip the computation process corresponding to the frozen target layer and no longer execute the related computation tasks of that layer. The tuner synchronously identifies various hardware resources occupied by the target layer based on the freeze signal, determines that the hardware resources are reschedulable, and thus realizes the release and reallocation of the hardware resources related to the frozen layer. In other words, this scheme can avoid the data processing device continuously reserving computation and storage-related hardware resources for the frozen layer in the target model, thereby reducing the invalid occupation of hardware resources. The released hardware resources can be scheduled to the training tasks of other network layers in the model, which can improve the utilization rate of hardware resources and the running performance of the data processing device when performing model training tasks.

[0032] The following describes the process of implementing a data processing device based on a Field Programmable Gate Array (FPGA) accelerator card.

[0033] Please see Figure 2 A single FPGA accelerator card (i.e., a single data processing device) includes: an on-chip high-speed storage layer (i.e., memory), a layer convergence state hardware detection module (i.e., detector), an automatic freeze hardware trigger module (i.e., trigger), an on-chip resource dynamic allocation module (i.e., allocator), and a customized on-chip pipelined parallel module (i.e., arithmetic unit). Each module is implemented using the FPGA's underlying hardware resources. Modules interact via an Advanced eXtensible Interface 4 (AXI4) high-speed bus. External data storage can be achieved by connecting to Double Data Rate Synchronous Dynamic Random Access Memory (DDR). A configuration interface is used for parameter configuration before training.

[0034] Please see Figure 3The on-chip high-speed storage layer consists of multiple BRAM (Block Random Access Memory) blocks arranged in a ping-pong cache configuration, divided into four areas: a weight cache, an intermediate result cache, a training data cache, and a status register area. It interacts with other hardware modules via the AXI4 high-speed bus. Its core functions include caching the weights of each layer of the large model, intermediate training results, and training data imported from external DDR4 memory. Simultaneously, the status register area stores the convergence and freeze states of each layer, providing real-time status data to other modules. The high-speed nature of the on-chip BRAM ensures nanosecond-level data access latency, meeting the real-time requirements of hardware-level acceleration and thus improving the efficiency of data retrieval during model training.

[0035] Please see Figure 4 The layer convergence state hardware detection module is based on the Configurable Logic Block (CLB) of the FPGA and builds a combinational logic operation pipeline. The core operation unit adopts the FPGA's Digital Signal Processor (DSP). This module includes: a weight reading unit, a multiplier array, an adder array, a similarity comparison unit, and a threshold comparison unit. This module can achieve hardware-level real-time weight similarity calculation without the intervention of a central processing unit (CPU) or graphics processing unit (GPU). During operation, it synchronously reads the current weights and reference convergence weights of each layer of the large model from the on-chip high-speed storage layer. After being transmitted by the weight reading unit, the multiplier array and adder array built by the DSP complete the element-wise parallel multiplication and accumulation of the weights. Then, the similarity comparison unit normalizes the calculation results to obtain the weight similarity. Finally, the threshold comparison unit compares the similarity with the convergence threshold preset by the configuration interface in real time. When the similarity reaches the threshold, it immediately outputs a convergence trigger signal to the automatic freeze hardware trigger module, which can realize the rapid detection of the convergence status and shorten the response time of the freeze trigger.

[0036] Please see Figure 5The automatic freeze hardware trigger module is based on the FPGA's CLB to construct timing logic control circuits, including three core units: a trigger signal receiving unit, a freeze status register, and an incremental bitstream reloading unit. It can achieve nanosecond-level freeze / unfreeze trigger control for large model layers. During operation, the module first acquires the convergence trigger signal output by the layer convergence status hardware detection module through the trigger signal receiving unit. Upon receiving the signal, it immediately sets the freeze status register of the corresponding network layer to the freeze state. Simultaneously, it calls the incremental bitstream reloading unit to perform local reloading only on the FPGA hardware configuration bitstream corresponding to that layer, without requiring a complete FPGA reconstruction. This shields the computational and storage hardware logic of that layer, achieving hardware-level freeze. If the frozen layer needs to be unfrozen during training, the module receives the unfreeze signal from the external training control terminal and, similarly, reloads the hardware configuration bitstream of that layer through the incremental bitstream reloading unit, quickly restoring the corresponding layer's hardware logic function. This enables rapid switching between freeze and unfreeze, improving the flexibility of the training process.

[0037] Please see Figure 6 The on-chip resource dynamic allocation module uses the FPGA's CLB to build hardware resource enable / disable control circuits, including four core units: a status recognition unit, a DSP resource allocation unit, a CLB resource allocation unit, and a BRAM resource allocation unit. The core functionality involves the complete release of frozen layer hardware resources and the adaptive dynamic allocation of active layer resources. During operation, the status recognition unit captures the layer freeze / unfreeze status signals output by the automatic freeze hardware trigger module in real time. When a target layer is identified as frozen, the corresponding DSP, CLB, and BRAM resource allocation units perform hardware-level disable operations on all on-chip DSP, CLB, and BRAM resources occupied by the frozen layer, completely releasing that portion of computing and storage resources. Simultaneously, the released idle hardware resources are adaptively allocated to the active layers of large, unconverged models according to the real-time computing needs of the active layers, maximizing the parallel computing capabilities of the active layers. If a unfreeze signal is detected, the corresponding layer's resources are quickly enabled and the allocated idle resources are reclaimed, thereby improving hardware resource utilization efficiency and reducing resource idleness and waste.

[0038] Please see Figure 7The customized on-chip pipelined parallel module is built on the FPGA's CLB and DSP to construct an inter-layer hardware pipeline adapted to the freeze mechanism. It includes: a layer state determination unit, a data reading unit, a ping-pong buffer unit, a forward computation unit, a backward propagation unit, a weight update unit, a data pass-through unit, and a pipeline output unit. High-speed data transmission between layers is achieved through pipeline registers. The core of the module implements hardware-level pipelined parallel training and can accurately adapt to the freeze mechanism. During operation, the layer state judgment unit first reads the layer freeze state signal of the on-chip high-speed storage layer in real time, classifies the network layer state, and thus realizes differentiated processing of network layers in different states, improving pipeline operation efficiency. For active layers, the data reading unit reads training data and active layer weights from the on-chip high-speed storage layer. After data buffering by the ping-pong caching unit, the entire training calculation process is completed sequentially through the forward calculation unit, backpropagation unit, and weight update unit. Then, the pipeline output unit writes the updated weights and intermediate results back to the on-chip high-speed storage layer in real time. For frozen network layers, the layer state judgment unit triggers the data pass-through unit, directly skipping all operations such as data reading, ping-pong caching, forward calculation, backpropagation, and weight update for that layer. The data stream corresponding to the frozen layer is directly transmitted to the pipeline output unit through the data pass-through unit without performing any redundant calculation and data processing steps, which greatly improves the overall training throughput.

[0039] Please see Figure 8The specific steps for model training using the single FPGA accelerator card are as follows: (1) Pre-training configuration: Through the FPGA configuration interface, write the weight convergence threshold to the layer convergence state hardware detection module, write the initial state of each layer of the large model to the state register area of ​​the on-chip high-speed storage layer, and at the same time complete the initial allocation ratio configuration of on-chip DSP, CLB, and BRAM resources to prepare for the subsequent training process. (2) Data / weight loading: Load the reference convergence weights and training data of the large model from the off-chip DDR to the corresponding cache area of ​​the on-chip high-speed storage layer to complete the pre-training data preparation and provide stable data support for the hardware module. (3) Hardware-level convergence detection: The layer convergence state hardware detection module reads the current weights and reference convergence weights of each layer of the large model in the on-chip high-speed storage layer in real time through the hardware pipeline, calculates the weight similarity, and judges whether the similarity reaches the convergence threshold. If it does not reach the threshold, execute step (4); if it does reach the threshold, execute step (5) to realize real-time monitoring of the convergence state. (4) Active layer hardware pipeline training: The customized on-chip pipeline parallel module reads the training data and active layer weights from the on-chip high-speed storage layer, completes the forward calculation, back propagation and weight update of the active layer through the hardware pipeline, and writes the updated weights back to the on-chip high-speed storage layer in real time. If the training is not completed, return to step (3) to continue the convergence detection. If the training is completed, execute step (8). If the frozen layer needs to be unfrozen, execute step (9) to ensure the continuity of active layer training. (5) Nanosecond-level freeze trigger: The automatic freeze hardware trigger module receives the trigger signal from the convergence detection module, sets the freeze state register of the corresponding layer to the frozen state, and at the same time shields the hardware logic of the layer through incremental bit stream reloading technology to achieve fast freezing. (6) Complete release of frozen layer hardware resources: After the on-chip resource dynamic allocation module recognizes the frozen state signal, it disables the DSP, CLB and BRAM resources corresponding to the frozen layer at the hardware level, completely releases the computing power and storage resources, and reduces resource idleness. (7) Resource adaptive allocation to active layer: The hardware resources released in step 6 are adaptively allocated to the non-converged active layer to improve the parallel computing capability of the active layer. Return to step (4) to continue training and further improve the efficiency of hardware resource utilization. (8) Output training results: Output the final weights and results of the large model training in the on-chip high-speed storage layer to complete the training. (9) Hardware-level unfreezing trigger: The automatic freezing hardware trigger module receives the external unfreezing signal, sets the frozen state register of the corresponding layer to the unfrozen state, and restores the hardware logic of the layer through incremental bitstream reloading technology to achieve fast unfreezing. (10) Restore frozen layer hardware resources: After the on-chip resource dynamic allocation module recognizes the unfreezing state signal, it restores the DSP, CLB, and BRAM resources corresponding to the layer and returns to step (3) to continue convergence detection to ensure that the unfrozen layer can participate normally in subsequent training.

[0040] Furthermore, based on the aforementioned single FPGA accelerator card, a cluster-level hardware module is added and the original pipeline module is optimized to build an FPGA cluster distributed system, realizing hardware-level load balancing and task allocation across FPGAs. At the same time, through the optimization and upgrading of the pipeline module, distributed parallel computing of active layer tasks is realized, maintaining the design principle of hardware-level execution without software intervention throughout the process, without changing the underlying freezing acceleration logic of the single FPGA node, thereby improving the overall efficiency of distributed training.

[0041] Please see Figure 9 This FPGA cluster distributed system includes several single FPGA accelerator card nodes (i.e., data processing devices), a cross-FPGA cluster hardware interconnect module (i.e., interconnect device), a cluster hardware load balancing unit (i.e., load balancing device), and a cluster-shared off-chip DDR memory array. Each FPGA node can be implemented as described in the aforementioned embodiment, and the customized on-chip pipeline parallel modules of each node have been optimized and upgraded, with the addition of a cross-node data synchronization unit. The cross-FPGA cluster hardware interconnect module is the core communication and status transmission carrier for each node in the cluster, connecting one end to the optimized pipeline modules of all FPGA nodes and the other end to the cluster hardware load balancing unit. The cluster hardware load balancing unit is the core of task allocation for the cluster, establishing bidirectional data interaction with the optimized pipeline modules of all FPGA nodes. The cluster-shared off-chip DDR memory array provides unified external data storage and loading support for all FPGA nodes, enabling cluster sharing of training data and weight data.

[0042] It should be noted that the cross-FPGA cluster hardware interconnect module consists of a high-speed serial interface of the FPGA and a hardware module implemented by a CLB. Its core function is to achieve high-speed hardware-level data synchronization between FPGA nodes, with communication latency far lower than that of software network protocols. Simultaneously, it can collect real-time status information of each FPGA node, such as the number of frozen layers, the number of active layers, and remaining hardware resources, and transmit all collected node status data to the cluster hardware load balancing unit, providing data support for cluster task allocation. The cluster hardware load balancing unit consists of hardware task allocation logic implemented by the FPGA's CLB. Its core function is to achieve hardware-level active layer task allocation based on the resource status data of each FPGA node transmitted by the cross-FPGA cluster hardware interconnect module. It evenly distributes the active layer tasks of the large model to FPGA nodes with sufficient resources according to the remaining resources of each node, eliminating the communication and waiting overhead caused by software scheduling, and achieving precise load balancing of cluster computing power. The customized on-chip pipelined parallel module is optimized by adding a cross-node data synchronization unit composed of FPGA BRAM ping-pong cache and high-speed serial interface on the basis of the original module hardware composition and core functions in Example 1. The core improvement is to realize hardware-level high-speed synchronization of intermediate calculation results of active layers between FPGA nodes, support distributed pipelined parallel computing of active layer tasks of large models, and effectively improve the parallelism and overall speedup ratio of large model training.

[0043] Accordingly, the cross-FPGA cluster hardware interconnect module collects real-time status information of each FPGA node, including the number of frozen layers, released hardware resources, and remaining computing power. This collected cluster-wide node status data is then transmitted in real-time and at high speed to the cluster hardware load balancing unit. Based on the received resource status data of each FPGA node, the cluster hardware load balancing unit allocates the active layer tasks of the large model to each FPGA node according to the pre-defined load balancing rules, ensuring that nodes with sufficient resources undertake a corresponding proportion of the training tasks. Each FPGA node's customized on-chip pipelined parallel module establishes hardware-level data interaction with other FPGA nodes through its newly added cross-node data synchronization unit, achieving high-speed, real-time synchronization of intermediate computation results during active layer training and completing the distributed pipelined parallel training of the large model's active layer tasks.

[0044] Furthermore, based on the aforementioned single FPGA accelerator card, a CXL switching matrix is ​​introduced to construct an FPGA heterogeneous computing system. A new CXL controller, CXL cache consistency unit, and CXL cluster resource pooling unit are added, thereby realizing the collaborative functions of hardware-level freezing, cross-node resource pooling, and CXL high-speed consistency interconnection. This can solve the problems of insufficient FPGA off-chip storage bandwidth, cross-node cache inconsistency, local allocation of frozen layer resources, and low efficiency of heterogeneous computing interconnection.

[0045] This approach retains the aforementioned hardware-level load balancing and cross-node data synchronization logic, requiring only protocol replacement, module addition, and hardware upgrades to achieve a smooth upgrade from FPGA clusters to CXL heterogeneous clusters. Specifically, the high-speed serial interface is replaced with the CXL protocol to achieve higher bandwidth and lower latency interconnection; a CXL switching matrix, centered on CXL switches, is introduced as the high-speed interconnect backbone for the entire heterogeneous cluster; each FPGA node adds a CXL controller, a CXL cache coherency unit, and a CXL interconnect module, while a CXL cluster resource pooling unit is added to the CXL switching matrix; the aforementioned cluster-shared off-chip DDR storage array is replaced with a CXL memory pool to provide high-speed off-chip storage support; and high-speed interconnection support for heterogeneous computing devices such as CPUs / GPUs is added to enable collaborative computing between FPGAs and heterogeneous devices.

[0046] Please see Figure 10A distributed system based on the CXL protocol can include: several upgraded FPGA nodes (data processing devices), CXL switches (switching devices), a CXL memory pool, and CPU / GPU heterogeneous computing devices. The CXL switch includes a CXL switching matrix and a CXL cluster resource pooling unit. Multiple FPGA nodes constitute the core execution entity of the cluster, responsible for core operations such as training computation of large model active layers and hardware-level freezing. The CXL switching matrix, with the CXL switch as its core hardware and combined with FPGA-implemented cluster scheduling logic, is the core of data routing and scheduling for the entire heterogeneous cluster. One end establishes a high-speed direct connection with all FPGA nodes through the CXL interconnect modules within each FPGA node; the other end establishes bidirectional connections with the CXL cluster resource pooling unit, the CXL memory pool, and the CPU / GPU heterogeneous computing devices, responsible for data forwarding and task scheduling between all components, achieving high-speed interconnection across the entire cluster. The CXL cluster resource pooling unit, implemented by the FPGA's CLB, is deployed within the CXL switching matrix and is not responsible for storage, only for resource scheduling. The core of establishing bidirectional data interaction with all FPGA nodes is to collect and integrate the hardware resources released from the frozen layers of each FPGA node, construct a cluster-shared resource pool, and complete cross-node and cross-device resource allocation. The CXL memory pool, composed of high-speed storage devices, is independent of the FPGA nodes and the CXL exchange matrix. It connects to all FPGA nodes and heterogeneous computing devices through the CXL exchange matrix, responsible for storing large model weights, training data, and intermediate computation results, providing high-speed off-chip storage support for the entire cluster and solving the problem of insufficient on-chip storage bandwidth in FPGAs. CPU / GPU heterogeneous computing devices establish high-speed interconnects with the FPGA cluster through the CXL exchange matrix, achieving computational collaboration with FPGA nodes. The CXL exchange matrix allocates adaptation tasks in large model training, collaborating with FPGA nodes to complete training computations and improve overall computational efficiency. The connection logic of each component is as follows: FPGA nodes connect to the CXL exchange matrix through internal CXL interconnect modules. The CXL exchange matrix connects to the CXL cluster resource pooling unit, the CXL memory pool, and heterogeneous computing devices, respectively. All connections achieve high-speed communication through the CXL protocol.

[0047] More specifically, the newly added CXL module and CXL controller within the FPGA node consist of CXL protocol parsing logic implemented in the FPGA's CLB and an AXI4-CXL bridging unit, compatible with CXL 2.0 and CXL 3.0 protocols. Its core function is to achieve bidirectional bridging between the FPGA's on-chip AXI4 bus and the CXL protocol, supporting CXL.mem, CXL.cache, and CXL.io sub-protocols. This allows the FPGA node to directly access the CXL memory pool via the CXL protocol, while also enabling high-speed interconnection between the FPGA node and heterogeneous computing devices such as CPUs and GPUs via the CXL protocol. It is the core of the FPGA node's CXL protocol adaptation.

[0048] The CXL cache consistency unit consists of a consistency verification pipeline implemented by the FPGA's DSP and a cache directory register implemented by BRAM. Its core function is to verify the data consistency between the FPGA's on-chip BRAM cache and the CXL memory pool, the caches of other FPGA nodes, and the caches of heterogeneous computing devices in real time. This verification is completed in nanoseconds without software intervention, ensuring the accuracy of cross-node and cross-device data access and solving the problem of cross-node cache inconsistency.

[0049] The CXL interconnect module consists of CXL cluster communication logic implemented in the FPGA's CLB and a high-speed serial transceiver. Its core function is to serve as a dedicated communication interface between FPGA nodes and the CXL exchange matrix, enabling high-speed direct communication between FPGA nodes and the CXL exchange matrix. It supports direct transmission of weights and intermediate calculation results between nodes and between nodes and heterogeneous devices, eliminating the need for host forwarding, significantly reducing communication latency within the cluster, and replacing the aforementioned cross-FPGA cluster hardware interconnect module.

[0050] Secondly, a new cluster-level module, the CXL switching matrix, is added. It uses CXL switches as the core hardware and incorporates cluster scheduling logic implemented in FPGAs. Its core function is to build a high-speed interconnect backbone for the entire heterogeneous cluster, achieving full interconnection of N FPGA nodes. Simultaneously, it enables high-speed unified interconnection between the FPGA cluster and the CXL memory pool, as well as heterogeneous CPU / GPU computing devices. It is compatible with protocol interfaces for multiple types of computing devices and is responsible for data forwarding and hardware-level task scheduling between all components. It is the core of heterogeneous cluster collaborative operation.

[0051] The CXL cluster resource pooling unit, implemented by the FPGA's CLB, is deployed within the CXL switching matrix. Its core function is to collect real-time information on the DSP, CLB, and BRAM hardware resources released from the frozen layer of each FPGA node via the CXL protocol, integrate the idle resources of all nodes to construct a shared cluster resource pool, and update the resource pool status in real time. Simultaneously, based on the active layer computing power requirements of each FPGA node and heterogeneous computing device, it allocates hardware resources in the shared resource pool at the hardware level via the CXL protocol, achieving cross-node and cross-device pooling and sharing of frozen layer resources, solving the problem of only local allocation of frozen layer resources. The CXL memory pool, composed of high-speed storage devices, is deployed independently. Its core function is to replace the aforementioned shared off-chip DDR storage array, providing high-speed off-chip storage services for the entire cluster. It stores large model reference convergence weights, training data, and intermediate computation results, and achieves high-speed data communication with FPGA nodes and heterogeneous computing devices via the CXL protocol, solving the problem of insufficient off-chip storage bandwidth for FPGAs.

[0052] Correspondingly, the distributed system of the CXL protocol can achieve: high-speed CXL data loading, where reference convergence weights and training data for large models start from the CXL memory pool and are loaded directly onto the on-chip high-speed storage layer of each FPGA node via the CXL exchange matrix, the CXL interconnect module inside the FPGA node, and the CXL controller inside the FPGA node, replacing some local DDR4 data access and significantly improving the bandwidth and efficiency of loading massive training data. Cross-node resource status synchronization, where the on-chip resource dynamic allocation module of each FPGA node transmits the DSP, CLB, and BRAM hardware resource information released from the frozen layer to the CXL cluster resource pooling unit deployed in the CXL exchange matrix in real time via the CXL controller inside the FPGA node, the CXL interconnect module inside the FPGA node, and the CXL exchange matrix, achieving high-speed synchronization of the entire cluster resource status. Cluster resource pooling construction, where the CXL cluster resource pooling unit integrates the idle hardware resource information transmitted by all FPGA nodes to build a shared resource pool for the CXL cluster, and updates the resource information of the resource pool in real time according to the resource release and allocation status of each node, ensuring accurate synchronization of resource status. Cross-node resource hardware allocation: The CXL cluster resource pooling unit allocates hardware resources from the cluster's shared resource pool to resource-constrained FPGA nodes based on the real-time computing power requirements of each FPGA node's active layer. This is achieved through the CXL exchange matrix, the CXL interconnect module within the FPGA node, and the CXL controller within the FPGA node, enabling idle resource scheduling. CXL consistent data synchronization: Intermediate computation results from the active layer of each FPGA node are directly transmitted to other FPGA nodes and heterogeneous computing devices via the CXL controller, CXL interconnect module, and CXL exchange matrix within the FPGA node. During transmission, the CXL cache consistency unit within the FPGA node performs nanosecond-level data consistency checks to ensure the accuracy of data transmitted across nodes and devices. Frozen layer resource remote migration: Weight data from the frozen layer of each FPGA node is migrated from the local on-chip high-speed storage layer to the CXL memory pool via the CXL controller, CXL interconnect module, and CXL exchange matrix within the FPGA node. This completely releases the on-chip BRAM resources, freeing up all on-chip storage resources for active layer training. Heterogeneous device collaborative computing, with the CXL exchange matrix as the scheduling core, allocates different types of tasks in large model training to FPGA nodes and CPU / GPU heterogeneous computing devices through the CXL protocol. FPGA nodes, through their internal CXL interconnect modules and CXL controllers, achieve high-speed data interaction and computing power collaboration with heterogeneous computing devices to complete training tasks and improve overall computing efficiency.

[0053] This embodiment enables hardware-level freezing and dynamic resource allocation, significantly improving the utilization rate of single-node computing power. It also further achieves hardware-level load balancing and task allocation across FPGA clusters, eliminating software protocol communication overhead. Combined with the CXL protocol, it can also achieve high-speed remote storage access, high-speed consistent interconnection between FPGA and CPU / GPU, nanosecond-level cross-node cache consistency verification, and cross-node pooling sharing of frozen layer resources, enhancing the collaborative computing capabilities of heterogeneous clusters. Each node can be used for AI model training and inference in embedded devices such as drones, robots, and smart cars. It can also be applied to cloud service providers' elastic computing power scheduling platforms for multiple users, building a cloud-native FPGA computing power pool based on the CXL protocol. Combined with hardware-level freezing and real-time resource release mechanisms, it enables dynamic allocation and recycling of training computing power, improving the utilization rate of cloud hardware resources. It can also be extended to large-scale real-time inference tasks such as natural language processing and computer vision. By replacing the layer convergence detection logic with inference frequency detection logic, hardware freezing is performed on model layers with low inference frequency, releasing hardware resources for high-frequency inference layers.

[0054] The following describes a data processing system provided by an embodiment of this application. This data processing system described below can be referred to in conjunction with other embodiments described herein. In particular, it corresponds to the FPGA cluster distributed system in the foregoing embodiments.

[0055] This application discloses a data processing system, including a load balancing device, an interconnection device connected to the load balancing device, and a plurality of other data processing devices as described in other embodiments connected to the interconnection device.

[0056] Interconnect devices are used to collect resource usage information from various data processing devices.

[0057] Load balancing devices are used to allocate tasks to each data processing device according to resource usage information and in a preset load balancing manner.

[0058] In one embodiment, each data processing device includes a data synchronization unit; the data synchronization unit is used to realize data synchronization between the data processing devices.

[0059] For more detailed information on the working process of each device in this embodiment, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0060] As can be seen, this embodiment provides a data processing device that can avoid the data processing device continuously reserving hardware resources such as computing and storage for the frozen layer in the model, and can also realize cross-device data synchronization and parallel task computing.

[0061] The following describes another data processing system provided by an embodiment of this application. This data processing system described below can be referred to in conjunction with other embodiments described herein. In particular, it corresponds to the distributed system of the CXL protocol in the foregoing embodiments.

[0062] This application discloses a data processing system, including a switching device and several other data processing devices as described in other embodiments. Each data processing device is connected to the switching device via a cache coherence protocol. The cache coherence protocol can be CXL (Compute Express Link), a high-speed interface protocol that optimizes the interaction between computing, storage, and communication resources within a data center. CXL actually consists of three sub-protocols: CXL.io, CXL.cache, and CXL.mem. CXL.io is used for initialization, linking, device identification and enumeration, and register access, providing a non-consistent load / store interface for the device. CXL.cache is used for cache access, defining the interaction between the processor and the device, allowing connected CXL devices to use request and response methods to efficiently cache processor memory with extremely low latency. CXL.mem is used for memory access, using load and store commands to provide the processor with access to the device's attached memory, where the processor acts as the master device and the CXL device acts as the slave device, supporting both volatile and persistent memory architectures. These protocols are dynamically multiplexed together, enabling data transfer at a speed of 32GT / s through the standard PCIe 5.0 physical layer.

[0063] In one embodiment, the switching device includes a first interconnection unit, a resource pooling unit, and a controller. The first interconnection unit is used to achieve cache-consistent full interconnection of each data processing device. The resource pooling unit is used to collect resource usage information of each data processing device, determine idle resources based on the resource usage information, and construct a shared resource pool from the idle resources. The controller is used to allocate tasks to each data processing device according to the shared resource pool. The first interconnection unit is a cache-consistent switching matrix, such as a CXL switching matrix.

[0064] In one embodiment, each data processing device includes a cache consistency controller, a cache consistency management unit, and a second interconnection unit; the cache consistency controller is used to implement bidirectional bridging of the current data processing device bus and the cache consistency protocol, as well as the cache consistency access process between the current data processing device and the switching device; the cache consistency management unit is used to implement cache synchronization between each data processing device, as well as between the current data processing device and the switching device; the second interconnection unit is used to implement cache consistency interconnection between the current data processing device and the switching device.

[0065] For more detailed information on the working process of each device in this embodiment, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0066] As can be seen, this embodiment provides a data processing device that can avoid the data processing device continuously reserving hardware resources such as computing and storage for the frozen layer in the model, and can also realize cross-device data synchronization and parallel task computing through CXL.

[0067] The following describes an electronic device provided by an embodiment of this application. The electronic device described below can be referred to in conjunction with other embodiments described herein. The electronic device described in this embodiment can be the aforementioned data processing device, or any functional module within the aforementioned data processing device.

[0068] This application discloses an electronic device, including: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the methods disclosed in any of the above embodiments.

[0069] Furthermore, embodiments of this application also provide an electronic device. This electronic device can be, for example,... Figure 11 The server shown can also be as follows: Figure 12 The terminal shown. Figure 11 and Figure 12 These are all diagrams illustrating the structure of an electronic device according to an exemplary embodiment. The content in the diagrams should not be considered as any limitation on the scope of this application.

[0070] Figure 11 This is a schematic diagram of a server structure provided in an embodiment of this application. The server may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory stores a computer program, which is loaded and executed by the processor to implement the relevant steps in the data processing disclosed in any of the foregoing embodiments.

[0071] In this embodiment, the power supply is used to provide operating voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0072] In addition, the memory, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system, computer programs and data, etc., and the storage method can be temporary storage or permanent storage.

[0073] The operating system manages and controls the various hardware devices and computer programs on the server to enable the processor to perform operations and processes on the data in the memory. It can be Windows Server, Netware, Unix, Linux, etc. In addition to computer programs capable of performing the data processing methods disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The data may include application update information and application developer information.

[0074] Figure 12 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal may include, but is not limited to, a smartphone, tablet computer, laptop computer, or desktop computer.

[0075] Typically, the terminal in this embodiment includes a processor and a memory.

[0076] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor can be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and coprocessors. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which handles computational operations related to machine learning.

[0077] The memory may include one or more computer non-volatile storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory is used to store at least the following computer program, which, after being loaded and executed by the processor, is capable of implementing the relevant steps in the data processing method executed by the terminal side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory may also include operating systems and data, and the storage method may be temporary or permanent storage. The operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, application update information.

[0078] In some embodiments, the terminal may further include a display screen, an input / output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0079] Those skilled in the art will understand that Figure 12 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than illustrated.

[0080] The following describes a non-volatile storage medium provided in an embodiment of this application. The non-volatile storage medium described below can be referred to in conjunction with other embodiments described herein.

[0081] A non-volatile storage medium is provided for storing a computer program, wherein the computer program, when executed by a processor, implements the data processing method disclosed in the foregoing embodiments. The non-volatile storage medium is a computer-readable non-volatile storage medium, which, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon include an operating system, computer programs, and data, and the storage method can be temporary storage or permanent storage.

[0082] The following describes a computer program product provided by an embodiment of this application. The computer program product described below can be referred to in conjunction with other embodiments described herein.

[0083] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed data processing method.

[0084] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the steps in any of the above embodiments.

[0085] Any of the components, modules, units, devices, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0086] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0087] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of non-volatile storage medium known in the art.

[0088] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing device, characterized in that, Includes memory, arithmetic logic unit (ALU), detectors, flip-flops, and tuners; The memory is used to store model input data and weight parameters of each layer in the target model; it is also used to store the updated weight parameters of each layer in the target model obtained from model training and intermediate training results; The arithmetic unit is used to train the target model using the model input data in the memory, and to transmit the updated weight parameters and the intermediate training results to the memory. It is also used during model training to skip target operations performed by the target layer based on the frozen signal of the target layer in the target model; The detector is used to use the updated weight parameters in the memory to confirm that the current weight parameters of the target layer meet the convergence condition, and then outputs the target layer's freeze signal to the trigger. The trigger is used to mark the target layer as frozen based on the signal to be frozen, and to output the frozen signal to the arithmetic unit and the tuner; The allocator is used to determine, based on the frozen signal, the corresponding hardware resources occupied by the target layer in the memory, the arithmetic unit, and the detector as allocable resources; The detector includes a weight reading unit, a multiplier array, an adder array, a similarity comparison unit, and a threshold comparison unit. The weight reading unit is used to read the updated weight parameters and the reference weight parameters of each layer in the target model from the memory. The multiplier array is used to multiply the updated weight parameters element-wise to obtain a first result; and to multiply the reference weight parameters element-wise to obtain a second result. The adder array is used to perform an accumulation operation on the first result to obtain a first accumulation result; and to perform an accumulation operation on the second result to obtain a second accumulation result; The similarity comparison unit is used to compare the first accumulated result and the second accumulated result to obtain the weighted similarity. The threshold comparison unit is used to confirm whether the current weight parameters of each layer in the target model meet the convergence condition based on the comparison result of the weight similarity and the preset threshold; when it is confirmed that the current weight parameters of the target layer meet the convergence condition, the unit outputs the signal to be frozen to the trigger.

2. The data processing device according to claim 1, characterized in that, The memory includes a weight area, an intermediate result area, a training data area, and a status register area; The weight area is used to store the weight parameters and the updated weight parameters; The intermediate results area is used to store the intermediate training results; The training data area is used to store the model input data; The status register area is used to record whether each layer in the target model has been frozen.

3. The data processing device according to claim 2, characterized in that, The memory also includes a ping-pong switching control unit; The ping-pong switching control unit is used to control the ping-pong buffers in the weight area, the intermediate result area, the training data area, and the status register area, respectively.

4. The data processing device according to claim 1, characterized in that, The arithmetic unit includes a layer state judgment unit, a data reading unit, a ping-pong buffer unit, a forward calculation unit, a backward propagation unit, a weight update unit, a data pass-through unit, and a pipeline output unit; The layer state determination unit is used to determine whether each layer in the target model has been frozen; The data reading unit is used to read the data required for model training from the memory; The ping-pong caching unit is used to perform ping-pong caching on the computational process data of each layer in the target model. The forward computation unit is used to perform the forward computation process of each layer in the target model to obtain the intermediate training results; The backpropagation unit is used to execute the backpropagation process of each layer in the target model; The weight update unit is used to update the weight parameters of each layer in the target model to obtain the updated weight parameters. The data pass-through unit is used to skip the target operation performed by the target layer during model training based on the frozen signal; The pipeline output unit is used to transmit the updated weight parameters and the intermediate training results to the memory.

5. The data processing device according to claim 1, characterized in that, The trigger includes a trigger signal receiving unit, a freeze state register, and an incremental bit stream reloading unit; The trigger signal receiving unit is used to receive the signal to be frozen; The freeze state register is used to set the state flag bit of the target layer according to the signal to be frozen, so as to mark the target layer as frozen. The incremental bitstream reloading unit is used to disable the target operation of the target layer.

6. The data processing device according to claim 1, characterized in that, The detector is also used to output a thawing signal of the target layer to the trigger if the target layer needs to be thawed; The trigger is used to mark the target layer as thawed according to the thawing signal, and output the thawed signal to the arithmetic unit and the tuner; The allocator is used to allocate corresponding hardware resources from the memory, the arithmetic unit, and the detector to the target layer based on the thawed signal; The arithmetic unit is used to perform target operations without skipping the target layer during model training, based on the unfrozen signal.

7. The data processing device according to claim 1, characterized in that, The allocator includes a status identification unit, a computing resource allocation unit, a configuration resource allocation unit, and a storage resource allocation unit; The status identification unit is used to confirm that the target layer is in a frozen state based on the frozen signal, and to send a disable command to the computing resource allocation unit, the configuration resource allocation unit and the storage resource allocation unit respectively; The computing resource allocation unit is used to cause the target layer to release the computing resource blocks occupied in the computing unit according to the received disable instruction; The configuration resource allocation unit is used to cause the target layer to release the configuration block occupied in the detector according to the received disable instruction; The storage resource allocation unit is used to cause the target layer to release the cache block occupied in the memory according to the received disable instruction.

8. The data processing apparatus according to any one of claims 1 to 7, characterized in that, The allocator is also used to allocate the allocable resources to non-frozen layers in the target model other than the target layer.

9. A data processing system, characterized in that, It includes a load balancing device, an interconnection device connected to the load balancing device, and a plurality of data processing devices as described in any one of claims 1 to 8 connected to the interconnection device; The interconnection device is used to collect resource usage information from each data processing device; The load balancing device is used to allocate tasks to each data processing device according to the resource usage information and in a preset load balancing mode.

10. The data processing system according to claim 9, characterized in that, Each data processing device includes: a data synchronization unit; The data synchronization unit is used to achieve data synchronization between various data processing devices.

11. A data processing system, characterized in that, It includes a switching device and a plurality of data processing devices as described in any one of claims 1 to 8; each data processing device is connected to the switching device via a cache coherence protocol.

12. The data processing system according to claim 11, characterized in that, The switching device includes a first interconnection unit, a resource pooling unit, and a controller; The first interconnect unit is used to achieve full interconnection for cache consistency of each data processing device; The resource pooling unit is used to collect resource usage information of each data processing device, determine idle resources based on the resource usage information, and construct a shared resource pool from the idle resources. The controller is used to assign tasks to each data processing device according to the shared resource pool.

13. The data processing system according to claim 11, characterized in that, Each data processing device includes a cache consistency controller, a cache consistency management unit, and a second interconnection unit; The cache coherence controller is used to implement bidirectional bridging of the current data processing device bus and the cache coherence protocol, as well as the cache coherence access process between the current data processing device and the switching device; The cache consistency management unit is used to realize cache synchronization between various data processing devices, as well as between the current data processing device and the switching device; The second interconnection unit is used to implement cache-coherent interconnection between the current data processing device and the switching device.

14. The data processing system according to claim 12, characterized in that, The first interconnect unit is a cache consistency exchange matrix.

Images