Data processing system for realizing data prefetching

Abstract

The data processing system for realizing data prefetching of the present invention not only realizes the data prefetching control function and the software-hardware synchronization through the frame synchronization pulse signal and the controller, but also can be applied to single-core systems and multi-core systems, and is more flexible to use. The controller of the present invention has simple control and can be realized with a single thread.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and particularly to a data processing system for implementing data prefetching. Background Art

[0002] In recent years, the research on deep neural networks (DNNs) has achieved rapid development and has been preliminarily applied. On the other hand, spiking neural networks (SNNs) have attracted the attention of the academic and industrial communities in recent years due to their low power consumption and closer resemblance to the human brain. In the hardware acceleration of deep neural networks and spiking neural networks, software instructions are required to control the operation of hardware modules. A typical control process is that when executing an operation of a layer in an algorithm model, the software starts the hardware, the hardware begins to read data from the memory, the hardware performs the operation, and after the operation is completed, the hardware issues an interrupt related to the completion of execution. After receiving the interrupt signal, the software performs subsequent processing.

[0003] The above software and hardware synchronization method has some drawbacks: First, the hardware notifies the software running on the CPU with an interrupt signal, which limits the real-time performance. If the time required for a hardware operation is short, for example, 0.5 ms, it is difficult for the software to respond to the interrupt in a shorter time, and at this time, interrupt processing becomes the bottleneck of system performance. Second, reading data from the memory occurs after the hardware is started by the software, and the operation cannot occur before the data is read in. Reading data from an external memory may require a large delay, for example, 1000 clock cycles. During this period, the hardware cannot perform operations. This limits the performance of the hardware.

[0004] Aiming at the drawback of untimely software interrupt processing, two levels of software control are adopted in the hardware acceleration of deep neural networks. The fine-grained control software runs on a customized microcontroller, and the microcontroller is tightly coupled with the hardware operation unit to achieve the start-stop control of the hardware unit; the coarse-grained control software runs on the CPU, and the software on the CPU controls the operation of the microcontroller, and the software on the CPU responds to the microcontroller at a lower frequency. Such a solution can avoid frequent interrupt processing by the software on the CPU. This synchronization solution is currently mainly applied in single-core systems, and one microcontroller only controls the start-stop of one operation core. A spiking neural network is a typical multi-core system, and multiple operation cores need to work together, and the software can perform subsequent operations only after all the operation cores have ended. The above software and hardware synchronization solution cannot be directly applied to multi-core systems.

[0005] Aiming at the drawback that data cannot be prefetched in traditional synchronization methods, both software and hardware need to have the ability to prefetch data. A typical software instruction sequence for data prefetching is as follows: First step, control the hardware access module to read data block 1; Second step, start the hardware operation module to perform operations on data block 1, and at the same time control the hardware access module to read data block 2; Third step, start the hardware operation module to perform operations on data block 2, and at the same time control the hardware access module to read data block 3, and so on. This method can achieve prefetching of stored data and is mainly applied in single-core systems. In multi-core systems, if it is required not to perform operations or output results at some time points or to wait for the global synchronization signal to be valid before subsequent operations can be carried out, it is very difficult to achieve the above functions with the above method. Summary of the Invention

[0006] In view of the above-mentioned drawbacks of the prior art, the purpose of the present invention is to provide a data processing system for realizing data prefetching to solve the above technical problems in the prior art.

[0007] To achieve the above object and other related objects, the present invention provides a data processing system for realizing data prefetching, the system includes: a frame synchronization pulse signal generating device for generating a changing frame synchronization pulse signal; wherein, the changing state of the frame synchronization pulse signal includes: a valid pulse signal and a non-valid pulse signal; an operation array connected to the frame synchronization pulse signal generating device, including: one or more operation cores respectively synchronized with the frame synchronization pulse signal; a controller connected to the frame synchronization pulse signal generating device and the operation array, for prefetching control of frame data related to the obtained prefetch data instruction, and based on the input frame synchronization pulse signal and the obtained non-prefetch data instruction, controlling to execute the operation corresponding to the prefetch data instruction on the operation array, so as to enable each operation core to perform operations on the frame data; wherein, one or more frame data correspond to one frame of data; wherein, the frame synchronization pulse signal generating device generates a non-valid pulse signal when each operation core performs operations on one frame of data, and generates the valid pulse signal after each operation core finishes operations on one frame of data.

[0008] In an embodiment of the present invention, when the frame synchronization pulse signal generating device receives the busy signal corresponding to each operation core performing operations on one frame of data fed back from the operation array, it generates a non-valid pulse signal; when it receives the idle signal corresponding to each operation core finishing operations on one frame of data fed back from the operation array, it generates the valid pulse signal.

[0009] In an embodiment of the present invention, the controller includes: an instruction cache module, a prefetch data instruction processing module, a data cache module, a non-prefetch data instruction processing module, and an execution module; wherein, the instruction cache module is used to read and store the prefetch data instruction and the non-prefetch data instruction from the outside; the prefetch data instruction processing module is connected to the instruction cache module and the data cache module, and is used to process the sequentially input prefetch data instructions, control the data cache module to obtain the frame data related to the prefetch data instruction from the external storage, and generate the status information corresponding to the stored prefetch data instruction; the data cache module is connected to the prefetch data instruction processing module, and is used to obtain and store the frame data related to the prefetch data instruction from the external storage based on the control of the prefetch data instruction processing module; the non-prefetch data instruction processing module is connected to the instruction cache module, and is used to control the generation of the corresponding continue execution instruction based on the sequentially input non-prefetch data instruction and the input frame synchronization pulse signal; the execution module is connected to the prefetch data instruction processing module, the non-prefetch data instruction processing module, and the data cache module, and is used to control the stop of the operation corresponding to the prefetch data instruction to the arithmetic array between the processing of two frames of instruction streams based on the received continue execution instruction and the obtained status information of the prefetch data instruction, and control the execution of the operation corresponding to the prefetch data instruction to the arithmetic array after the data operation of each arithmetic core in one frame is completed.

[0010] In an embodiment of the present invention, the prefetch data instruction processing module includes: a data prefetch instruction cache module, which is used to store the prefetch data instruction sent by the instruction cache module; wherein, the types of the prefetch data instruction include: valid prefetch data instruction and pause execution signal; a data prefetch instruction decoding module, which is connected to the data instruction cache module, and is used to decode the prefetch data instruction obtained from the data instruction cache module. If the prefetch data instruction is a valid prefetch data instruction, a prefetch signal for the data cache module to prefetch the frame data related to the valid prefetch data instruction is generated after decoding and sent to the data cache module, and the corresponding valid status information is generated for the execution module to obtain the corresponding frame data from the data cache module and execute the corresponding operation to the arithmetic array; if the prefetch data instruction is a pause execution signal, the corresponding pause status information is generated after decoding for the execution module to pause the execution of the corresponding operation to the arithmetic array; a status cache module, which is connected to the data prefetch instruction decoding module, and is used to store the valid status information and the pause status information of the prefetch data instruction.

[0011] In an embodiment of the present invention, the non-prefetch data instruction processing module includes: a non-data prefetch instruction cache module for non-prefetch data instructions sent by the instruction cache module; wherein, the types of the non-prefetch data instructions include: wait instructions and continue instructions; a non-data prefetch instruction processing module connected to the non-data prefetch instruction cache module, configured to, when the non-prefetch data instruction obtained from the non-data prefetch instruction cache module is the wait instruction and the input frame synchronization pulse signal is a valid pulse signal, process the next non-prefetch data instruction; if the input frame synchronization pulse signal is in a non-valid pulse signal state, pause processing the next non-prefetch data instruction until the frame synchronization pulse signal changes to a valid pulse signal; when the non-prefetch data instruction obtained from the non-data prefetch instruction cache module is a continue instruction, generate a corresponding continue execution instruction and send it to the execution unit for the execution module to continue obtaining the status information of the prefetch data instruction from the status cache module, and obtain the corresponding frame data from the data cache module according to the status information to perform the operation corresponding to the prefetch data instruction on the arithmetic array.

[0012] In an embodiment of the present invention, the types of the pause execution signals include: a single pause execution signal inserted between valid prefetch data instructions of two adjacent frames and a prefetch pause execution signal.

[0013] In an embodiment of the present invention, the manner in which, if the prefetch data instruction is a pause execution signal, corresponding pause status information is generated after decoding for the execution module to pause performing the corresponding operation on the arithmetic array includes: if the prefetch data instruction is a single pause execution signal, directly generate the corresponding pause status information after decoding for the execution module to pause performing the operation corresponding to the next valid prefetch data instruction; if the prefetch data instruction is a prefetch pause execution signal, generate a prefetch signal for the data cache module to prefetch frame data related to the prefetch pause execution signal after decoding and send it to the data cache module, and generate the corresponding pause status information for the execution module to pause performing the operation corresponding to the prefetch pause execution signal.

[0014] In an embodiment of the present invention, the instruction cache module is further configured to send the prefetch data instruction to the prefetch data instruction cache module and send the non-prefetch data instruction to the non-prefetch data instruction cache module.

[0015] In an embodiment of the present invention, the prefetch data instructions stored in the instruction cache module are arranged by frame; wherein, multiple prefetch data instructions correspond to one frame.

[0016] In an embodiment of the present invention, the effective pulse signal is a rising edge signal or a falling edge signal.

[0017] As described above, the present invention is a data processing system for realizing data prefetching, and has the following beneficial effects: through the frame synchronization pulse signal and the controller, the present invention not only realizes the data prefetching control function but also realizes the software and hardware synchronization, can be applied to single-core systems and multi-core systems, and is more flexible to use. The controller of the present invention is simple to control and can be realized with a single thread. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It shows a schematic structural diagram of a data processing system for realizing data prefetching in an embodiment of the present invention.

[0019] Figure 2 It shows a schematic structural diagram of a controller in an embodiment of the present invention.

[0020] Figure 3 It shows a schematic structural diagram of a data processing system with a data prefetching function in an embodiment of the present invention.

[0021] Figure 4 It shows a schematic structural diagram of a controller for realizing data prefetching in an embodiment of the present invention.

[0022] Figure 5 It shows a schematic diagram of an instruction stream for realizing synchronization in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The following specific examples illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0024] It should be noted that in the following description, reference is made to the accompanying drawings, which describe several embodiments of the present invention. It should be understood that other embodiments may also be used, and mechanical composition, structure, electrical, and operational changes may be made without departing from the spirit and scope of the present invention. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present invention is only defined by the claims of the published patent. The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. Spatially relative terms, such as "upper", "lower", "left", "right", "below", "beneath", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between one element or feature shown in the figure and another element or feature.

[0025] Throughout the specification, when it is said that a part is "connected" to another part, this includes not only the case of "direct connection", but also the case of "indirect connection" in which other elements are placed therebetween. In addition, when it is said that a certain part "includes" a certain constituent element, unless there is a particularly contrary record, it does not exclude other constituent elements, but means that other constituent elements may also be included.

[0026] The first, second, and third terms mentioned therein are used to describe various parts, components, regions, layers, and / or segments, but are not limited thereto. These terms are only used to distinguish one part, component, region, layer, or segment from other parts, components, regions, layers, or segments. Therefore, the first part, component, region, layer, or segment described below may refer to the second part, component, region, layer, or segment within the scope not exceeding the present invention.

[0027] Furthermore, as used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising", "including" indicate the presence of the stated features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or meaning any one or any combination. Thus, "A, B, or C" or "A, B, and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B, and C". An exception to this definition occurs only when the combination of elements, functions, or operations is inherently mutually exclusive in some way.

[0028] The present invention not only realizes the data prefetch control function through the frame synchronization pulse signal and the controller, but also realizes the synchronization of software and hardware. It can be applied to single-core systems and multi-core systems and is more flexible to use. The controller of the present invention is simple to control and can be realized with a single thread.

[0029] The following will be a detailed description of the embodiments of the present invention with reference to the accompanying drawings, so that those skilled in the technical field of the present invention can easily implement it. The present invention can be embodied in many different forms and is not limited to the embodiments described herein.

[0030] As Figure 1 Shown is a schematic structural diagram of a data processing system for implementing data prefetching in an embodiment of the present invention.

[0031] The system includes:

[0032] A frame synchronization pulse signal generating device 11 for generating a changing frame synchronization pulse signal; wherein, the changing states of the frame synchronization pulse signal include: a valid pulse signal and a non-valid pulse signal;

[0033] An arithmetic array 12 connected to the frame synchronization pulse signal generating device 11, including: one or more arithmetic cores, respectively synchronized with the frame synchronization pulse signal; that is, all arithmetic cores are synchronized with the same frame synchronization pulse signal;

[0034] A controller 13 connected to the frame synchronization pulse signal generating device 11 and the arithmetic array 12, for prefetch control of frame data related to the obtained prefetch data instruction, that is, controlling whether to perform prefetch of frame data; and based on the input frame synchronization pulse signal and the obtained non-prefetch data instruction, controlling to execute the operation corresponding to the prefetch data instruction on the arithmetic array, that is, controlling whether to execute the operation corresponding to the prefetch data instruction on the arithmetic array, for each arithmetic core to perform arithmetic operations on the frame data; wherein, one or more frame data correspond to one frame of data;

[0035] Wherein, the frame synchronization pulse signal generating device generates a non-valid pulse signal when each arithmetic core performs arithmetic operations on one frame of data, and generates the valid pulse signal after each arithmetic core finishes arithmetic operations on one frame of data. For example, when each arithmetic core calculates three frame data of the first frame, the frame synchronization pulse signal generating device generates a non-valid pulse signal; when each arithmetic core finishes calculating the three frame data of the first frame, the frame synchronization pulse signal generating device generates a valid pulse signal.

[0036] The above-mentioned data processing system for implementing data prefetching not only has the data prefetch function, but also realizes the synchronization of software and hardware.

[0037] Optionally, the valid pulse signal is a rising edge signal or a falling edge signal. In contrast, the non-valid pulse signal is a non-rising edge signal or a non-falling edge signal, which can be set according to specific circumstances.

[0038] Optionally, the frame synchronization pulse signal generating device 11 is in an automatic switching mode. When receiving the busy signal corresponding to each arithmetic core in the arithmetic array 12 for performing data operations of one frame, it generates a non-valid pulse signal; when receiving the idle signal corresponding to each arithmetic core in the arithmetic array 12 after the data operations of each pair of frames are completed, it generates the valid pulse signal to indicate the start of the next frame.

[0039] It should be noted that the busy signal is a flag indicating whether the operations of one frame are still in progress. That is, the arithmetic array 12 feeds back this signal during calculations and can feed back the idle signal when not calculating.

[0040] Optionally, the form of the frame synchronization pulse signal generated by the frame synchronization pulse signal generating device 11 can also be a counting method, that is, a frame synchronization pulse signal is generated at a certain time interval; for example, a period is fixed at 1 millisecond.

[0041] Optionally, as Figure 2 shown, the controller includes: an instruction cache module 21, a prefetch data instruction processing module 22, a data cache module 23, a non-prefetch data instruction processing module 24, and an execution module 25;

[0042] Among them,

[0043] The instruction cache module 21 is used to read and store the prefetch data instructions and non-prefetch data instructions from the outside; specifically, read and store the prefetch data instructions and non-prefetch data instructions from the outside in sequence; optionally, read the prefetch data instructions and non-prefetch data instructions from an external memory.

[0044] The prefetch data instruction processing module 22 is connected to the instruction cache module 21 and the data cache module 23, and is used to process the sequentially input prefetch data instructions, control the data cache module to obtain the frame data related to the prefetch data instruction from the external memory, and generate the status information corresponding to the stored prefetch data instruction.

[0045] The data cache module 23 is connected to the prefetch data instruction processing module 22, and is used to obtain and store the frame data related to the prefetch data instruction from the external memory based on the control of the prefetch data instruction processing module, and send the information (including but not limited to the storage address) of the stored frame data back to the prefetch data instruction processing module 22.

[0046] The non-prefetch data instruction processing module 24 is connected to the instruction cache module 23 and is used to control the generation of corresponding continue execution instructions based on the sequentially input non-prefetch data instructions and the input frame synchronization pulse signal.

[0047] The execution module 25 is connected to the prefetch data instruction processing module 33, the non-prefetch data instruction processing module 24, and the data cache module 23. It is used to control the suspension of the operation corresponding to the prefetch data instruction to the arithmetic array during the processing of two frames of instruction streams based on the received continue execution instructions and the obtained status information of the prefetch data instructions. After the data operation of each arithmetic core in one frame is completed, it obtains the corresponding frame data from the data cache module 23 and controls the execution of the operation corresponding to the prefetch data instruction to the arithmetic array. For controlling the suspension of the operation corresponding to the prefetch data instruction to the arithmetic array during the processing of two frames of instruction streams, it can also be applied to the following situations, for example; each core in the arithmetic array has two data caches. Time 1: Cache a stores the first frame of data. Time 2: The data in cache a (the first frame) is being operated on, and at the same time, the execution unit in the invention writes data to cache b (the second frame). Time 3: The data in cache b (the second frame) is being operated on, and at the same time, the execution unit in the invention writes data to cache a (the third frame). If the data transfer speed is very fast and exceeds the operation speed, the data transfer operation needs to be controlled. For example: Time 2: If the data of the first frame is being operated on and the data of the second frame has been written to cache b, if the data transfer instruction of the third frame is continued to be executed, then the data of the third frame will either be written to cache a (the first frame data in cache a is still in use, and if the data is overwritten, the operation will be abnormal), or written to cache b (then the data of the second frame will be lost). Therefore, a pause needs to be inserted between two frames of instruction streams, waiting for the first frame operation to be completed and releasing cache a before the data of the third frame can be transferred to cache a (Time 3).

[0048] It should be noted that the prefetch data instruction processing module 22 and the non-prefetch data instruction processing module 24 execute in parallel; the execution of the continue execution instruction on the non-prefetch data instruction processing module 24 can unlock the execution operation of the data prefetch processing channel. The execution operations of the prefetch data instruction processing module 22 and the non-prefetch data instruction processing module 24 are synchronized on the continue execution instruction, thereby realizing the synchronization of the instruction execution operation on the prefetch data instruction processing module 22 with other hardware.

[0049] Optionally, the prefetch data instructions stored in the instruction cache module 21 are arranged by frame; among them, multiple prefetch data instructions correspond to one frame. For example, the first frame corresponds to the first instruction of the first frame, the second instruction of the first frame, and the third instruction of the first frame; the second frame corresponds to the first instruction of the second frame, the second instruction of the second frame, and the third instruction of the second frame.

[0050] Optionally, the prefetch data instruction processing module 22 includes:

[0051] A data prefetch instruction cache module 221 for storing the prefetch data instructions sent by the instruction cache module; wherein, the types of the prefetch data instructions include: valid prefetch data instructions and a pause execution signal for causing the execution module 25 to pause from performing corresponding operations on the arithmetic array;

[0052] A data prefetch instruction decoding module 222, connected to the data instruction cache module 221, for decoding the prefetch data instructions obtained from the data instruction cache module 221;

[0053] If the prefetch data instruction is a valid prefetch data instruction, a prefetch signal is generated after decoding and sent to the data cache module for the data cache module to prefetch frame data related to the valid prefetch data instruction from the external storage; and valid state information corresponding to the valid prefetch data instruction is generated based on the decoded data and the information of the frame data cached by the data cache module, for the execution module 25 to extract the corresponding frame data from the data cache module based on this information and perform corresponding operations on the arithmetic array.

[0054] If the prefetch data instruction is a pause execution signal, corresponding pause state information is generated after decoding for the execution module 25 to pause from performing corresponding operations on the arithmetic array.

[0055] Among them, data prefetch operations do not require synchronization. As long as there is free cache space in the data cache module, data can be prefetched. When the prefetch data instruction can be executed, the prefetched data in the data cache module can be quickly retrieved and quickly executed.

[0056] A status cache module 223, connected to the data prefetch instruction decoding module 222, for storing the status information of the prefetch data instructions for the execution module 25 to sequentially obtain the status information of each of the prefetch data instructions from the status cache module.

[0057] Optionally, the types of the pause execution signals include: a single pause execution signal inserted between the valid prefetch data instructions of two adjacent frames and a prefetch pause execution signal. Among them, the single pause execution signal only has a single pause function, and the prefetch pause execution signal not only has a pause function but also has a prefetch function.

[0058] For example, the arrangement order of the prefetch data instructions cached in the data prefetch instruction cache module is:

[0059] The first instruction of the first frame, the second instruction of the first frame, the third instruction of the first frame, a single pause execution signal, the first instruction of the second frame, the second instruction of the second frame, the third instruction of the second frame;

[0060] That is, after the execution module 25 executes the first instruction of the first frame, the second instruction of the first frame, and the third instruction of the first frame to the arithmetic array respectively, when the pause status information of the single pause execution signal is obtained, the execution of the next instruction is paused; until a continue execution instruction from the non-data prefetch instruction cache module is received, the instruction of the next frame is continued to be executed.

[0061] Take another example. The arrangement order of the prefetch data instructions cached in the data prefetch instruction cache module is: the first instruction of the first frame, the second instruction of the first frame, the third instruction of the first frame, a prefetch pause execution signal (the first instruction of the second frame), the second instruction of the second frame, the third instruction of the second frame;

[0062] That is, after the execution module 25 executes the first instruction of the first frame, the second instruction of the first frame, and the third instruction of the first frame to the arithmetic array respectively, when the pause status information of the prefetch pause execution signal (the second frame) is obtained, the operation of the instruction is paused until a continue execution instruction from the non-data prefetch instruction cache module is received, and then the instruction is continued to be executed.

[0063] Optionally, the method for generating corresponding pause status information after decoding the prefetch data instruction as a pause execution signal for the execution module 25 to pause the execution of the corresponding operation to the arithmetic array includes:

[0064] If the prefetch data instruction is a single pause execution signal, corresponding pause status information is directly generated after decoding for the execution module 25 to pause the operation corresponding to the next valid prefetch data instruction;

[0065] If the prefetch data instruction is a prefetch pause execution signal, a prefetch signal for the data cache module to prefetch frame data related to the prefetch pause execution signal is generated and sent to the data cache module after decoding, and corresponding pause status information is generated for the execution module 25 to pause the operation corresponding to the prefetch pause execution signal to the arithmetic array.

[0066] Optionally, the non-prefetch data instruction processing module 24 includes:

[0067] A non-data prefetch instruction cache module 241 for non-prefetch data instructions sent by the instruction cache module; wherein, the types of the non-prefetch data instructions include: wait instructions, continue instructions, etc.;

[0068] The non-data prefetch instruction processing module 242, connected to the non-data prefetch instruction cache module 241, is configured to process the next non-prefetch data instruction when the non-prefetch data instruction obtained from the non-data prefetch instruction cache module 241 is the waiting instruction and the input frame synchronization pulse signal is a valid pulse signal; if the input frame synchronization pulse signal is in a non-valid pulse signal state, it will pause processing the next non-prefetch data instruction until the frame synchronization pulse signal changes to a valid pulse signal, and then process the next prefetch data instruction;

[0069] When the non-prefetch data instruction obtained from the non-data prefetch instruction cache module 241 is a continue instruction, a corresponding continue execution instruction is generated and sent to the execution unit, so that the execution module 25 can continue to obtain the status information of the prefetch data instruction from the status cache module 223, and obtain the corresponding frame data from the data cache module according to the status information, so as to execute the operation corresponding to the prefetch data instruction on the arithmetic array.

[0070] Optionally, the instruction cache module further has a classification function, which is used to send the prefetch data instruction to the prefetch data instruction cache module and send the non-prefetch data instruction to the non-data prefetch instruction cache module.

[0071] Optionally, the non-data prefetch instruction processing module 242 is also connected to other hardware signals, such as indicator. When decoding wait_indicator, it will wait for the hardware signal indicator; thus, synchronization between the non-data prefetch processing channel and other hardware is achieved.

[0072] Optionally, data prefetch operations do not require synchronization. As long as the data cache is empty, data can be prefetched. When the data prefetch processing channel can execute, the prefetched data in the data cache can be quickly retrieved and quickly executed.

[0073] To better illustrate the above data processing system for data prefetch, the present invention provides the following specific embodiments.

[0074] Embodiment 1: A data processing system with a data prefetch function. As Figure 3 shown in the structural schematic diagram of the system;

[0075] The system includes:

[0076] An arithmetic queue with multiple arithmetic cores, each arithmetic core is synchronized with the same hardware signal, which will be named sync in the following text. The sync signal is similar to a frame synchronization pulse signal. After the rising edge of the sync signal arrives, if the arithmetic core is started, it will start to operate.

[0077] The sync generation unit is used to generate sync signal pulses. Generating sync signal pulses is in the automatic switching mode: that is, when the previous frame operation of the arithmetic array ends, sync automatically changes and generates a valid pulse to indicate the start of the next frame. The busy signal is a flag indicating whether a frame operation is still in progress, and it is fed back from the arithmetic array to the sync generation unit.

[0078] The controller is connected to the sync signal. To achieve data prefetching, instructions are divided into data prefetching instructions and non-data prefetching instructions. Data prefetching instructions need to prefetch data; non-data prefetching instructions do not need to prefetch data. At the same time, an instruction that pauses the corresponding execution function of the data prefetching instruction is defined, which is hereinafter referred to as exe_stop; an instruction that resumes the corresponding execution function of the data prefetching instruction is defined, which is hereinafter referred to as exe_resume. After exe_stop, the data prefetching instruction can prefetch data after decoding, but cannot be executed until the resume signal is generated after the exe_resume instruction is decoded. After the resume signal is valid, the data prefetching instruction after exe_stop continues to execute.

[0079] It should be noted that exe_stop can only have a single pause function; it can also be extended to add a pause function for data prefetching. Assume the instruction name is prefetch_with_exe_stop, which means this instruction prefetches data, and at the same time, this instruction and subsequent data prefetching instructions can prefetch data from external storage before the exe_resume instruction is executed, but the execution operation (such as not being able to transfer data to the arithmetic array) is paused. For simplicity, in the following text, exe_stop only has a single pause function.

[0080] If the executed instruction is wait_sync, the controller will wait for the valid edge of sync to arrive. After the valid edge of sync arrives, the controller ends the wait and processes the subsequent instructions. Among them, the instructions before wait_sync correspond to the current frame n. When the controller executes wait_sync, the arithmetic array performs the operation of frame n. When the operation of frame n ends, the hardware signal busy becomes 0, and the valid edge of sync will be generated. After the controller detects the valid edge of sync, it executes the instructions of frame n + 1.

[0081] Figure 4It is a block diagram of a controller with data prefetching function. External instructions enter the instruction cache, and the instruction classifier distributes the instructions in the instruction cache to different paths: Data prefetch instructions (including exe_stop instructions) enter the data prefetch processing channel. The data prefetch instructions are decoded in sequence. After decoding, corresponding prefetch data operations are performed, and at the same time, the state is passed to the execution unit. The data read from the external memory is written into the data cache; The execution unit, without seeing the exe_stop instruction, reads the data in the data cache and performs the required operations according to the status information (such as transporting data to the arithmetic array); When the execution unit sees the exe_stop instruction, the execution unit pauses reading the data cache and the status cache, so that subsequent instructions pause the execution operation (such as transporting data to the arithmetic array) until a valid resume signal is issued on the non-data prefetch channel, and then the execution operation can continue. The execution operation starts from the instruction where it was paused (the next instruction after exe_stop). Non-data prefetch instructions (including exe_resume instructions) enter the non-data prefetch processing channel, and the non-data prefetch instructions are decoded in sequence. If it is found that the current instruction is a wait_sync instruction, it will wait until the valid edge of sync arrives and then execute the next non-data prefetch instruction; If it is found that the current instruction is exe_resume, a valid resume signal is immediately sent to the data prefetch instruction channel, thereby unlocking the execution operation of the data prefetch channel.

[0082] The data prefetch instruction processing channel and the non-data prefetch processing channel execute in parallel; The execution of exe_resume on the non-data prefetch channel can unlock the execution operation of the data prefetch processing channel. The execution operations of the non-data prefetch processing channel and the prefetch data processing channel are synchronized on exe_resume, thereby realizing the synchronization of the instruction execution operation on the prefetch data processing channel with other hardware.

[0083] Among them, such as Figure 5As shown, the instruction stream of the current frame 1 is before Exe_stop. The instructions between Exe_stop and Wait_sync are all the instructions to be executed in frame 2. These instructions are data prefetch instructions, which enter the data prefetch processing channel to prefetch the data of frame 2 during the operation of frame 1. Due to the existence of exe_stop, the execution units on the data prefetch processing channel pause working after the start of processing exe_stop. The data prefetch instruction processing channel and the non-data prefetch processing channel execute in parallel. The incomplete data prefetch of multiple instructions in frame 2 does not affect the entry of the Wait_sync instruction into the non-data prefetch processing channel. Even if the data prefetch processing channel is still decoding the first few instructions of frame 2, wait_sync can enter the non-data prefetch processing channel. When the non-data prefetch processing channel finishes executing the non-data prefetch instructions (if any) before wait_sync, it starts to decode wait_sync. As long as the operation of frame 1 has not ended, the non-data prefetch processing channel is in a waiting state; while the non-data prefetch processing channel is waiting, the data prefetch processing channel is performing data prefetch for the instructions after exe_stop. As long as there is still space in the data cache, the data prefetch is ongoing. When the operation of frame 1 ends (busy changes from 1 to 0) and the sync valid edge of frame 2 arrives, the non-data prefetch processing channel ends waiting and continues to execute the subsequent non-data prefetch instructions. When the non-data prefetch processing channel sees the exe_resume instruction, it immediately sends a valid resume signal to the data prefetch processing channel. The execution units on the data prefetch processing channel resume working from the pause point (the first data prefetch instruction of frame 2), and the data of frame 2 is processed and operated on.

[0084] Data processing is carried out through the above scheme, which has the following advantages:

[0085] (1) The conventional software-hardware synchronization schemes in existing deep neural network hardware accelerators are mainly applied to single-core systems. The present invention realizes the synchronization between software and hardware in a multi-core system.

[0086] (2) The synchronization between software and hardware is realized, which is difficult to achieve with the conventional software-hardware synchronization schemes in deep neural network hardware accelerators, and it can be applied to both single-core systems and multi-core systems.

[0087] (3) The present invention can achieve relatively flexible synchronization. For example, it is required that a certain time passes after the sync valid edge before the subsequent instructions can be executed. Instructions for waiting for a fixed time can be inserted between wait_sync and exe_resume. As long as the non-data prefetch channel supports a counter. If it is required to wait for other hardware signals, wait_indicator can be extended to other corresponding instructions, while the conventional software-hardware synchronization schemes in deep neural network hardware accelerators are not flexible enough.

[0088] Compared with the conventional software-hardware synchronization scheme in deep neural network hardware accelerators, the software control of this method is simple and can be implemented with a single thread.

[0089] In summary, the data processing system for realizing data prefetching according to the present invention not only realizes the data prefetching control function but also realizes software-hardware synchronization through the frame synchronization pulse signal and the controller. It can also be applied to single-core systems and multi-core systems and is more flexible to use. The controller of the present invention has simple control and can be implemented with a single thread. Therefore, the present invention effectively overcomes various disadvantages in the prior art and has high industrial utilization value.

[0090] The above embodiments are only used to exemplarily illustrate the principles and effects of the present invention, rather than to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A data processing system for implementing data prefetching, characterized in that: The system comprises: A frame synchronization pulse signal generating device, configured to generate a changing frame synchronization pulse signal; wherein the changing state of the frame synchronization pulse signal includes: a valid pulse signal and a non-valid pulse signal; A computing array, connected to the frame synchronization pulse signal generating device, comprising: one or more computing cores, each synchronized with the frame synchronization pulse signal; a controller connected to the frame synchronization pulse signal generating device and the operation array, configured to perform prefetch control on frame data associated with the acquired prefetch data instruction, and control the operation array to execute the operation corresponding to the prefetch data instruction based on the input frame synchronization pulse signal and the acquired non-prefetch data instruction, so that each operation core performs operation on the frame data; wherein one or more frame data correspond to one frame of data; The frame synchronization pulse signal generating device generates a non-valid pulse signal when each operation core performs a data operation for one frame, and generates the valid pulse signal after each operation core completes a data operation for one frame; The controller includes: an instruction cache module, a prefetch data instruction processing module, a data cache module, a non-prefetch data instruction processing module and an execution module; wherein the instruction cache module is used to read and store the prefetch data instruction and the non-prefetch data instruction from the outside; the prefetch data instruction processing module is connected to the instruction cache module and the data cache module, and is used to process the prefetch data instructions input in sequence, control the data cache module to obtain frame data related to the prefetch data instruction from the external storage, and generate and store status information corresponding to the prefetch data instruction; the data cache module is connected to the prefetch data instruction processing module, and is used to read and store the prefetch data instruction from the external storage based on the control of the prefetch data instruction processing module. The non-prefetch data instruction processing module is connected to the instruction cache module and is used to control the generation of the corresponding continue execution instruction based on the non-prefetch data instruction and the frame synchronization pulse signal input in sequence; the execution module is connected to the prefetch data instruction processing module, the non-prefetch data instruction processing module and the data cache module and is used to control the stopping of the execution of the operation corresponding to the prefetch data instruction to the operation array between the processing of two frames of instruction streams based on the received continue execution instruction and the status information of the obtained prefetch data instruction, and control the execution of the operation corresponding to the prefetch data instruction to the operation array after the data operation of one frame of each operation core is completed.

2. The data processing system for implementing data prefetching according to claim 1, characterized in that: The frame synchronization pulse signal generating device generates a non-valid pulse signal when receiving a busy signal from the operation array when each corresponding operation core performs a frame of data operation; and generates the valid pulse signal when receiving an idle signal from the operation array after each corresponding operation core completes a frame of data operation.

3. The data processing system for implementing data prefetching according to claim 1, wherein: The prefetch data instruction processing module includes: A data prefetch instruction cache module, configured to store prefetch data instructions sent by the instruction cache module; wherein the types of the prefetch data instructions include: valid prefetch data instructions and pause execution signals; a data prefetch instruction decoding module, connected to the data instruction cache module, for decoding a prefetch data instruction obtained from the data instruction cache module; if the prefetch data instruction is a valid prefetch data instruction, generating a prefetch signal for the data cache module to prefetch frame data related to the valid prefetch data instruction and sending it to the data cache module, and generating corresponding valid status information for the execution module to obtain the corresponding frame data from the data cache module and perform a corresponding operation on the operation array; if the prefetch data instruction is a pause execution signal, generating corresponding pause status information for the execution module to pause the execution of the corresponding operation on the operation array after decoding; The status cache module is connected to the data prefetch instruction decoding module and is used to store the valid status information and pause status information of the prefetch data instruction.

4. The data processing system for implementing data prefetching according to claim 3, wherein: The non-prefetch data instruction processing module includes: A non-data prefetch instruction cache module, configured to receive non-data prefetch instructions sent by the instruction cache module; wherein the types of the non-data prefetch instructions include: wait instructions and continue instructions; A non-data prefetch instruction processing module is connected to the non-data prefetch instruction cache module, and is used to process the next non-prefetch data instruction when the non-prefetch data instruction obtained from the non-data prefetch instruction cache module is the wait instruction and the input frame synchronization pulse signal is a valid pulse signal; if the input frame synchronization pulse signal is in a non-valid pulse signal state, the processing of the next non-prefetch data instruction will be suspended until the frame synchronization pulse signal changes to a valid pulse signal; when the non-prefetch data instruction obtained from the non-data prefetch instruction cache module is a continue instruction, a corresponding continue execution instruction is generated and sent to the execution module, so that the execution module continues to obtain the status information of the prefetch data instruction from the status cache module, and obtains the corresponding frame data from the data cache module based on the status information, so as to execute the operation corresponding to the prefetch data instruction to the operation array.

5. The data processing system for implementing data prefetching according to claim 3, characterized in that: The types of the execution pause signal include: a single execution pause signal inserted between valid prefetch data instructions of two adjacent frames and a prefetch execution pause signal.

6. The data processing system for implementing data prefetching according to claim 5, characterized in that: If the prefetch data instruction is a pause execution signal, generating corresponding pause status information after decoding so that the execution module can pause executing the corresponding operation on the operation array includes: If the prefetch data instruction is a single pause execution signal, corresponding pause status information is directly generated after decoding, so that the execution module can pause the execution of the operation corresponding to the next valid prefetch data instruction to the operation array; If the prefetch data instruction is a prefetch pause execution signal, a prefetch signal is generated after decoding for the data cache module to prefetch the frame data related to the prefetch pause execution signal and is sent to the data cache module. Corresponding pause state information is also generated for the execution module to pause executing the operation corresponding to the prefetch pause execution signal on the operation array.

7. The data processing system for implementing data prefetching according to claim 1, characterized in that: The instruction cache module is further configured to send the prefetch data instruction to the prefetch data instruction cache module, and send the non-prefetch data instruction to the non-prefetch data instruction cache module.

8. The data processing system for implementing data prefetching according to claim 1, characterized in that: The pre-fetch data instructions stored in the instruction cache module are arranged in frames; wherein, a plurality of pre-fetch data instructions correspond to one frame.

9. The data processing system for implementing data prefetching according to claim 3, characterized in that: The effective pulse signal is a rising edge signal or a falling edge signal.

Images