Data processing device and method and chip

By adopting array distribution processing modules and pulsation transmission methods in the data processing device, the problem of large register consumption in the pulsation array is solved, and more efficient data processing and layout are achieved.

CN119002996APending Publication Date: 2024-11-22TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310567368.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When using pulsating arrays for data processing, a large number of registers are required for delay processing, resulting in large register consumption and long data processing delays.

Method used

Using a processing module distributed in an array, each processing module includes a tap register and a computing unit. The data is delayed through the tap register and broadcast to the computing unit. The data is transmitted by a pulsating transmission method between the processing modules to reduce register deployment. Synchronize the transmission volume with data.

Benefits of technology

It reduces register consumption and data processing delay, improves data processing efficiency, simplifies the chip back-end layout and routing, and improves the surface efficiency ratio.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002996A_ABST
    Figure CN119002996A_ABST
Patent Text Reader

Abstract

The invention discloses a data processing device and method and a chip, relates to the technical field of computers, and is used for reducing consumption of registers, reducing data processing delay and improving data processing efficiency. In the application, the data processing device comprises at least two processing modules distributed in an array, and each processing module comprises a beat register and at least two arithmetic units; wherein the beat register in one processing module is used for carrying out delay processing on to-be-processed data to obtain target data after delay processing, and broadcasting the target data to at least two arithmetic units contained in one processing module; transmitting the target data to a beat register in the next processing module in the same row as the position of one processing module; and the operation unit in one processing module is used for processing the target data and transmitting an obtained first processing result to the next target unit which is located in the same column as the operation unit.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] In order to speed up data processing efficiency, matrix operations are proposed, and systolic arrays are used as the core structure of data processing in matrix operation units that implement matrix operations; the systolic array allows data to flow in the array of operation units.

[0003] At present, when using a pulsating array for data processing, the data needs to be adjusted in form and enter the pulsating array in a certain order and in a time-sharing manner. Therefore, it is proposed to beat both the horizontal and vertical dimensions of the pulsating array. To match this timing, the horizontal and vertical data need to be triangulated and delayed. The strategy of triangulation delay processing is to add a row of data, and then add a delay of 1 beat (or one cycle) to each row of data.

[0004] It can be seen that a large number of registers are required for delay processing when triangulating the input data; and each operation unit in the pulsating array also needs to use registers to beat in the downward and rightward propagation directions, resulting in more registers being consumed; and setting a large number of registers leads to a large data processing delay.

[0005] Therefore, how to reduce register consumption and lower data processing delay is a technical problem that needs to be solved at present. Summary of the invention

[0006] The embodiments of the present application provide a data processing device, method and chip to reduce register consumption, lower data processing delay and improve data processing efficiency.

[0007] In a first aspect, an embodiment of the present application provides a data processing device, the device comprising: at least two processing modules distributed in an array; each processing module comprises: a beat register and at least two operation units; wherein:

[0008] A beat register in a processing module is used to delay processing of data to be processed, obtain target data after delay processing, broadcast the target data to at least two operation units included in a processing module, and transmit the target data to the beat register in the next processing module located in the same row as a processing module;

[0009] The operation unit in a processing module is used to process the target data and transmit the obtained first processing result to the next target unit located in the same column as the operation unit.

[0010] In a second aspect, an embodiment of the present application provides a data processing method, which is applied to the data processing device provided in the first aspect of the embodiment of the present application, and the method includes:

[0011] Performing delay processing on the data to be processed through a beat register in a processing module to obtain target data after delay processing, broadcasting the target data to at least two operation units included in a processing module, and transmitting the target data to a beat register in a next processing module located in the same row as the processing module;

[0012] The target data is processed by a computing unit in a processing module, and a first processing result is transmitted to a next target unit located in the same column as the computing unit.

[0013] In a third aspect, an embodiment of the present application provides a computing device, comprising: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program to implement the steps of the data processing method provided in the embodiment of the present application.

[0014] In a fourth aspect, an embodiment of the present application provides a chip, the chip includes a data processing device provided by an embodiment of the present application, and is used to execute the steps of the data processing method provided by an embodiment of the present application.

[0015] The beneficial effects of this application are as follows:

[0016] The embodiments of the present application provide a data processing device, method and chip, which relate to the field of computer technology. The data processing device provided in the present application includes: at least two processing modules distributed in an array; each processing module includes: a beat register and at least two operation units; wherein:

[0017] After the beat register in a processing module performs delay processing on the data to be processed, the target data obtained after the delay processing is broadcast to at least two operation units contained in the processing module; when broadcasting the target data to at least two operation units, each operation unit is independent of each other, and the operation units do not need to be connected through registers. Compared with the method of setting registers between two adjacent operation units in the prior art, the register deployment is reduced; reducing the register deployment means reducing the number of registers deployed on the chip, that is, reducing the chip area occupied by the registers. Therefore, there is no need to deploy a large number of registers on the chip, which reduces the chip area consumption; at the same time, the broadcast method enables all operation units in the same processing module to receive the target data at the same time and process the target data synchronously. Compared with the prior art, when a register for data delay processing is set between two adjacent operation units, resulting in a difference of one beat or one cycle in the reception of data by the two adjacent operation units, and a difference of one beat or one cycle in data processing, the implementation method of the present application reduces data processing delay and improves data processing efficiency;

[0018] And, the target data is transmitted to the beat register in the next processing module located in the same row as a processing module; from the function of the beat register given above, it can be seen that: after the target data is transmitted to the beat register in the next processing module, the beat register in the next processing module will delay the target data and broadcast the delayed data to at least two computing units in the same processing module; it can be seen that before the target data enters the processing unit in the next processing module, the data is first delayed, that is, the data is beat; and the pulsating transmission method is to beat the data at every distance, forming a beat or a cycle delay; therefore, in the embodiment of the present application, the processing data corresponding to the computing units between two adjacent processing modules in the same row adopts the pulsating transmission method. Since the pulsating transmission method has the characteristics of reducing the amount of synchronous data transmission and reducing data congestion, the required bandwidth of the data bus and the complexity of the data processing path are reduced on the basis of reducing the amount of synchronous data transmission, which further alleviates the difficulty of back-end layout and wiring convergence and is more convenient for chip back-end layout and wiring;

[0019] Furthermore, the operation unit in one processing module processes the target data and transmits the obtained first processing result to the next target unit located in the same column as the operation unit; thus, the processing of the target data can be successfully completed;

[0020] In summary, the embodiment of the present application proposes a data processing device that combines broadcasting and pulsation. On the basis of ensuring the smooth processing of target data, it reduces the consumption of registers, further reduces the chip area consumption, is beneficial to the chip back-end layout and wiring, and at the same time reduces data processing delays and improves data processing efficiency, thereby improving the area efficiency.

[0021] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 A schematic diagram of systolic array data processing in the related art;

[0024] Figure 2A schematic diagram of a triangulation delay process in the related art;

[0025] Figure 3 A schematic diagram of a pulsating array in the related art;

[0026] Figure 4 A schematic diagram of a computing module in a systolic array in the related art;

[0027] Figure 5 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of another data processing device provided in an embodiment of the present application;

[0030] Figure 8 A schematic diagram of a data processing device provided in an embodiment of the present application;

[0031] Fig. 9 A schematic diagram of another data processing device provided in an embodiment of the present application;

[0032] Fig.10 A schematic diagram of the structure of a computing unit provided in an embodiment of the present application;

[0033] Fig.11 A schematic diagram of the structure of a multiplication-addition subunit provided in an embodiment of the present application;

[0034] Fig.12 A schematic diagram of a tree adder provided in an embodiment of the present application;

[0035] Fig.13 A schematic diagram of a specific structure of a multiplication-addition subunit provided in an embodiment of the present application;

[0036] Fig.14 A schematic diagram of the structure of an accumulation subunit provided in an embodiment of the present application;

[0037] Fig.15 A schematic diagram of a specific structure of a computing unit provided in an embodiment of the present application;

[0038] Fig.16 A schematic diagram of a triangulated delay provided in an embodiment of the present application;

[0039] Fig.17 A flow chart of a data processing method provided in an embodiment of the present application;

[0040] Fig.18A flowchart of a specific implementation method of data processing provided in an embodiment of the present application;

[0041] Fig.19 A schematic diagram of feature graph data processing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and beneficial effects of this application clearer, the technical solution in the embodiment of this application will be clearly and completely described below in conjunction with the drawings in the embodiment of this application. Obviously, the described embodiment is only a part of the embodiment of this application, not all the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0043] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0044] Systolic Array: An array structure. "Pulsation" means that the working mode and process of data is like the working mode and process of the human blood circulation system. In this array structure, data "flows" rhythmically between the processing elements (PEs) of the array in a predetermined "pipeline" manner. During the data flow process, all processing modules process the data flowing through them in parallel at the same time to achieve a very high parallel processing speed.

[0045] Chip: Also known as microcircuit, microchip, integrated circuit (IC). It refers to a silicon chip containing an integrated circuit. It is very small and is often part of a computing device or other electronic device.

[0046] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0047] Broadcast: The data is simultaneously propagated to a group of components for processing data with a specific logical relationship; in the embodiment of the present application, all sub-data in the data to be processed are simultaneously propagated to all computing units in the same processing module.

[0048] Dot product: The elements of two vectors are multiplied together and the sum is a number (a dot), similar to how a vector finally gives a number (a dot); the dot product is named after the result of the operation.

[0049] The following is a brief introduction to the design concept of the embodiments of the present application.

[0050] In order to speed up data processing efficiency, matrix operations are proposed, and systolic arrays are used as the core structure of data processing in matrix operation units that implement matrix operations; the systolic array allows data to flow in the array of operation units.

[0051] Currently, there are two processing methods when using a systolic array for data processing;

[0052] Method 1: The pulsation array is fully pulsated in both the horizontal and vertical directions.

[0053] When the pulsation array uses full pulsation beats in both horizontal and vertical dimensions, the data needs to be adjusted in form and enter the pulsation array in a certain order and in time. In order to match this timing, the horizontal and vertical data need to be triangulated and delayed.

[0054] For example, see Figure 1 , Figure 1 The figure is a schematic diagram of a systolic array data processing in the related art; specifically, the horizontal data and the vertical data are triangulated and delayed according to the first-in first-out queue (FIFO), and the triangulated delayed data is transmitted to the systolic array. The strategy of triangulation delay processing is to add a row of data, and then add a delay of one beat (or one cycle) to each row of data; see Figure 2 , Figure 2 is a schematic diagram of a triangulation delay process in the related art. Figure 2 Taking 16 data as an example, the first data enters the systolic array without delay processing, the second data enters the systolic array after being delayed by one beat through a register, the third data enters the systolic array after being delayed by two beats through two registers, and so on.

[0055] The triangulated delayed data is transferred to the systolic array, and the data is processed by the computing units of the systolic array. Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a systolic array in the related art. The systolic array includes a plurality of computing units distributed in an array, and a register is arranged between two adjacent computing units arranged in a row direction. Figure 3 Corresponding to pair 2, taking 16 data as an example, the systolic array is a 16×16 systolic array, that is, 16 rows and 16 columns of operation units are set; see Figure 4 , Figure 4This is a schematic diagram of a computing unit in the related art. It can be seen that when the data processed by the computing unit is transmitted in the column direction, registers also need to be set.

[0056] In summary, the full-pulsation solution in both horizontal and vertical directions requires a large number of registers for delay processing when triangulating the input data; and each computing unit in the pulsation array also needs to set registers when transmitting data, and use registers to beat data (or delay data), resulting in more registers being consumed; and setting a large number of registers leads to greater data processing delays.

[0057] Method 1: Use horizontal pulsation and no vertical pulsation.

[0058] The method of horizontal pulsation and vertical non-pulsation is adopted. This method is usually implemented in an accumulation tree structure. Since the result is output from the bottom of the systolic array, it is far away from the multiplication unit above. At this time, for a large-scale systolic array, a high-bandwidth data bus and a complex data processing path are required, which makes it very difficult to converge the chip back-end layout and routing, which is not conducive to the chip back-end layout and routing.

[0059] Therefore, how to reduce register consumption, lower data processing delays, and facilitate chip back-end layout and routing is a technical problem that needs to be solved at present.

[0060] In view of this, the embodiments of the present application provide a data processing device, method and chip, which relate to the field of computer technology and are used to reduce register consumption, reduce data processing delay, improve data processing efficiency, and facilitate chip back-end layout and wiring.

[0061] The data processing device provided in the present application includes: at least two processing modules distributed in an array; each processing module includes: a beat register and at least two operation units; wherein: after the beat register in a processing module delays the data to be processed, the target data obtained after the delay processing is broadcast to at least two operation units contained in a processing module, and the target data is transmitted to the beat register in the next processing module located in the same row as the processing module, that is, the target data is transmitted to the next processing module by using a pulsation transmission method; the operation unit in a processing module processes the target data, and transmits the first processing result obtained to the next target unit located in the same column as the operation unit. A data processing device combining broadcast and pulsation is proposed, wherein the broadcast transmission method is adopted within the processing module, and the pulsation transmission method is adopted between the processing modules, which reduces the consumption of registers, further reduces the consumption of chip area, is beneficial to the layout and wiring of the chip backend, and at the same time reduces the data processing delay, improves the data processing efficiency, and thus improves the area efficiency ratio.

[0062] After introducing the design concept of the embodiment of the present application, the application scenario set in the present application is briefly described below. It should be noted that the following scenario is only used to illustrate the embodiment of the present application and is not limited. In the specific implementation, the technical solution provided by the embodiment of the present application can be flexibly applied according to actual needs.

[0063] See also Figure 5 , Figure 5 A schematic diagram of an application scenario provided in an embodiment of the present application; in this application scenario, a data processing device is mainly involved; the acquired data is input into the data processing device, and the data processing device performs matrix operations on the data. In a possible implementation, a processor supporting the processing function of the data processing device provided in an embodiment of the present application is provided in the data processing device.

[0064] Exemplarily, taking the acquired data as AI data in the AI ​​field as an example, the processor in the data processing device that supports the processing function of the data processing apparatus provided in the embodiment of the present application is an AI processor.

[0065] In the field of AI, AI data is processed through neural networks, and a large number of convolutional layers are involved in the neural network, and convolution operations are performed on AI data through the convolutional layers. Therefore, when AI data is processed by a data processing device, the convolution operation is mapped to the matrix operation unit for execution, and the AI ​​data is processed in the matrix operation unit using a systolic array.

[0066] Considering that when AI data is processed by a data processing device, the performance of the matrix operation unit directly affects the performance of the AI ​​processor, in the related art, high-performance AI processors usually have larger matrix operation units. While the larger matrix operation unit significantly improves the computing power of the AI ​​processor, it consumes a huge chip area, and in the process of chip back-end implementation, the high-bandwidth data bus and complex data processing path of the matrix operation unit make it difficult to converge the layout and routing. At this time, an AI processor that supports the processing function of the data processing device provided by the embodiment of the present application is used to reduce the resource consumption of the pre-triangulation delay first-in-first-out queue in the use of registers, broadcasts in each processing module, reduces the consumption of the beat register, and performs pulsation propagation between processing modules, which is beneficial to the layout and routing implemented in the back end.

[0067] In a possible application scenario, the data processing device may be a terminal device or a server.

[0068] When the data processing device is a terminal device, the structure of the data processing device can be as follows: Figure 6As shown, it includes: a communication component 610, a memory 620, a display unit 630, a camera 640, a sensor 660, an audio circuit 660, a Bluetooth module 670, a processor 680 and other components.

[0069] The communication component 610 is used to communicate with the server. In some embodiments, a wireless fidelity (WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The computing device can help users send and receive information through the WiFi module.

[0070] The memory 620 can be used to store software programs and data. The processor 680 executes various functions and data processing of the terminal device by running the software programs or data stored in the memory 620. The memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one type of disk storage device, flash memory device, or other volatile solid-state storage device. The memory 620 stores an operating system that enables the terminal device to run. In the present application, the memory 620 can store an operating system and various application programs, and can also store code for executing the data processing method of the embodiment of the present application.

[0071] The display unit 630 can also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal device. Specifically, the display unit 630 may include a display screen 632 disposed on the front of the terminal device. The display screen 632 may be configured in the form of a liquid crystal display, a light emitting diode, etc.

[0072] The display unit 630 can also be used to receive input digital or character information and generate signal input related to user settings and function control of the terminal device. Specifically, the display unit 630 may include a touch screen 631 arranged on the front of the terminal device, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0073] The touch screen 631 can be covered on the display screen 632, or the touch screen 631 and the display screen 632 can be integrated to realize the input and output functions of the terminal device, and the integrated display screen can be referred to as a touch display screen. In this application, the display unit 630 can display the application and the corresponding operation steps.

[0074] Camera 640 can be used to capture static images. There can be one or more cameras 640. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the processor 680 to be converted into a digital image signal. Exemplarily, in an embodiment of the present application, the camera 640 can capture images for image recognition; wherein, image recognition is only one scenario that can be realized in an embodiment of the present application.

[0075] The terminal device may further include at least one sensor 660, such as an acceleration sensor 661, a distance sensor 662, a fingerprint sensor 663, and a temperature sensor 664. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0076] The audio circuit 660, the speaker 661, and the microphone 662 can provide an audio interface between the user and the terminal device. The audio circuit 660 can transmit the electrical signal converted from the received audio data to the speaker 661, which is converted into a sound signal for output. The terminal device can also be equipped with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 662 converts the collected sound signal into an electrical signal, which is received by the audio circuit 660 and converted into audio data, and then the audio data is output to the communication component 610 to be sent to, for example, another terminal device, or the audio data is output to the memory 620 for further processing.

[0077] The Bluetooth module 670 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable computing device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 670 to exchange data.

[0078] The processor 680 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 620 and calling data stored in the memory 620. In some embodiments, the processor 680 may include one or more processing modules; the processor 680 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes an operating system, a user interface, and an application program, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 680. In this application, the processor 680 can run an operating system, an application program, a user interface display and a touch response, as well as a data processing method of an embodiment of the present application. In addition, the processor 680 is coupled to the display unit 630. In one possible implementation, the processor 680 is an AI processor that supports data processing in the field of AI.

[0079] When the data processing device is a server, the structure of the data processing device can be as follows: Figure 7 As shown, it includes a memory 701 , a communication module 703 and one or more processors 702 .

[0080] The memory 701 is used to store computer programs executed by the processor 702. The memory 701 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0081] The memory 701 may be a volatile memory, such as a random-access memory (RAM); the memory 701 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 701 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 701 may be a combination of the above memories.

[0082] The processor 702 may include one or more central processing units (CPUs) or digital processing modules, etc. The processor 702 is used to implement the data processing method provided in the embodiment of the present application when calling the computer program stored in the memory 701. In one possible implementation, the processor 702 is an AI processor that supports data processing in the AI ​​field.

[0083] The communication module 703 is used to communicate with terminal devices and other servers.

[0084] The specific connection medium between the memory 701, the communication module 703 and the processor 702 is not limited in the embodiment of the present application. Figure 7 In the embodiment, the memory 701 and the processor 702 are connected via a bus 704. Figure 7 The connections between the other components are only for illustration and are not intended to be limiting. The bus 704 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 7 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0085] The memory 701 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the data processing method provided in the embodiment of the present application. The processor 702 is used to execute the above data processing method.

[0086] It should be noted that processing data in the AI ​​field by means of matrix operations is only an exemplary implementation method. The data processing device and the corresponding data processing method provided in the embodiments of the present application are not limited to data in the AI ​​field, but can also be applied to other fields.

[0087] Based on the above application scenarios, the data processing device provided by the exemplary embodiment of the present application is described below in combination with the application scenarios described above according to the accompanying drawings. It should be noted that the above application scenarios are only shown for the purpose of facilitating the understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. In addition, although the operations of the present application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0088] In an embodiment of the present application, the data processing device includes: at least two processing modules distributed in an array; each processing module includes: a beat register and at least two operation units.

[0089] For example, see Figure 8 , Figure 8 A schematic diagram of a data processing device provided in an embodiment of the present application; wherein: Figure 8 Take the data processing device as an example, which includes 16 processing modules 800 distributed in a 4×4 array, and each processing module 800 includes a beat register 8001 and 4 operation units 8002. That is, Figure 8 The data processing device shown includes 4 rows of processing modules, and each row includes 4 processing modules.

[0090] It should be noted that Figure 8 The structure shown is only an exemplary description, wherein the number of rows and columns of the processing modules distributed in an array can be adjusted according to actual applications, and the number of computing units included in each processing module can also be adjusted according to actual applications.

[0091] Since the operations performed by each processing module 800 are the same, Figure 8 Taking a processing module 800 shown in FIG. 1 as an example, the process of data processing by the processing module is described; wherein:

[0092] The beat register 8001 is used to perform delay processing on the data to be processed, obtain the target data after the delay processing, and broadcast the target data to the four operation units 8002 included in the processing module where the beat register 8001 is located, and transmit the target data to the beat register in the next processing module located in the same row as the processing module where the beat register 8001 is located;

[0093] The operation unit 8002 is used to process the target data and transmit the obtained first processing result to the next target unit located in the same column as the operation unit 8002.

[0094] It should be noted that the operation unit 8002 may be a dot product unit (DPU); when the operation unit is a click unit, the processing module is a DPU cluster.

[0095] In the present application, data is transmitted to the operation unit within the processing module by broadcasting. The operation units within the processing module are independent of each other and do not need to be isolated by beat registers. Therefore, the number of beat registers is reduced, and the data processing delay caused by the beat registers is avoided, thereby improving data processing efficiency. At the same time, on the basis of reducing the number of beat registers, the number of beat registers deployed on the chip is simultaneously reduced, thereby reducing chip area consumption. For the operation units between the processing modules, the data is beat by the beat register before entering the operation units in the two adjacent processing modules, that is, the data is transmitted to the operation units in the next adjacent processing module in the row by a pulsating transmission method, that is, the data is beat delayed once every certain distance to reduce the amount of synchronous data transmission and avoid congestion caused by simultaneous data transmission. On the basis of reducing the amount of synchronous data transmission, the data bus bandwidth can be reduced, the complexity of the data processing path can be reduced, and the convergence difficulty of the chip back-end layout and wiring can be further alleviated, making the layout and wiring of the chip back-end more convenient. Therefore, a combination of broadcast and pulsation is adopted to reduce the consumption of beat registers and further reduce chip area consumption, which is beneficial to the back-end layout and wiring of the chip, while reducing data processing delays and improving data processing efficiency, thereby improving the area efficiency.

[0096] In the embodiment of the present application, the data processing device further includes: a full addition unit 801, which is located below the last row of processing modules in the at least two processing modules 800 distributed in an array. Fig. 9 , Fig. 9 A schematic diagram of another data processing device provided in an embodiment of the present application; illustratively, for the last row of processing modules, a full addition unit 801 is provided under each operation unit in the processing module.

[0097] It should be noted that Fig. 9 The data processing device shown is only an exemplary description, and can be adjusted according to actual conditions during application deployment; for example, the full addition unit can be used as part of the last row of processing modules, or the full addition unit can be used as part of the last row of computing units.

[0098] based on Fig. 9 It can be seen from the given data processing device that the position of the operation unit currently performing data processing in the array distribution determines whether the output result of the operation unit currently performing data processing is transmitted to the operation unit in the next processing module in the same column or to the full addition unit.

[0099] In one possible implementation, when the processing module where the computing unit currently performing data processing is located is located in a non-last row of the array distribution, that is, the computing unit currently performing data processing is not the last row, the target unit is the computing unit corresponding to the next processing module in the same column as the computing unit currently performing data processing; that is, the computing unit currently performing data processing transmits the first processing result obtained by data processing to the next computing unit in the same column as the computing unit currently performing data processing.

[0100] For example, Figure 8 The data processing device shown is illustrated, assuming that the operation unit currently performing data processing is the first operation unit in the processing module in the third row and first column. Therefore, after the operation unit currently performing data processing processes the target data and obtains the corresponding first processing result, the first processing result is transmitted to the first operation unit in the processing module in the fourth row and first column.

[0101] In another possible implementation, when the processing module where the operation unit currently performing data processing is located is located in the last row of the array distribution, that is, the operation unit currently performing data processing is in the last row, the target unit is a full addition unit; that is, the operation unit currently performing data processing transmits the first processing result obtained by the data processing to the full addition unit that is currently located in the same column as the operation unit currently performing data processing.

[0102] For example, Figure 8 The data processing device shown is illustrated, assuming that the operation unit currently performing data processing is the first operation unit in the processing module of the fourth row and the first column. Therefore, after the operation unit currently performing data processing processes the target data and obtains the corresponding first processing result, the first processing result is transmitted to the full addition unit connected to the first operation unit in the processing module of the fourth row and the first column.

[0103] In a possible implementation, the full addition unit 801 includes a full adder 8011 with two inputs and one output; wherein:

[0104] The full adder 8011 is used to perform addition processing on the carry result (carry) and the sum result (sum) in the first processing result.

[0105] In the present application, a two-input single-output full adder is set for the operation units in the bottom column of the array distribution to complete the addition of carry results and sum results. Except for the operation units outside the last column, other operation units do not need to be equipped with full adders, which greatly reduces path delay and resource consumption.

[0106] In the embodiment of the present application, since the structure of each processing module 800 is consistent and the operations performed by each processing module 800 are consistent, the operations performed by the operation unit 8002 contained in each processing module 800 are consistent; therefore, the process of data processing by the operation unit 8002 is described in detail below, taking one operation unit 8002 as an example.

[0107] See also Fig.10 , Fig.10 A structural schematic diagram of an operation unit provided in an embodiment of the present application, the operation unit 8002 includes: a multiplication and addition sub-unit 8021 and an accumulation sub-unit 8022; exemplarily, the output end of the multiplication and addition sub-unit 8021 is connected to the input end of the accumulation sub-unit 8022, and the input end of the multiplication and addition sub-unit 8021 is connected to the output end of the beat register 8001.

[0108] In a possible implementation, the beat register 8001 is used to delay the data to be processed to obtain the target data after the delay processing; wherein the data to be processed includes at least two groups of sub-data, and each group of sub-data includes two sub-data.

[0109] Since the beat register 8001 only delays the data and does not perform any operation on the data itself, that is, the data itself does not change after the delay processing, the target data contains at least two groups of sub-data, and each group of sub-data contains two sub-data.

[0110] In a possible implementation, the multiplication-addition subunit 8021 is used to perform a multiplication operation on two sub-data in each group of sub-data respectively to obtain a multiplication result corresponding to each group of sub-data, and to perform an addition operation on the at least two obtained multiplication results to obtain a multiplication-addition result;

[0111] The multiplication and addition result is in the form of retaining the carry, that is, the output of the multiplication and addition subunit 8021 has two results, namely the carry result and the sum result; that is, the multiplication and addition result includes the carry result and the sum result.

[0112] Exemplarily, the target data includes 4 groups of sub-data, each group of sub-data includes two sub-data, at this time, the multiplication and addition sub-unit performs a multiplication operation on the two sub-data in the first group of sub-data to obtain the multiplication result corresponding to the first group of sub-data, and performs a multiplication operation on the two sub-data in the second group of sub-data to obtain the multiplication result corresponding to the second sub-data, and so on, to obtain the multiplication result corresponding to the third group of sub-data and the multiplication result corresponding to the fourth group of sub-data. Afterwards, the obtained four multiplication results are added to obtain the multiplication and addition result.

[0113] In a possible implementation, the accumulation subunit 8022 is used to add the multiplication and addition result input by the multiplication and addition subunit 8021 and the second processing result to obtain the first processing result;

[0114] The second processing result is the first processing result output by the previous operation unit located in the same column as the operation unit 8002 where the accumulation subunit 8022 is located.

[0115] The first processing result is in the form of retaining the carry, that is, the output of the accumulation subunit 8022 has two results, namely the carry result and the sum result; that is, the first processing result includes the carry result and the sum result.

[0116] In the embodiment of the present application, the multiplication and addition subunit 8021 includes: at least two multipliers (mul) 80211, and a tree adder (adder tree) 80212. Fig.11 , Fig.11 A schematic diagram of the structure of a multiplication-addition subunit provided in an embodiment of the present application; illustratively, the multiplication-addition subunit 8021 includes four multipliers 80211 and a tree adder 80212. It should be noted that: Fig.11 The number of multipliers included in the multiplication-addition subunit shown in the figure is only for example and can be adjusted according to actual scenarios in actual applications.

[0117] In a possible implementation, a multiplier 80211 in the multiplication-addition subunit is used to perform a multiplication operation on two sub-data in a group of sub-data in the target data to obtain a corresponding multiplication result;

[0118] Among them, one multiplier corresponds one to one with a group of sub-data, one multiplier only processes one group of sub-data, and the multiplication result is in the form of retaining the carry; that is, the output of the multiplier has two results, namely the carry result and the sum result; that is, the multiplication result includes the carry result and the sum result.

[0119] Therefore, in order to ensure the utilization rate of the multipliers and improve the data processing efficiency, in the embodiment of the present application, the number of sub-data groups in the data to be processed is consistent with the number of multipliers included in the multiplication and addition sub-unit. Fig.11 For the multiplication-addition subunit shown, the data to be processed should contain 4 groups of sub-data.

[0120] It should be noted that the data to be processed corresponding to each operation unit and the number of sub-data groups in the data to be processed are determined based on the structure of the data processing device provided in the embodiment of the present application.

[0121] In one possible implementation, after obtaining the benchmark data, the benchmark data is grouped based on the number of rows of processing units in the data processing device and the number of multipliers contained in the operation unit to obtain at least one data to be processed; wherein the benchmark data is all data input into the data processing device for data processing.

[0122] Specifically, based on the number of rows of array distribution and the number of multipliers in the operation unit, the obtained reference data is grouped and processed to obtain at least one to-be-processed data consistent with the number of rows; wherein the number of sub-data groups in each to-be-processed data is consistent with the number of multipliers.

[0123] In the present application, multiple data to be processed that are consistent with the number of processing module rows are obtained to ensure that there is data to be processed input in each row of processing modules, and the data to be processed contains multiple groups of sub-data that are consistent with the number of multipliers. Each multiplier processes a group of sub-data, and multiple multipliers synchronously process multiple groups of sub-data, thereby ensuring data processing efficiency and utilization of processing modules in the data processing device.

[0124] After obtaining at least one data to be processed, the at least one data to be processed is input into the array-distributed processing module for data processing. Since the data processing device adopts the horizontal and vertical dimensional spring pulsation beat mode, in order to match this timing, it is necessary to perform triangulation delay processing on the at least one data to be processed obtained after the reference data is grouped, so as to enter the array distribution of the data processing device according to the first-in-first-out rule; due to the characteristics of triangulation delay processing, it is known that for each additional group of data to be processed, one more delay operation will be performed, that is, the difference between two adjacent groups of data to be processed is one cycle.

[0125] For example, taking the reference data as 16 horizontal sub-data × 16 vertical sub-data, based on Figure 8 to Figure 15 The determined data processing device groups the reference data, from Figure 8 to Figure 15 It can be seen that the data processing device includes 4 row processing modules, each processing module includes 4 multipliers, and the reference data is divided into 4 groups to obtain 4 groups of data to be processed, each group of data to be processed includes 4 horizontal sub-data×4 vertical sub-data, that is, each group of data to be processed includes 4 groups of sub-data, and each group of sub-data includes one horizontal sub-data and one vertical sub-data; for example, data to be processed 1 includes horizontal sub-data (0-3) and vertical sub-data (0-3), data to be processed 2 includes horizontal sub-data (4-7) and vertical sub-data (4-7), data to be processed 3 includes horizontal sub-data (8-11) and vertical sub-data (8-11), and data to be processed 4 includes horizontal sub-data (12-15) and vertical sub-data (12-15);

[0126] After obtaining 4 sets of data to be processed, when performing triangulation delay processing on the 4 sets of data to be processed, the difference between data to be processed 1 and data to be processed 2 is one cycle, the difference between data to be processed 2 and data to be processed 3 is one cycle, and the difference between data to be processed 3 and data to be processed 4 is one cycle; and when performing triangulation delay processing on the 4 sets of data to be processed, both the horizontal sub-data and the vertical sub-data need to be triangulated and delayed; see Fig.16 , is a schematic diagram of a triangulated delay provided in an embodiment of the present application, taking the horizontal sub-data (0~16) as an example: on the basis of not performing beat delay on the processed data 1, the horizontal sub-data (0~3) does not need to pass through the register to be directly input into the array distribution, and at this time, it is necessary to perform a one-beat delay on the processed data 2, that is, the horizontal sub-data (4~7) each passes through a register to perform data beat, and one register beats one beat; it is necessary to perform a two-beat delay on the processed data 3, that is, the horizontal sub-data (8~11) each passes through two registers to perform data beat; it is necessary to perform a three-beat delay on the processed data 4, that is, the horizontal sub-data (12~15) each passes through three registers to perform data beat.

[0127] Therefore, in an embodiment of the present application, the timing of all sub-data in each data to be processed is consistent, and two adjacent data to be processed differ by one processing cycle, that is, the data to be processed corresponding to two adjacent processing modules in the same column differ by one processing cycle; wherein the difference in processing cycles is used to process the accumulation operation of the first processing result output by the previous operation unit and the multiplication and addition result obtained by the current operation unit.

[0128] In the present application, broadcasting is used for data transmission in each processing module. At this time, the data input into the processing module is of the same timing, which reduces the resource consumption of triangulation delay first-in-first-out.

[0129] In a possible implementation, the multiplier adopts an incomplete multiplier, that is, the multiplication result output by the multiplier is not the final multiplication result, but is in a carry-preserving form to reduce the delay of a full adder.

[0130] In a possible implementation, the tree adder 80212 is used to perform an addition operation on the multiplication results output by all the multipliers 80211 in the multiplication-addition subunit 8021 to obtain a multiplication-addition result;

[0131] Among them, the multiplication and addition results are in the form of retaining the carry; that is, the output of the tree adder has two results, namely the carry result and the sum result.

[0132] In the embodiment of the present application, the tree adder 80212 includes at least two stages of adders.

[0133] In a possible implementation, the adder may be at least one of a three-input and two-output carry-save adder (CSA) and a two-input and one-output full adder.

[0134] Preferably, in order to reduce data processing delay, the adder in the tree adder 80212 uses a three-input and two-output carry-preserving adder; and the number of levels of the three-input and two-output carry-preserving adder in the tree adder 80212 is determined by the number of multiplication results output by the multiplier in the multiplication-addition subunit. For example, if the input is 4 data, the tree adder is set with two levels of carry-preserving adders; if the input is 6 data, the tree adder is set with three levels of carry-preserving adders, and so on.

[0135] For example, Fig.11 The four multipliers 80211 included in the multiplication and addition subunit 8021 are matched, and the tree adder 80212 in the multiplication and addition subunit 8021 is set as a four-stage carry-preserving adder, see Fig.12 , Fig.12 A schematic diagram of a tree adder provided in an embodiment of the present application. The input of the tree adder 80212 is all the data output by the four multipliers, that is, the input 8 data are in0 to in7, and the output is out0 and out1.

[0136] In the present application, the number of levels of carry-retaining adders in the tree adder is determined according to the amount of data input into the tree adder to improve data processing efficiency and reduce data processing delay.

[0137] In a possible implementation, the multiplication-addition subunit 8021 further includes: a shift register 80213 matching the number of multipliers, and an alignment shift (asft) 80214 matching the number of multipliers. Fig.13 , Fig.13 A schematic diagram of a specific structure of a multiplication and addition subunit provided in an embodiment of the present application; exemplarily, Fig.11 Based on, when the multiplication and addition subunit 8021 includes 4 multipliers, the multiplication and addition subunit 8021 also includes 4 shift registers and 4 docking shifters; wherein:

[0138] The output end of each multiplier 80211 is connected to the input end of a shift register 80213 , the output end of a shift register 80213 is connected to the input end of a phase shifter 80214 , and the output end of the phase shifter 80214 is connected to the input end of the accumulation subunit 8022 .

[0139] In one possible implementation, a multiplication result output by a multiplier 80211 is delayed by a shift register 80213, and the delayed multiplication result is input into a phasor shifter 80214, and a phasor shift is performed on the delayed multiplication result output by a shift register 80213 by a phasor shifter 80214.

[0140] It should be noted that the role of the order shift is to perform a mantissa alignment operation according to the exponent of the floating-point number.

[0141] In the embodiment of the present application, the accumulation subunit 8022 includes: two delay registers 80221, two phase shifters 80222 and a compressor 80223; see Fig.14 , Fig.14 A schematic diagram of the structure of an accumulation subunit provided in an embodiment of the present application; wherein:

[0142] The input end of one of the two delay registers 80221 is connected to the output end of the multiplication-addition subunit 8021, and is used to delay the multiplication-addition result output by the multiplication-addition subunit 8021. The output end of the other delay register 80221 is connected to the output end of the previous operation unit in the same column as the operation unit to which the accumulation subunit 8022 belongs, and is used to delay the second processing result, and the second processing result is the first processing result output by the previous operation unit.

[0143] The output ends of the two delay registers 80221 are respectively connected to the input ends of the two step shifters 80222, wherein the output end of one delay register 80221 is connected to the input end of one step shifter 80222; at this time, the two step shifters respectively perform step shift processing on the multiplication and addition result after the delay processing, and perform step shift processing on the second processing result after the delay processing;

[0144] The output ends of the two order shifters 80222 are connected to the input end of an adder 80223; at this time, the adder adds the multiplication and addition result after the order shift processing and the second processing result after the order shift processing to obtain a first processing result.

[0145] In one possible implementation, the compressor 80223 is a four-input and two-output carry-preserving adder (4-2CSA); wherein the compressor 80223 includes two stages of 3-2CSA; the first processing result is in the form of carry-preserving, that is, the output of the accumulation subunit 8022 has two results, namely, a carry result and a sum result; that is, the first processing result includes a carry result and a sum result.

[0146] In the present application, after the output result of the multiplication and addition sub-unit is clocked by a delay register, it is accumulated with the first processing result output by the upper operation unit. The first processing result obtained by accumulation adopts a carry-retaining method, that is, the carry result and the sum result are output, thereby avoiding the delay of the full adder.

[0147] In summary, the present application embodiment provides a specific structural diagram of a computing unit, see Fig.15 ,in Fig.15 The processing of data by each component involved in the process is described above and will not be repeated here.

[0148] In an embodiment of the present application, the data processing device can be applied to image processing scenarios in the AI ​​field. At this time, each group of sub-data in the target data contains two sub-data: weight (weight, wt) sub-data and feature map (featuremap, fm) sub-data.

[0149] Below, taking data processing of weight sub-data and feature map sub-data as an example, the beneficial effects brought about by the data processing device provided in the embodiment of the present application are discussed in detail.

[0150] Assume that the feature map (fm) data is fm(0~15) and the weight (wt) is wt(0~15).

[0151] The feature map data and weights are processed using the implementation method in the relevant technology:

[0152] A 16x16 systolic array will be used, see the attached Figure 3 In the 16x16 systolic array, there is a register between the row-wise operation units, and the total number of registers set is 16×16; the triangulation delay of the feature map data and the triangulation delay of the weight refer to Figure 2 During the triangulation delay processing, 120 registers need to be set for delay processing of the feature map data, and 120 registers need to be set for delay processing of the weight data; at this time, from the first feature map data and the first weight data entering the 16×16 systolic array to the last operation unit in the systolic array outputting the processing result, it takes (16+16+delay) clock cycles, where delay is the calculation delay, and consumes a large number of registers.

[0153] Using the implementation method provided in the embodiment of the present application, the feature map data and weights are processed:

[0154] Corresponding to the above-mentioned 16×16 systolic array, the data processing device is configured as a processing module distributed in a 4×4 array, including a total of 16 processing modules. Each processing module includes 4 computing units, and each computing unit is divided into 4 paths. Therefore, the broadcast granularity of the data to be processed in a processing module is 4×4.

[0155] Due to the use of a 4x4 broadcast granularity, the clock cycle for the pulsating propagation of data from left to right is n / 4, where n is the pulsating array width, and in the embodiment of the present application, n=16, and the clock cycle for the pulsating propagation of the accumulated data from top to bottom is k / 4, where k is the pulsating array height, and in the embodiment of the present application, k=16. From the first feature map data and the first weight data entering the data processing device provided in the embodiment of the present application to the last operation unit in the data processing device outputting the processing result, (n+k) / 4+delay, i.e. (16+16) / 4+delay clock cycles are required, where delay is the calculation delay. It can be seen that compared with the processing method in the related art, the implementation method of the present application reduces the data processing delay.

[0156] When the broadcast granularity of 4x4 is adopted, the input data of each processing module is 4, and the processing module is used to process the dot product operation of 4 inputs. The data is transmitted in the processing module by broadcasting. Since this broadcast is local, the layout and wiring overhead is relatively small; the processing modules are isolated by register beats, which greatly reduces the consumption of registers. Compared with the full pulse method in the related art, the combination of broadcast and pulse consumes 1 / 4 of the beat registers in the full pulse method in the related art.

[0157] Since the broadcast operation is performed at a 4x4 granularity, every 4 feature map data are input into a processing module as a group, and the input timing of the 4 feature map data input into the processing module is exactly the same, so there is no need to perform a beat operation between the 4 feature map data; in two adjacent rows of processing modules, the timing of a processing module in one row and its downstream processing module differs by one clock cycle, so the feature map data corresponding to a processing module in one row needs to be delayed by one beat compared to the feature map data corresponding to the processing module in the previous row, and therefore the feature map data input into processing modules in different rows need to be beat processed.

[0158] Exemplarily, fm(0-3) is input into the processing module in the first row, fm(4-7) is input into the processing module in the second row, fm(8-11) is input into the processing module in the third row, and fm(12-15) is input into the processing module in the fourth row.

[0159] Before the feature map data enters the array distribution processing module, it needs to be triangulated and delayed. For details, see Fig.16At this time, there is no need to perform beat processing between fm(0-3), but there is a clock cycle difference between fm(0-3) and fm(4-7), so a beat is required between these two groups of signals; there is a clock cycle difference between fm(4-7) and fm(8-11), so a beat is required between these two groups of signals; there is a clock cycle difference between fm(8-11) and fm(12-15), so a beat is required between these two groups of signals. Therefore, when performing triangular delay processing on feature map data, a total of 24 registers are required, which greatly reduces the consumption of registers compared to the prior art.

[0160] Below, the implementation method of the embodiment of the present application is described in detail by taking the processing module in the first row and the first column performing data processing on fm(0-3) and wt(0-3) as an example. Fig.17 , Fig.17 A schematic diagram of feature graph data processing provided in an embodiment of the present application.

[0161] For a computing unit in a processing module, the computing unit performs the following operations:

[0162] The delayed fm(0-3) and wt(0-3) are processed by four multipliers. The output of each multiplier has two results, namely carry and sum. The multiplier here uses an incomplete multiplier, that is, the output result is not the final multiplication result, but the form of retaining the carry, which can reduce the delay of a full adder. Among them, the first row of multipliers processes fm(0) and wt(0), the first row of multipliers processes fm(1) and wt(1), the third row of multipliers processes fm(2) and wt(2), and the fourth row of multipliers processes fm(4) and wt(4).

[0163] In the embodiment of the present application, since the weight data in each operation unit is different, the weight data is initially loaded, that is, the weight data needs to be prepared first, and the feature map data can enter the operation unit to perform normal calculations. Due to the characteristics of the convolution operation, the weight data needs to perform a dot product operation with multiple feature map data, so the weight data can be resident in the array distribution.

[0164] The outputs of the four multipliers are delayed by four delay registers, and the delayed data are input to four order shifters for order shifting.

[0165] The result after the order shift processing is sent to the tree adder, and the result after the order shift processing is processed by the tree adder; at this time, there are 4 inputs, each input includes two results, carry and sum, and a total of 8 groups of data. That is, the input is in0~7, and the output is out0 and out1. For details, please refer to the attached Fig.12 ,The tree adder is composed of 3-2 CSAs, and the carry-preserving adder will preserve the propagation of the carry bit, and has lower latency compared to the full adder.

[0166] After the output of the tree adder is tapped, it is accumulated with the output of the previous operation unit in the same row. To avoid delay, the output of the operation unit still uses the carry-retaining method to output carry and sum. Before accumulation, the output of the tree adder and the output of the previous operation unit in the same row need to be sent to asft for a shift operation, and then sent to 4-2CSA for a carry-retaining addition operation, and the output is carry and sum.

[0167] In the present application, under the operation unit in the bottom column of the array distribution, a two-input full adder needs to be instantiated to complete the addition of carry and sum. Except for the operation units in the last column, there is no need to instantiate full adders, which greatly reduces path delay and resource consumption.

[0168] Based on the same inventive concept, the present application also provides a data processing method, which is applied to the data processing device provided in the present application. Fig.18 , Fig.18 A data processing method flow chart provided in an embodiment of the present application includes the following steps:

[0169] Step S1800, delay processing is performed on the data to be processed through a beat register in a processing module to obtain target data after delayed processing, and the target data is broadcast to at least two computing units included in a processing module, and the target data is transmitted to the beat register in the next processing module located in the same row as a processing module.

[0170] In a possible implementation, the target data includes at least two groups of sub-data, each group of sub-data includes two sub-data; and the target data is processed by a computing unit in a processing module, including:

[0171] Through the multiplication and addition sub-unit in the operation unit, two sub-data in each group of sub-data are multiplied to obtain the multiplication results corresponding to each group of sub-data, and at least two multiplication results are added to obtain the multiplication and addition result; wherein the multiplication and addition result is in the form of retaining the carry, and the multiplication and addition result includes the carry result and the sum result;

[0172] The multiplication and addition result is added to the second processing result through the accumulation subunit in the operation unit to obtain the first processing result; wherein the second processing result is the first processing result output by the previous operation unit located in the same column as the operation unit, and the first processing result is in the form of retaining the carry, and the first processing result includes the carry result and the sum result.

[0173] In a possible implementation, the multiplication-addition subunit includes: at least two multipliers, and a tree adder; the number of sub-data groups in the target data is consistent with the number of multipliers in the multiplication-addition subunit, and each sub-data group includes weight sub-data and feature map sub-data; wherein:

[0174] The multiplication and addition subunit in the operation unit performs multiplication and addition processing on the target data to obtain a corresponding multiplication and addition result, including:

[0175] Through a multiplier, a multiplication operation is performed on the weight sub-data and the feature map sub-data in a sub-data in the target data to obtain a corresponding multiplication result; wherein the multiplication result is in a carry-retaining form, and the multiplication result includes a carry result and a sum result;

[0176] All multiplication results output by the at least two multipliers are added together by the tree adder to obtain the multiplication-addition result; wherein the tree adder includes at least two levels of carry-preserved adders, and the carry-preserved adders are three-input and two-output adders.

[0177] In a possible implementation, the multiplication-addition subunit further includes: a shift register matching the number of multipliers, and a pair-order shifter matching the number of multipliers;

[0178] After obtaining the corresponding multiplication result, before adding all the multiplication results output by at least two multipliers through the tree adder, the method further includes:

[0179] The multiplication result output by a multiplier is delayed by a shift register;

[0180] A multiplication result after delay processing outputted from a shift register is subjected to a phase shift process by a phase shifter.

[0181] In a possible implementation, the multiplication-addition result and the second processing result are added by an accumulation subunit in the operation unit to obtain the first processing result, including:

[0182] Delaying the multiplication and addition result through a delay register in the accumulation subunit, and delaying the second processing result through another delay register in the accumulation subunit;

[0183] Performing a pair of order shifters in the accumulation subunit to perform a pair of order shifters on the delayed multiplication and addition result, and performing a pair of order shifters in the accumulation subunit to perform a pair of order shifters on the delayed second processing result;

[0184] The compressor in the accumulation subunit performs addition processing on the multiplication-addition result after the order shift processing and the second processing result after the order shift processing to obtain the first processing result.

[0185] In a possible implementation, before delaying the data to be processed by using a beat register in a processing module, the method further includes:

[0186] Determine the number of rows included in the array distribution corresponding to a processing module and the number of multipliers included in the computing unit in a processing module;

[0187] Based on the number of rows and the number of multipliers, the obtained benchmark data are grouped and processed to obtain at least one data to be processed that is consistent with the number of rows; wherein the number of sub-data in each data to be processed is consistent with the number of multipliers, and the timing of the sub-data in each data to be processed is consistent, and the data to be processed corresponding to two adjacent processing modules in the same column differ by one processing cycle.

[0188] Step S1801, processing target data through a computing unit in a processing module, and transmitting the obtained first processing result to the next target unit located in the same column as the computing unit.

[0189] In a possible implementation, if a processing module is located in a non-last row in the array distribution, the target unit is a corresponding computing unit in the next processing module located in the same column as the computing unit;

[0190] If a processing module is located in the last row in the array layout, the target unit is the full addition unit located in the same column as the operation unit.

[0191] In a possible implementation manner, after the first processing result is transmitted to a full addition unit located in the same column as the operation unit, the method further includes:

[0192] The carry result and the sum result in the first processing result are added by the full adder in the full addition unit.

[0193] See also Fig.19 , Fig.19 A flowchart of a specific implementation method of data processing provided in an embodiment of the present application includes the following steps:

[0194] Step S1900, obtaining reference data, wherein the reference data is all data input into the data processing device;

[0195] Step S1901, determining the number of rows included in the array distribution in the data processing device and the number of multipliers in an operation unit;

[0196] Step S1902, based on the number of rows and the number of multipliers, the obtained reference data is grouped and processed to obtain at least one to-be-processed data that is consistent with the number of rows; the number of sub-data in each to-be-processed data is consistent with the number of multipliers;

[0197] Step S1903, inputting at least one to-be-processed data obtained by grouping into a processing module distributed in an array according to a first-in first-out rule;

[0198] Step S1904, performing delay processing on a to-be-processed data to obtain target data after delay processing, and broadcasting the target data to at least two computing units included in a processing module;

[0199] Step S1905, performing multiplication operations on each sub-data in the target data to obtain corresponding multiplication results;

[0200] Step S1906, performing delay processing on each multiplication result;

[0201] Step S1907, performing order shift processing on each delayed multiplication result;

[0202] Step S1908, adding all the multiplication results after the order shifting process to obtain a multiplication-addition result;

[0203] Step S1909, delay processing is performed on the multiplication and addition result, and delay processing is performed on the second processing result; the second processing result is the first processing result output by the previous operation unit;

[0204] Step S1910, performing a rank shift process on the delayed multiplication and addition result, and performing a rank shift process on the delayed second processing result;

[0205] Step S1911, adding the multiplication and addition result after the order shifting process and the second processing result after the order shifting process to obtain a first processing result;

[0206] Step S1912, determining whether the processing module currently performing data processing is located in the last row of the array distribution, if so, executing step S1913, otherwise executing step S1914;

[0207] Step S1913, transmitting the obtained first processing result to the next full addition unit in the same column, and performing addition processing on the carry result and the sum result in the first processing result through the full adder in the full addition unit;

[0208] Step S1914: transmitting the obtained first processing result to the next target unit in the same column.

[0209] It should be noted that steps S1904 to S1911 are executed by a computing unit in the processing module.

[0210] In the present application, data is transmitted to the operation unit in the processing module by broadcasting. The operation units in the processing module are independent of each other and do not need to be isolated by beat registers. Therefore, the number of beat registers is reduced and the data processing delay caused by the beat registers is avoided, thereby improving data processing efficiency. At the same time, on the basis of reducing the number of beat registers, the number of beat registers deployed on the chip is simultaneously reduced, thereby reducing chip area consumption. For the operation units between the processing modules, the data is beat by the beat register before entering the operation units in the two adjacent processing modules, that is, the data is transmitted to the operation units in the next adjacent processing module in the row by a pulsating transmission method, that is, the data is beat delayed once every certain distance to reduce the amount of synchronous data transmission and avoid congestion caused by simultaneous data transmission. On the basis of reducing the amount of synchronous data transmission, the data bus bandwidth can be reduced, the complexity of the data processing path can be reduced, and the convergence difficulty of the chip back-end layout and wiring can be further alleviated, making the layout and wiring of the chip back-end more convenient. Therefore, a combination of broadcast and pulsation is adopted to reduce the consumption of beat registers and further reduce chip area consumption, which is beneficial to the back-end layout and wiring of the chip, while reducing data processing delays and improving data processing efficiency, thereby improving the area efficiency.

[0211] After introducing the data processing apparatus and method according to the exemplary embodiment of the present application, another exemplary embodiment of the computing device according to the present application is introduced next.

[0212] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0213] In a possible implementation, the computing device provided in the embodiment of the present application may include at least a processor and a memory, wherein the memory stores a program code, and when the program code is executed by the processor, the processor executes any step of the data processing method of various exemplary embodiments in the present application.

[0214] It should be noted that the computing device provided in the embodiment of the present application may be the data processing device proposed above, that is, it includes a terminal device and a server. Figure 6 and attached Figure 7 , I will not repeat them here.

[0215] In some possible implementations, the embodiments of the present application further provide a chip, which includes the data processing device provided by the embodiments of the present application, and is used to implement the steps of the data processing method provided by the embodiments of the present application.

[0216] In some possible implementations, an embodiment of the present application provides a chip system, which includes a processor for supporting a computing device to implement the functions involved in the above embodiments.

[0217] In a possible implementation, the chip system further includes a memory, the memory is used to store the computer program and data necessary for the computing device, and the processor is used to call and run the computer program from the memory, so that the computing device equipped with the chip system executes the data processing method in the embodiment of the present application. The chip system can be composed of chips, or it can include chips and other discrete devices.

[0218] It should be noted that although several units (or modules) of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units (or modules) described above can be concretized in one unit (or module). Conversely, the features and functions of one unit (or module) described above can be further divided into multiple units (or modules) for concretization. Of course, when implementing the present application, the functions of each unit (or module) can also be implemented in the same one or more software or hardware.

[0219] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0220] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program commands. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0221] These computer program commands may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the commands stored in the computer readable memory produce an article of manufacture comprising a command device, the command device implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0222] These computer program commands can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the commands executed on the computer or other programmable device provide the instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0223] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0224] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing device, characterized in that: The device comprises: at least two processing modules distributed in an array; each processing module comprises: a beat register and at least two operation units; wherein: A beat register in a processing module, used to perform delay processing on the data to be processed, obtain target data after the delay processing, broadcast the target data to at least two computing units included in the processing module, and transmit the target data to the beat register in the next processing module located in the same row as the processing module; The computing unit in the one processing module is used to process the target data and transmit the obtained first processing result to the next target unit located in the same column as the computing unit.

2. The device according to claim 1, characterized in that The target data includes at least two groups of sub-data, each group of sub-data includes two sub-data; the operation unit includes: a multiplication and addition sub-unit and an accumulation sub-unit; The multiplication-addition subunit is used to perform multiplication operations on two sub-data in each group of sub-data respectively to obtain multiplication results corresponding to each group of sub-data, and to add at least two obtained multiplication results to obtain a multiplication-addition result; wherein the multiplication-addition result is in the form of retaining carry, and the multiplication-addition result includes a carry result and a sum result; The accumulation subunit is used to add the multiplication and addition result and the second processing result to obtain the first processing result; wherein the second processing result is the first processing result output by the previous operation unit located in the same column as the operation unit, and the first processing result is in the form of retaining the carry, and the first processing result includes the carry result and the sum result.

3. The device according to claim 2, characterized in that The multiplication-addition subunit includes: at least two multipliers and a tree adder; the number of sub-data groups in the target data is consistent with the number of multipliers in the multiplication-addition subunit, and each group of sub-data includes weight sub-data and feature map sub-data; wherein: A multiplier, used to perform a multiplication operation on the weight sub-data and the feature map sub-data in a set of sub-data to obtain a corresponding multiplication result; wherein the multiplication result is in a carry-retaining form, and the multiplication result includes a carry result and a sum result; The tree adder is used to perform an addition operation on all multiplication results output by the at least two multipliers to obtain the multiplication-addition result.

4. The device according to claim 3, characterized in that The tree adder includes at least two stages of adders.

5. The device according to claim 3, characterized in that The multiplication-addition subunit further includes: a shift register matching the number of multipliers, and a pair-order shifter matching the number of multipliers; a shift register, used for delaying the multiplication result output by the multiplier; A phasor shifter is used to perform phasor shift processing on the delayed multiplication result output by the shift register.

6. The device according to claim 2, characterized in that The accumulation subunit includes: two delay registers, two phase shifters and a compressor; The two delay registers are used to delay processing of the multiplication and addition result and to delay processing of the second processing result respectively; The two rank shifters are used to perform rank shift processing on the multiplication and addition result after the delay processing, and to perform rank shift processing on the second processing result after the delay processing, respectively; The compressor is used to perform addition processing on the multiplication and addition result after the order shift processing and the second processing result after the order shift processing to obtain the first processing result.

7. The device according to any one of claims 1 to 6, characterized in that: The data to be processed corresponding to the one processing module includes sub-data having a number consistent with the number of multipliers in the one operation unit; and the time sequence of the sub-data in the data to be processed corresponding to the one processing module is consistent.

8. The device according to claim 7, characterized in that The difference between the to-be-processed data corresponding to two adjacent processing modules in the same column is one processing cycle.

9. The device according to any one of claims 1 to 6, characterized in that: If the processing module is located in a non-last row in the array distribution, the target unit is a corresponding computing unit in the next processing module located in the same column as the computing unit.

10. The device according to any one of claims 1 to 6, characterized in that: The device further comprises: a full addition unit, the full addition unit being located below the last row of processing modules of the at least two processing modules distributed in the array, the full addition unit comprising a full adder; If the one processing module is located in the last row in the array distribution, the target unit is a full addition unit located in the same column as the operation unit; The full addition unit is used to perform addition processing on the carry result and the sum result in the first processing result through the full adder.

11. A data processing method, characterized in that: Applied to the data processing device according to any one of claims 1 to 10, the method comprises: Performing delay processing on the data to be processed through a beat register in a processing module to obtain target data after the delay processing, broadcasting the target data to at least two computing units included in the processing module, and transmitting the target data to the beat register in the next processing module located in the same row as the processing module; The target data is processed by a computing unit in the one processing module, and a first processing result obtained is transmitted to a next target unit located in the same column as the computing unit.

12. The method according to claim 11, characterized in that The target data includes at least two groups of sub-data, each group of sub-data includes two sub-data; The processing of the target data by a computing unit in the one processing module includes: Through the multiplication and addition sub-unit in the operation unit, two sub-data in each group of sub-data are multiplied to obtain the multiplication results corresponding to each group of sub-data, and at least two multiplication results are added to obtain the multiplication and addition result; wherein the multiplication and addition result is in the form of retaining the carry, and the multiplication and addition result includes the carry result and the sum result; The multiplication and addition result is added to the second processing result through the accumulation subunit in the operation unit to obtain the first processing result; wherein the second processing result is the first processing result output by the previous operation unit located in the same column as the operation unit, and the first processing result is in the form of retaining the carry, and the first processing result includes the carry result and the sum result.

13. The method according to claim 12, characterized in that The multiplication-addition subunit includes: at least two multipliers and a tree adder; the number of sub-data groups in the target data is consistent with the number of multipliers in the multiplication-addition subunit, and each group of sub-data includes weight sub-data and feature map sub-data; wherein: The multiplication and addition subunit in the operation unit performs multiplication and addition processing on the target data to obtain a corresponding multiplication and addition result, including: Through a multiplier, a multiplication operation is performed on the weight sub-data and the feature map sub-data in a sub-data in the target data to obtain a corresponding multiplication result; wherein the multiplication result is in a carry-retaining form, and the multiplication result includes a carry result and a sum result; All multiplication results output by the at least two multipliers are added together by the tree-like adder to obtain the multiplication-addition result; wherein the tree-like adder includes at least two stages of adders.

14. The method according to claim 13, characterized in that The multiplication-addition subunit further includes: a shift register matching the number of multipliers, and a pair-order shifter matching the number of multipliers; After obtaining the corresponding multiplication results, before adding all the multiplication results output by the at least two multipliers through the tree adder, the method further includes: Delaying the multiplication result output by the multiplier through a shift register; A phase shifter is used to perform phase shift processing on the delayed multiplication result output by the shift register.

15. The method according to claim 12, characterized in that The step of adding the multiplication and addition result to the second processing result through the accumulation subunit in the operation unit to obtain the first processing result includes: Delay processing the multiplication and addition result through a delay register in the accumulation subunit, and delay processing the second processing result through another delay register in the accumulation subunit; Performing a pair of order shifters in the accumulation subunit to perform a pair of order shifters on the delayed multiplication and addition result, and performing a pair of order shifters in the accumulation subunit to perform a pair of order shifters on the delayed second processing result; The compressor in the accumulation subunit performs addition processing on the multiplication-addition result after the order shift processing and the second processing result after the order shift processing to obtain the first processing result.

16. The method according to any one of claims 11 to 15, characterized in that: Before delaying the data to be processed by using a beat register in a processing module, the method further includes: Determining the number of rows included in the array distribution corresponding to the one processing module and the number of multipliers included in the computing unit in the one processing module; Based on the number of rows and the number of multipliers, the obtained benchmark data is grouped and processed to obtain at least one data to be processed that is consistent with the number of rows; wherein the number of sub-data in each data to be processed is consistent with the number of multipliers, and the timing of the sub-data in each data to be processed is consistent, and the data to be processed corresponding to two adjacent processing modules in the same column differ by one processing cycle.

17. The method according to any one of claims 11 to 15, characterized in that: If the one processing module is located in a non-last row in the array distribution, the target unit is a corresponding computing unit in the next processing module located in the same column as the computing unit; If the one processing module is located in the last row in the array distribution, the target unit is a full addition unit located in the same column as the operation unit.

18. The method according to claim 17, characterized in that After transmitting the obtained first processing result to the full addition unit located in the same column as the operation unit, the method further includes: The carry result and the sum result in the first processing result are added together by the full adder in the full addition unit.

19. A chip, characterized in that: The chip comprises the data processing device according to any one of claims 1-11.