Method and apparatus for processing neural network feature maps by multiple accelerators

Abstract

The method comprises the following steps: reading first feature data related to a neural network feature map and first weight data from a first shift register array in a first accelerator in a plurality of neural network accelerators; performing a preset operation on the first feature data and the first weight data based on the first accelerator to obtain a first operation result; shifting first overlapping feature data required by a second accelerator in the first feature data to a second shift register array of the second accelerator based on a preset shift rule; and performing a preset operation on second feature data including the first overlapping feature data read from the second shift register array and second weight data read by the second accelerator to obtain a second operation result. The method can realize the reuse of data in the overlapping area while reducing the memory space requirement of the accelerator.

Description

Technical Field

[0001] This disclosure relates to artificial intelligence technology, and in particular to a method and apparatus for processing feature maps of a neural network using multiple accelerators. Background Technology

[0002] As the size of feature maps and computational demands increase in convolutional neural networks, multiple accelerators are often required to collaboratively process the same feature map. Common techniques involve splitting the feature map in at least one of its width and height directions, then processing the resulting sub-feature maps collaboratively through multiple accelerators, such as performing convolution operations. The convolution result is obtained by combining the results from multiple accelerators. However, during convolution operations, the split sub-feature maps often have overlapping regions at their boundaries that need to be reused across multiple accelerators. For the feature data in these overlapping regions, two common approaches are used: one is to store the overlapping region's feature data in the memory of each accelerator, allowing each accelerator to complete its own computational task independently; the other is to store only non-overlapping feature data in the memory of each accelerator core, with the overlapping region's feature data transmitted between accelerators via NOC (Network on Chip) communication. The first approach results in a large memory requirement within the accelerator, while the second approach leads to wasted NOC bandwidth due to data transmission between accelerators. Summary of the Invention

[0003] To address the aforementioned technical problems such as the large storage space requirements within accelerators and the waste of NOC bandwidth for data transmission between accelerators, this disclosure is proposed. Embodiments of this disclosure provide a method and apparatus for processing neural network feature maps using multiple accelerators.

[0004] According to one aspect of the present disclosure, a method for processing a neural network feature map using multiple accelerators is provided, comprising: reading first feature data related to the neural network feature map from a first shift register array in a first accelerator of the multiple neural network accelerators and reading first weight data corresponding to the first feature data from a first buffer in the first accelerator; performing a preset operation on the first feature data and the first weight data based on the first accelerator to obtain a first operation result; shifting first overlapping feature data in the first feature data that is needed by a second accelerator of the multiple neural network accelerators from the first shift register array to a second shift register array of the second accelerator based on a preset shift rule; reading second feature data including the first overlapping feature data from the second shift register array in the second accelerator and reading second weight data corresponding to the second feature data from a second buffer in the second accelerator; and performing a preset operation on the second feature data and the second weight data based on the second accelerator to obtain a second operation result.

[0005] According to another aspect of the present disclosure, an apparatus for processing neural network feature maps using multiple accelerators is provided, comprising: multiple neural network accelerators; each neural network accelerator includes a controller, a shift register array, a buffer, and an operation array for preset operations; for a first accelerator among the multiple neural network accelerators, a first controller in the first accelerator reads first feature data related to the neural network feature map from the first shift register array in the first accelerator and reads first weight data corresponding to the first feature data from the first buffer in the first accelerator; the first controller controls the first operation array in the first accelerator to perform preset operations on the first feature data and the first weight data to obtain a first operation result; the first controller controls the first shift register array based on preset shift rules to shift first overlapping feature data in the first feature data that is needed by a second accelerator among the multiple neural network accelerators to the second shift register array of the second accelerator; a second controller in the second accelerator reads second feature data including the first overlapping feature data from the second shift register array and reads second weight data corresponding to the second feature data from the second buffer in the second accelerator; the second controller controls the second operation array in the second accelerator to perform preset operations on the second feature data and the second weight data to obtain a second operation result.

[0006] Based on the method and apparatus for processing neural network feature maps using multiple accelerators provided in the above embodiments of this disclosure, the data in overlapping regions is shifted by shift register arrays within multiple accelerators. This allows the data in overlapping regions within one accelerator to be shifted to adjacent accelerators, enabling each accelerator to store non-overlapping data separately. This reduces the storage space requirements within the accelerators, achieves data reuse, significantly reduces bandwidth waste in the NOC, lowers power consumption, and improves accelerator performance.

[0007] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0008] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0009] Figure 1 This is an exemplary application scenario of the method for processing neural network feature maps using multiple accelerators provided in this disclosure;

[0010] Figure 2 This is a flowchart illustrating a method for processing neural network feature maps using multiple accelerators, provided by an exemplary embodiment of this disclosure:

[0011] Figure 3 This is a flowchart illustrating a method for processing neural network feature maps using multiple accelerators, provided in another exemplary embodiment of this disclosure.

[0012] Figure 4 This is a schematic diagram of the connection of each shift register array provided in an exemplary embodiment of this disclosure;

[0013] Figure 5 This is a flowchart illustrating a method for processing neural network feature maps using multiple accelerators, provided in yet another exemplary embodiment of this disclosure.

[0014] Figure 6 This is a schematic diagram of the data shifting process of a shift register array in one operation cycle provided by an exemplary embodiment of the present disclosure;

[0015] Figure 7 This is a schematic diagram of the connection of a shift register array provided in another exemplary embodiment of this disclosure;

[0016] Figure 8This is a schematic diagram of the structure of an apparatus for processing neural network feature maps using multiple accelerators, provided in an exemplary embodiment of this disclosure.

[0017] Figure 9 This is a schematic diagram of the structure of an apparatus for processing neural network feature maps using multiple accelerators, provided in another exemplary embodiment of this disclosure;

[0018] Figure 10 This is a schematic diagram showing the connection of four accelerators and a target shift register array provided in an exemplary embodiment of this disclosure;

[0019] Figure 11 This is a schematic diagram of the specific structure of an accelerator provided in an exemplary embodiment of this disclosure;

[0020] Figure 12 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed herein. Detailed Implementation

[0021] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0022] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0023] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0024] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0025] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0026] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0027] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0028] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0029] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0030] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0031] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0032] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0033] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0034] This disclosure outlines

[0035] In realizing this disclosure, the inventors discovered that as the size of feature maps and the computational power requirements in convolutional neural networks increase, multiple accelerators are often needed to collaboratively process the same feature map. In related technologies, the feature map is typically split in at least one direction (width or height), and the resulting sub-feature maps are then processed collaboratively by multiple accelerators, such as through convolution operations. The convolution operation results of the multiple convolution operation accelerators are then combined to obtain the corresponding convolution operation result for the feature map. However, during the convolution operation, the split sub-feature maps often have overlapping regions at their boundaries that need to be reused by multiple accelerators. For example, when performing convolution operations on the boundary pixels of a sub-feature map, if the convolution kernel size is not 1*1, but rather 3*3, the feature values ​​of another sub-feature map adjacent to that boundary of the sub-feature map need to be used. Similarly, the convolution of another sub-feature map at its boundary pixels also requires the feature values ​​of the adjacent sub-feature map. These regions that are reused by adjacent sub-feature maps are called overlapping regions. For feature data in overlapping regions, related technologies typically employ two approaches. One approach involves additional storage of the feature data in the memory of each accelerator, allowing each accelerator to independently complete its computational tasks. The other approach involves storing only non-overlapping feature data in the memory of each accelerator core. The feature data in overlapping regions is then transmitted between the memories of each accelerator via NOC (Network on Chip) communication. The first approach results in a large memory storage requirement within the accelerator, while the second approach leads to wasted NOC bandwidth due to data transmission between accelerators.

[0036] Exemplary Overview

[0037] Figure 1This is an exemplary application scenario of the method for processing neural network feature maps using multiple accelerators provided in this disclosure. When convolution operations are required on neural network feature maps, the method for processing neural network feature maps using multiple accelerators (based on a device for processing neural network feature maps using multiple accelerators) of this disclosure can be used. Shift register arrays can be configured in the neural network accelerators (hereinafter referred to as accelerators). The shift register arrays in each neural network accelerator are interconnected, and are used to transfer feature data of overlapping regions between neural network accelerators based on data shifting of the register arrays. The figure uses four accelerators as an example; in practical applications, the number of accelerators can be set according to actual needs. Specifically, first feature data related to the neural network feature map can be read from the first shift register array of the first accelerator in the first of multiple neural network accelerators, and first weight data corresponding to the first feature data can be read from the first buffer in the first accelerator. Based on the first accelerator, a preset operation is performed on the first feature data and the first weight data to obtain a first operation result. Based on a preset shift rule, the first overlapping feature data required by the second accelerator in the first feature data is shifted from the first shift register array to the second shift register array of the second accelerator. Second feature data including the first overlapping feature data is read from the second shift register array in the second accelerator, and second weight data corresponding to the second feature data is read from the second buffer in the second accelerator. Based on the second accelerator, a preset operation is performed on the second feature data and the second weight data to obtain a second operation result. Multiple neural network accelerators work together to complete the convolution operation of the neural network feature map through multiple shifts to obtain the output feature map corresponding to the neural network feature map.

[0038] The first accelerator is any accelerator among multiple neural network accelerators that stores overlapping region feature data required by other accelerators, such as neural network accelerator 1 (which needs to provide overlapping region feature data for neural network accelerator 0), neural network accelerator 2 (which needs to provide overlapping region feature data for neural network accelerator 1), and so on. The second accelerator is an accelerator that has overlapping region feature data with the first accelerator; for example, when the first accelerator is neural network accelerator 1, the second accelerator can be neural network accelerator 0. The preset operation can be multiply-accumulate (MAC) operation. Each accelerator can include multiple multiply-accumulate operation units, which form a MAC array. Each multiply-accumulate operation unit can perform the operation of multiplying a feature value with its corresponding weight and adding the product to the previous accumulation result. Based on this, through multiple shifts of the register array, multiple feature values ​​required for the convolution of a pixel are provided to each multiply-accumulate operation unit. For example, a 3*3 convolution requires 9 feature values, enabling the multiply-accumulate operation unit to complete the convolution operation for that pixel. By using shift register arrays within multiple accelerators to shift data in overlapping regions, data from overlapping regions within one accelerator can be shifted to adjacent accelerators. This allows each accelerator to store non-overlapping data independently, achieving data reuse while reducing storage space requirements within the accelerator. This significantly reduces the bandwidth waste caused by NOC data transmission between accelerators, lowers power consumption, and effectively improves accelerator performance.

[0039] Exemplary methods

[0040] Figure 2 This is a flowchart illustrating a method for processing neural network feature maps using multiple accelerators, provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, specifically, for example, in-vehicle computing platforms. Figure 2 As shown, it includes the following steps:

[0041] Step 201: Read first feature data related to the neural network feature map from the first shift register array in the first accelerator of the plurality of neural network accelerators, and read first weight data corresponding to the first feature data from the first buffer in the first accelerator.

[0042] The first accelerator can be any one of multiple neural network accelerators, and the shift register array included in the first accelerator is called the first shift register array. Since the shift register array is used to provide feature data for the accelerator's operations, its size can be determined based on the configuration of the computational units in the accelerator. For example, if the MAC array in the accelerator includes a 2x2 array of computational units, then the shift register array can be a 2x2 register array. The shift register array can perform two-degree-of-freedom shifts. The first feature data is either pre-configured feature data related to the neural network feature map in the first shift register array, or it can be feature data related to the neural network feature map after data shifting; the specific configuration is not limited. The first buffer is pre-configured with weight data for performing preset operations on the feature maps of the neural network, such as the nine weight values ​​of each 3*3 convolution kernel. In each operation of the accelerator, the first weight data corresponding to the first feature data of the current operation can be read from the first buffer. For example, the first feature data includes four 2*2 feature values, which are respectively transmitted to the four operation units of the accelerator. The weight value corresponding to the current operation is provided to each operation unit for multiplying with the feature value, and the product result is added to the previous accumulated result.

[0043] Step 202: Perform a preset operation on the first feature data and the first weight data based on the first accelerator to obtain the first operation result.

[0044] The preset operation can be a multiply-accumulate operation, or other possible operations, such as averaging or maximizing operations in overlapping pooling layers, which can be set according to actual needs. The first accelerator may include at least one operation unit for the preset operation, such as a 2x2 MAC array, used to perform operations on the first feature data and the first weight data to obtain a first operation result. The first operation result is the multiply-accumulate result obtained in the current operation, and may include the same number of multiply-accumulate values ​​as the operation unit.

[0045] In an alternative embodiment, the first calculation result can be stored in a register for use in the next calculation.

[0046] Step 203: Based on a preset shift rule, shift the first overlapping feature data in the first feature data that is needed by the second accelerator among the multiple neural network accelerators from the first shift register array to the second shift register array of the second accelerator.

[0047] The preset shift rules can be determined based on the specific circumstances of the preset computation weight data. For example, if the convolution kernel size is 3*3, then after providing feature data to the accelerator once, it needs to be shifted 8 more times according to certain rules. This provides each computation unit of the accelerator with a total of 9 feature values ​​corresponding to the 9 weights of the convolution kernel, enabling each computation unit of the accelerator to complete the convolution operation for one pixel. The second accelerator is the one that needs to obtain overlapping region feature data from the first accelerator during the computation process. The first overlapping feature data can be partial feature data of the overlapping region.

[0048] Step 204: Read second feature data including first overlapping feature data from the second shift register array in the second accelerator, and read second weight data corresponding to the second feature data from the second buffer in the second accelerator.

[0049] Step 205: Perform preset operations on the second feature data and the second weight data based on the second accelerator to obtain the second operation result.

[0050] The specific operating principles of steps 204 and 205 are similar to those of steps 201 and 202 above, and will not be repeated here.

[0051] In an optional embodiment, steps 201 to 203 are performed simultaneously on multiple neural network accelerators. The shift register arrays in each neural network accelerator are connected according to a certain arrangement rule to form a whole shift register array, which synchronously shifts data. Multiple neural network accelerators can simultaneously complete the preset calculations for more pixels.

[0052] The method for processing neural network feature maps using multiple accelerators provided in this embodiment uses shift register arrays within multiple accelerators to shift data in overlapping regions. This allows data in overlapping regions within one accelerator to be shifted to adjacent accelerators, enabling each accelerator to store non-overlapping data. This reduces the storage space requirements within the accelerators while achieving data reuse, significantly reducing bandwidth waste in NOCs between accelerators, lowering power consumption, and effectively improving accelerator performance.

[0053] Figure 3 This is a flowchart illustrating a method for processing neural network feature maps using multiple accelerators, provided in another exemplary embodiment of this disclosure.

[0054] In one alternative embodiment, the shift register arrays in each neural network accelerator are connected to the target shift register arrays outside the multiple neural network accelerators according to a preset arrangement rule.

[0055] Before step 201, which involves reading the first feature data related to the neural network feature map from the first shift register array in the first accelerator and reading the first weight data corresponding to the first feature data from the first buffer in the first accelerator, the method of this disclosure further includes:

[0056] Step 310: For each of the multiple neural network accelerators, based on the size of the shift register array in that accelerator, read the feature data required for the current operation cycle of that accelerator from the memory in that accelerator and write it into the shift register array in that accelerator.

[0057] The current computation cycle includes the computation before the shift and the computation after the shift by a preset number of times. The feature data required for the current computation cycle includes the first feature data to be processed and the feature data to be processed after the shift. The preset number of times can be determined based on the number of feature values ​​required by each computation unit in one computation cycle. For example, in convolution operations, the preset number of times can be determined based on the number of weight values ​​of the convolution kernel. The memory in each accelerator stores a portion of the feature data related to the neural network feature map configured for that accelerator that does not overlap with other accelerators. For example, the neural network feature map can be split along the width (W) direction and evenly distributed to each accelerator, and stored in the memory within each accelerator. Since the size of the shift register array is limited, it is usually not possible to write all the feature data in the memory of the accelerator into the shift register array at once. Therefore, it is necessary to write in batches. For example, after each write, a convolution operation (one computation cycle) is completed through multiple shifts before writing the feature data required for the next computation cycle.

[0058] Step 320: Read the third feature data of the next operation cycle of the first preset accelerator from the memory of the first preset accelerator among the multiple neural network accelerators, and write the third feature data into the target shift register array. The third feature data includes the overlapping feature data required by the second preset accelerator among the multiple neural network accelerators.

[0059] The target shift register array stores the overlapping feature data required by the second preset accelerator among multiple neural network accelerators during the current computation cycle. The first and second preset accelerators can be determined based on the arrangement rules between the multiple shift register arrays and the target shift register array, for example, for... Figure 1 The four accelerators shown can be configured such that accelerator 0 is the first preset accelerator. A target shift register array is set between accelerator 0 and accelerator 3, and then accelerator 3 is the second preset accelerator. The target shift register array is connected to the shift register array of accelerator 3, and the feature data of the overlapping area is provided to accelerator 3 through shifting.

[0060] For example, Figure 4 This is a schematic diagram of the connection of various shift register arrays provided in an exemplary embodiment of this disclosure, where COREi_SRG represents the shift register array of accelerator i (i = 0, 1, 2, 3), and Common_SRG represents the target shift register array. Each accelerator's shift register array is a 4*2 array that can be shifted bidirectionally in the height direction and unidirectionally in the width direction. That is, it can shift upwards and downwards in the height direction, and can cyclically shift without losing feature values ​​in the height direction, ensuring that the nine feature values ​​can be shifted sequentially to the same register for use by the same computation unit. The gray registers in each accelerator's shift register array are used to provide feature values ​​to the computation unit, i.e., to store the first feature data, while the white registers store the feature values ​​to be processed after shifting. In this example, the width of the neural network feature map is W = 16. After being segmented along the width direction, W0-W1 and W8-W9 are stored in the memory of accelerator 0, W2-W3 and W10-W11 are stored in the memory of accelerator 1, W4-W5 and W12-W13 are stored in the memory of accelerator 2, and W6-W7 and W14-W15 are stored in the memory of accelerator 3. During the first operation, the eight feature values ​​of W0H0-W0H3 and W1H0-W1H3 are written to the shift register array (CORE0_SRG) of accelerator 0. Other accelerators, as shown in the figure, write the eight feature values ​​of W8H0-W8H3 and W9H0-W9H3 required by accelerator 0 for the next operation cycle to the target shift register array, providing feature data for the overlapping region of accelerator 3. For the remaining feature data in the height direction, after completing the calculations in the width direction, it can be written to the shift register array starting from W0H2 to complete the convolution and other operations for the next batch of pixels, and so on, until the neural network feature map calculations are completed. Details are omitted here. This is just an exemplary connection method. In practical applications, it can also be set to bidirectional cyclic shifting in the width direction and unidirectional shifting in the height direction; or both the width and height directions can be set to bidirectional cyclic shifting, depending on the actual needs. Corresponding shift rules need to be set according to different connection methods to ensure that the corresponding calculations can be completed.

[0061] In an optional embodiment, step 320, writing the third feature data into the target shift register array, may include:

[0062] Write the third feature data or the overlapping feature data in the third feature data that is needed by the second preset accelerator into the target shift register array.

[0063] The target shift register array serves to provide the second preset accelerator with the feature data of the overlapping region required for the current computation cycle. Depending on the size of the accelerator's computation array and the convolution kernel, the size of the target shift register array can be the same as or different from the shift register array in the accelerator. For example, when the computation array is larger than the convolution kernel (e.g., an 8x8 array and a 5x5 kernel), the minimum number of registers in the target shift register array in the shift direction between accelerators can be set to 4 (i.e., the kernel width minus 1), which is sufficient to provide the second preset accelerator with the feature data of the overlapping region required to complete a 5x5 convolution. When the target shift register array differs in size from the shift register array in the accelerator, the feature data of the overlapping region required by the second preset accelerator can be read from the memory of the first preset accelerator in the third feature data of the next computation cycle and written to the target shift register array. For example, if the third feature data is 12x8, the 12x4 feature data belonging to the overlapping region can be written to the target shift register array; the specific size can be set according to actual needs.

[0064] This embodiment connects the shift register arrays within each accelerator and the target shift register array outside the accelerator according to a preset arrangement rule, thereby providing corresponding overlapping region feature data for each accelerator. This ensures that the edge accelerators after shifting can also obtain the overlapping region data they need, improving the accuracy and effectiveness of accelerator processing.

[0065] In an optional embodiment, after step 202, which involves performing a preset operation on the first feature data and the first weight data based on the first accelerator to obtain the first operation result, the method further includes:

[0066] Step 330: Based on the preset shift rules, shift the second overlapping feature data required by the second preset accelerator from the third feature data in the target shift register array to the shift register array in the second preset accelerator.

[0067] The specific operating principle of step 330 is similar to that of step 203, and will not be repeated here.

[0068] This embodiment provides feature data of the overlapping region for the operation of the second preset accelerator by shifting the target shift register array.

[0069] In an optional embodiment, before step 201, which involves reading the first feature data associated with the neural network feature map from the first shift register array in the first accelerator and reading the first weight data corresponding to the first feature data from the first buffer in the first accelerator, the method further includes:

[0070] Step 410: Based on the preset segmentation rules, the neural network feature map is segmented into non-overlapping feature data corresponding to each neural network accelerator.

[0071] The preset segmentation rules can be set according to the size of the accelerator's computation array and shift register array. The principle is to enable multiple accelerators to provide the required overlapping feature data to their computation arrays through data shifting via the shift register array. The neural network feature map is segmented into the same number of non-overlapping feature data portions as the accelerators according to the preset segmentation rules. Each portion of feature data can include multiple discontinuous parts of the neural network feature map. For example, the preset segmentation rules could be a rule that segments the 16-width neural network feature map along its width, or a rule that segments by height. The height-based segmentation rule is similar to the width-based segmentation rule; the specific rules can be set according to actual needs.

[0072] Step 420: Write the non-overlapping feature data corresponding to each neural network accelerator into the memory of each neural network accelerator.

[0073] Since the arrangement rules of the shift register array determine the relationship between the overlapping region feature data provided by each accelerator, the correspondence between each segment of feature data and each accelerator can be determined according to the arrangement rules of the shift register array of the accelerator. For example, in the example above, W0-W1 and W8-W9 are stored in the memory of accelerator 0, W2-W3 and W10-W11 are stored in the memory of accelerator 1, W4-W5 and W12-W13 are stored in the memory of accelerator 2, and W6-W7 and W14-W15 are stored in the memory of accelerator 3.

[0074] This embodiment segments the neural network feature map and writes the segmented non-overlapping feature data into the memory of each accelerator. This allows the memory in the accelerator to store only non-overlapping feature data, thereby reducing the storage space requirement of the memory in the accelerator and improving the performance of the accelerator.

[0075] Figure 5 This is a flowchart illustrating a method for processing neural network feature maps using multiple accelerators, provided in yet another exemplary embodiment of this disclosure.

[0076] In an optional embodiment, the preset operation is a multiplication-accumulation operation; step 202, which involves performing the preset operation on the first feature data and the first weight data based on the first accelerator to obtain the first operation result, includes:

[0077] Step 2021: For any multiply-accumulate operation unit in the first accelerator, determine the first feature value corresponding to the multiply-accumulate operation unit in the first feature data, and the first weight value corresponding to the multiply-accumulate operation unit in the first weight data.

[0078] The multiply-accumulate unit is used to perform multiply-accumulate operations on multiple sets of feature values ​​and weights. The multiply-accumulate unit may include a multiplier and an adder, used to perform product operations and accumulation operations, respectively. The first feature data includes the feature value (first feature value) required by each multiply-accumulate unit for this operation, and the weight value (first weight value) corresponding to this feature value is determined from the first weight data.

[0079] Step 2022: Based on the multiply-accumulate operation unit, the first feature value and the first weight value are multiplied to obtain the first product result.

[0080] Specifically, the first feature value and the first weight value are input into the multiply-accumulate operation unit, and the multiplier in the multiply-accumulate operation unit performs the product operation of the first feature value and the first weight value to obtain the first product result.

[0081] Step 2023: Add the first product result to the previous accumulation result corresponding to the multiply-accumulate operation unit to obtain the current accumulation result corresponding to the multiply-accumulate operation unit. The previous accumulation result is the multiply-accumulate result obtained by the multiply-accumulate operation unit in the previous operation.

[0082] The previously accumulated result can be stored in the adder in the multiply-accumulate unit. For example, the adder has a register to store the accumulated result each time. The first product result is transmitted to the adder, which performs the addition operation between the first product result and the previously accumulated result to obtain the current accumulated result. The current accumulated result can be written to the register to replace the previously accumulated result for the next accumulation operation.

[0083] Step 2024: Take the current accumulation result corresponding to each multiply-accumulate operation unit in the first accelerator as the first operation result.

[0084] When the first accelerator includes multiple multiply-accumulate operation units, each multiply-accumulate operation unit can obtain a current accumulation result, and the current accumulation results of multiple multiply-accumulate operation units are used as the first operation result of the first accelerator.

[0085] This embodiment implements the multiplication and accumulation operation of feature values ​​and weight values ​​of neural network feature maps through an accelerator for multiplication and accumulation operations. It can be used for processing overlapping regions of neural network feature maps, such as convolution operations, which require multiplication and accumulation operations. This greatly improves the flexibility of processing various operations. Combining multiple accelerators can improve the efficiency of processing various operations.

[0086] In an optional embodiment, the preset shift rule includes a preset number of shifts and a shift method corresponding to each shift; step 203, based on the preset shift rule, shifts the first overlapping feature data required by the second accelerator among multiple neural network accelerators from the first shift register array to the second shift register array of the second accelerator, including:

[0087] Step 2031: Determine the current number of shifts.

[0088] The preset number of operations can be determined based on the size of the convolution kernel. Specifically, the preset number of operations can be one less than the number of weight values ​​included in the convolution kernel. For example, if the convolution kernel size is 3*3, it includes 9 weight values, and the preset number of operations is 8. The shifting method can include shifting along the width direction and shifting along the height direction of the shift register array. Each method can include at least one direction. For example, the width direction includes at least one left shift and right shift, and the height direction includes at least one up shift and down shift. The specific settings can be configured according to actual needs. The current shift count can be maintained in real time during the processing of the current operation cycle. For example, the current shift count is initialized to 0. After completing the operations before the shift, the current shift count is determined to be 0, indicating that no shift has been performed and the first shift is about to begin. After the first shift is completed and the operation is finished, the current shift count is updated to 1, indicating that one shift has been completed and the second shift is about to begin, and so on. The current shift count can be determined before each shift.

[0089] In one optional embodiment, the shift register arrays can be connected along either the width or height direction, depending on actual needs. The number and arrangement of registers in the shift register array can be determined based on the connection direction. One of the two methods, width-direction shifting and height-direction shifting, can be called the inter-accelerator shifting method (first shifting method), and the other can be called the intra-accelerator shifting method (second shifting method). For example, when the shift register arrays are connected along the width direction, the width-direction shifting will shift data from one accelerator's shift register array to another accelerator's shift register array. In this case, the width-direction shifting is called the first shifting method, and the height-direction shifting is called the second shifting method.

[0090] In an optional embodiment, the first shift method can be subdivided into two opposite directions according to actual needs. For example, if the width direction is the first shift method, it can include two directions: left and right shifts along the width direction. Similarly, the second shift method can be subdivided into two opposite directions. For example, if the height direction is the second shift method, it can include two directions: upward and downward shifts along the height direction. Alternatively, the first shift method can be configured to shift in only one direction, or the second shift method can shift in only one direction. It can also be configured so that both the first and second shift methods can shift in two directions, depending on actual needs. In this case, the target movement direction of the shift method corresponding to the current shift count can be further determined, and the data shift in the shift register array can be controlled according to the target movement direction. For example, if the shift method corresponding to the current shift count is determined to be a left shift of the first shift method, then the data in the shift register array is controlled to shift to the left.

[0091] In an optional embodiment, each direction of movement can be directly used as the movement method. The movement method can include four types: leftward movement, rightward movement, upward movement, and downward movement. Two of these movement methods are inter-accelerator movement methods, and the other two are intra-accelerator movement methods. Details will not be elaborated further.

[0092] Step 2032a: In response to the shift mode corresponding to the current shift number being the first shift mode, the first overlapping feature data required by the second accelerator in the first feature data is shifted from the first shift register array to the second shift register array of the second accelerator based on the first shift mode; the shifted first shift register array includes the third overlapping feature data from the third shift register array of the third accelerator among the multiple neural network accelerators.

[0093] The first shifting method is an inter-accelerator shifting method. Therefore, the first overlapping feature data required by the second accelerator in the first feature data can be shifted from the first shift register array to the second shift register array of the second accelerator. Simultaneously, the third overlapping feature data required by the first accelerator in the third shift register array of the third accelerator will also be shifted to the first shift register array. The first overlapping feature data may only be a portion of the overlapping feature data in the overlapping region. For example, when the convolution kernel is large, the width or height of the overlapping region is large. Each inter-accelerator shift only shifts the feature data by one pixel, therefore multiple inter-accelerator shifts may be needed to complete the shifting of the overlapping region data required for the current computation cycle.

[0094] Step 2032b: In response to the shift mode corresponding to the current shift number being the second shift mode, the feature data in the first shift register array is shifted according to the second shift mode.

[0095] The second shift method refers to the shift method within the accelerator, which can provide characteristic data of the overlapping region within the accelerator for the current computation cycle.

[0096] For example, Figure 6 This is a schematic diagram illustrating the data shifting process of a shift register array in one operation cycle according to an exemplary embodiment of this disclosure. The shift register array in this example can be used to provide feature data for a 2x2 MAC array, used for convolution operations with kernels of 3x3 or less. This example demonstrates the shifting process for a 3x3 convolution operation. The preset number of operations is 8, completing one convolution operation from the state before shifting through up-up-left-down-down-left-up-up. Taking the first register in CORE0_SRG as an example, its initial state is W0H0. After shifting up, it becomes W0H1, then shifting up again to become W0H2, shifting left to become W1H2, shifting down to become W1H1, shifting down again to become W1H0, shifting left to become W2H0, shifting up to become W2H1, and shifting up again to become W2H2. This first register provides feature values ​​to the first operation unit in accelerator 0. It can be seen that by shifting the 3*3 9 feature values ​​sequentially to the first operation unit, the convolution operation on pixel W1H1 can be completed.

[0097] For example, Figure 7 This is a schematic diagram of the connection of a shift register array provided in another exemplary embodiment of this disclosure. In this example, the overall shift register can be bidirectionally cyclically shifted in the width direction and bidirectionally shifted in the height direction, or it can be set to unidirectionally shift upwards in the height direction. Under this connection method, the shift sequence should be: left shift - left shift - up shift - right shift - right shift - up shift - left shift - left shift, which can achieve the connection with... Figure 6 Same functionality. Details will not be elaborated further.

[0098] For example, if in Figure 7 Based on this, bidirectional cyclic shifting is also possible along the height direction of the shift register array, and the shift sequence can be adopted as described above. Figure 6 and Figure 7 The shift order in any example can be set to other shift orders according to actual needs, such as shift up-shift up-shift left-shift left-shift down-shift down-shift right-shift up, or shift left-shift left-shift up-shift up-shift right-shift right-shift down-shift left, etc., as long as the feature values ​​that need to be multiplied and accumulated can be shifted to the corresponding operation units in sequence. The specifics will not be elaborated further.

[0099] In practical applications, the size of each shift register array is not limited to the aforementioned 4*2; it can be set to a larger or smaller array depending on actual needs. Furthermore, depending on the size of the convolution kernel, the size of the target shift register array between accelerators can be the same as or different from the size of the shift register array within the accelerator. The number of accelerators can also be set according to actual needs. For example, for a MAC array with an accelerator operation array of T*T (taking 8*8 as an example) and a convolution kernel of M*M (taking 5*5 as an example), the size of the shift register array within each accelerator can be (T+(M-1))*T = 12*8, and the number in the height direction can also be greater than (T+(M-1)). The size of the target shift register array can be set to (T+(M-1))*(M-1) = 12*4, or the number in the width direction can also be greater than M-1, for example, 8, as long as it can provide the feature data of the overlapping region required by the second preset accelerator. The specific implementation principle is similar to the above example and will not be elaborated further.

[0100] The method disclosed herein also includes:

[0101] Step 510: Read the shifted fourth feature data from the shifted first shift register array, and read the fourth weight data corresponding to the fourth feature data from the first buffer.

[0102] Depending on the current shift count, the fourth feature data can be data that includes overlapping feature data from the shift register arrays of adjacent accelerators (e.g., ...). Figure 6 The example left-shifted feature data can be the data updated by its own internal shift (for example, after one up shift in the above example, the feature value of the first row register of CORE0_SRG is updated to the original feature value of the second row). After each shift, the feature values ​​in each register of the first shift register array are updated. The new feature value and corresponding weight value required by the operation unit can be read and given to the corresponding operation unit for multiplication and accumulation operation again.

[0103] Step 520: Based on the first accelerator, perform a preset operation on the fourth feature data, the fourth weight data, and the first operation result to obtain the fourth operation result.

[0104] For the specific operating principles of steps 510 and 520, please refer to the aforementioned embodiments.

[0105] Step 530: Use the fourth feature data as the first feature data and the fourth operation result as the first operation result, and repeat the step of determining the current number of shifts.

[0106] After each multiplication-accumulation operation, a next shift is required. Since the next shift may differ from the previous shifts, the current shift count needs to be determined to determine the next shift pattern. Referring to the example above, with 8 shifts, the first shift is an up shift, the second shift is an up shift, the third shift is a left shift, and so on. Based on the shift pattern corresponding to the current shift count, steps 2032a or 2032b, and subsequent steps 510 and 520, are executed.

[0107] Step 540: In response to the current number of shifts reaching a preset number, the fourth operation result is used as the target operation result of the current operation cycle corresponding to the first accelerator. The current operation cycle includes the operation before the shift and the operation after the preset number of shifts.

[0108] The determination of whether the current shift count has reached the preset number can be performed after each execution of step 2031 or after each completion of step 520, depending on the actual needs. For example, after step 2031, it can be determined whether the current shift count has reached the preset number. If not, step 2032a or step 2032b is executed. If the preset number has been reached, the fourth operation result, which is the first operation result, is taken as the target operation result of the current operation cycle. For convolution operations, the target operation result of one operation cycle of an accelerator is the convolution operation result of the same number of pixels as the number of operation units completed by the accelerator's operation array in the current operation cycle.

[0109] In an optional embodiment, step 502 may further include updating the current shift count. Specifically, the current shift count may be increased by 1. When determining the current shift count again, it will be increased by 1 compared to the previous one. The timing of updating the current shift count is not limited.

[0110] This embodiment achieves the reuse of feature data in overlapping areas by shifting the data a preset number of times. This ensures that each accelerator can complete multiplication and accumulation operations without storing feature data in overlapping areas. As a result, the reuse of data in overlapping areas can be achieved with less storage space within the accelerator, reducing the waste of NOC bandwidth resources between accelerators, reducing power consumption, and thus effectively improving accelerator performance.

[0111] In an optional embodiment, after step 540, in response to the current shift count reaching a preset number, and using the fourth operation result as the target operation result for the current operation cycle corresponding to the first accelerator, the method further includes:

[0112] Step 550: Read the fifth feature data corresponding to the next operation cycle of the first accelerator from the memory in the first accelerator.

[0113] Step 560: Write the fifth feature data into the first shift register array of the first accelerator.

[0114] The specific operating principles of steps 550 and 560 are the same as those of step 310, and will not be repeated here.

[0115] Step 570: Repeat the steps of reading the first feature data related to the neural network feature map from the first shift register array in the first accelerator and reading the first weight data corresponding to the first feature data from the first buffer in the first accelerator.

[0116] Specifically, during the first repeated execution of step 201, the first feature data is the feature data from the fifth feature data that was written into the register portion to provide feature values ​​for the arithmetic unit. In subsequent repeated executions, the first feature data is the shifted feature data from that register portion.

[0117] Step 580: In response to the first accelerator completing the processing of the neural network feature map related to the operation cycle, the first output feature sub-map corresponding to the neural network feature map is obtained based on the target operation results obtained by the first accelerator in each operation cycle.

[0118] Whether the first accelerator has completed the processing of the neural network feature map related computation cycles can be determined by factors such as the size of the feature map, the segmentation rules, and the number of computations each accelerator needs to complete after segmentation. For example, based on the size of the feature map and the segmentation rules, combined with the size of the accelerator's computation array, the number of clock cycles required for each accelerator can be calculated. During the computation process, the completion of the computation can be determined by counting the clock cycles.

[0119] Step 590: Based on the first output feature sub-maps corresponding to the multiple neural network accelerators, obtain the output feature map corresponding to the neural network feature map.

[0120] The specific representation of the first output feature sub-map can be set according to actual needs. For example, depending on the segmentation rule, the representation of the first output feature sub-map can be different. If the segmentation rule causes each accelerator to store multiple discontinuous parts of the neural network feature map, such as W0-W1 and W8-W9 mentioned above, then the first output feature sub-map can include the sub-maps corresponding to each part, and the pixel positions can be marked. The first output feature sub-maps of multiple accelerators are concatenated according to the pixel positions to obtain the output feature map. Alternatively, the first output feature sub-map can be the same size as the output feature map, and the effective data only includes the operation results of the part handled by that accelerator. For the parts not completed by that accelerator, they can be set to 0. The first output feature sub-maps of multiple accelerators are added together to obtain the output feature map corresponding to the neural network feature map. The specific representation is not limited, as long as the output feature map corresponding to the neural network feature map can be obtained.

[0121] In an optional embodiment, for each accelerator, steps 550 and 560 are executed synchronously as the first accelerator. After step 560, the overlapping feature data required by the second preset accelerator in the next operation cycle is read from the memory of the first preset accelerator and written into the target shift register array. The operation arrays of multiple accelerators perform operations synchronously, and after the operation is completed, the multiple shift register arrays and the target shift register array are shifted synchronously, realizing the processing of the neural network feature map through the collaborative work of multiple accelerators to obtain the output feature map.

[0122] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0123] Any of the methods for processing neural network feature maps using multiple accelerators provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the methods for processing neural network feature maps using multiple accelerators provided in this disclosure can be executed by a processor, such as by a processor calling corresponding instructions stored in memory to execute any of the methods for processing neural network feature maps using multiple accelerators mentioned in this disclosure. Further details will not be elaborated below.

[0124] Exemplary device

[0125] Figure 8 This is a schematic diagram of an apparatus for processing neural network feature maps using multiple accelerators, provided in an exemplary embodiment of this disclosure. The apparatus of this embodiment can be used to implement corresponding method embodiments of this disclosure, such as... Figure 8The device shown includes: a plurality of neural network accelerators 61, each neural network accelerator 61 including a controller 611, a shift register array 612, a buffer 613 and an arithmetic array 614 for preset operations.

[0126] The controller 611 is connected to the shift register array 612, the buffer 613, and the arithmetic array 614, respectively, and is used to control the shift register array 612, the buffer 613, and the arithmetic array 614 to complete preset operations. The buffer 613 is also connected to the arithmetic array 614 to provide the weight data required for the preset operations. The shift register array 612 is connected to the arithmetic array 614 to provide the feature data required for the preset operations. Multiple neural network accelerators 61 are connected sequentially through the shift register array 612 to realize the shift multiplexing of feature data between accelerators. Multiple neural network accelerators 61 can operate under the control of an external synchronous clock.

[0127] For the first accelerator 61a (shown in the figure only as an example of neural network accelerator 61a; in actual applications, the first accelerator can be any accelerator among multiple neural network accelerators that needs to provide overlapping feature data to other accelerators), the first controller 611a in the first accelerator 61a reads first feature data related to the neural network feature map from the first shift register array 612a in the first accelerator 61a and reads first weight data corresponding to the first feature data from the first buffer 613a in the first accelerator 61a; the first controller 611a controls the first arithmetic array 614a in the first accelerator 61a to perform preset operations on the first feature data and the first weight data to obtain the first operation result. The first controller 611a controls the first shift register array 612a based on a preset shift rule to shift the first overlapping feature data required by the second accelerator 61b (neural network accelerator 61b) in the first feature data to the second shift register array 612b of the second accelerator 61b; the second controller 611b in the second accelerator 61b reads the second feature data including the first overlapping feature data from the second shift register array 612b and reads the second weight data corresponding to the second feature data from the second buffer 613b in the second accelerator; the second controller 611b controls the second operation array 614b in the second accelerator 61b to perform preset operations on the second feature data and the second weight data to obtain the second operation result.

[0128] In this embodiment, for any accelerator 61, its controller 611 can be a control logic unit used for computation control within the accelerator 61. Triggered by an external clock, the accelerator 611 can be controlled to complete part of the computation of the neural network feature map of this disclosure. Multiple accelerators 61 can collaborate to obtain the output feature map corresponding to the neural network feature map. The buffer 613 is used to buffer the weight data required for computation. In each computation cycle, the controller 611 can write the weight data required for that cycle into the buffer 613. The computation array 614 is used for accelerated computation of the feature data. The computation array 614 can be a MAC array, and its size can be set according to actual needs. For example, it can be a 2*2 array, a 4*4 array, a 5*5 array, an 8*8 array, etc. The size of the shift register array 612 can be set according to actual needs, specifically determined based on the size of the computation array 614 and the specific computation to be performed. See the foregoing method embodiments for details, which will not be repeated here. The number of accelerators 61 can be set according to actual needs, for example, it can include 2, 4, 8, etc., without specific limitation. The shift register arrays 612 in the multiple accelerators 61 are connected according to a certain arrangement rule so that the feature data of the overlapping region can be shifted between the accelerators, thereby realizing the reuse of the feature data of the overlapping region. The specific processing flow of the neural network feature map by the multiple accelerators can be found in the corresponding method embodiments, and will not be repeated here.

[0129] Figure 9 This is a schematic diagram of an apparatus for processing neural network feature maps using multiple accelerators, provided in another exemplary embodiment of this disclosure.

[0130] In an optional embodiment, each neural network accelerator 61 further includes a memory 615 connected to the controller 611; the device of this disclosure also includes a target shift register array 62 connected to the shift register array 612 of a second preset accelerator among the plurality of neural network accelerators 61; the target shift register array 62 and the shift register arrays 612 in each neural network accelerator 61 are connected according to a preset arrangement rule, specifically, the preset arrangement rule may be that the shift register arrays 612 in each neural network accelerator 61 and the target shift register array 62 are connected in series according to a preset order and a preset direction of the array. The preset order may be the arrangement order of the plurality of neural network accelerators, and the preset direction refers to the width or height direction of the shift register array 612, which can be set according to actual needs. For example, shift register array 612 is a 4*2 array, meaning it includes 4 rows of registers in the height direction and 2 columns of registers in the width direction. N shift register arrays 612 are connected sequentially in the width direction of the array. The target shift register array 62 is connected to the last shift register array 612, forming a 4-row, 2(N+1)-column total shift register array. The movable direction and cyclic nature of the total shift register array in the width direction can be set according to actual needs. For example, it can shift left or right in the width direction, or both. For left shifts, it can be a cyclic left shift or a non-cyclic left shift; similarly, for right shifts, it can be a cyclic right shift or a non-cyclic right shift, and so on. The memories of the first accelerator 61a and the second accelerator 61b are represented by 615a and 615b, respectively.

[0131] For each accelerator 61 in the plurality of neural network accelerators, the controller 611 in the accelerator 61 reads the feature data required for the current operation cycle of the accelerator 61 from the memory 615 in the accelerator 61 and writes it into the shift register array 612 in the accelerator 61, based on the size of the shift register array 612 in the accelerator 61. The current operation cycle includes the operation before the shift and the operation after the shift by a preset number of times. The feature data required for the current operation cycle includes the first feature data to be processed and the feature data to be processed after the shift. The controller 611 of the first preset accelerator in the plurality of neural network accelerators reads the third feature data of the next operation cycle of the first preset accelerator from the memory 613 of the first preset accelerator and writes the third feature data into the target shift register array 62. The third feature data includes the overlapping feature data required by the second preset accelerator in the plurality of neural network accelerators.

[0132] In this embodiment, the controller 611 in the accelerator 61 can be connected to the memory 615 via the NOC within the accelerator. The memory 615 can be the SRAM (Static Random-Access Memory) in the accelerator 61. The specific operating principle of this embodiment is described in the foregoing method embodiments and will not be repeated here.

[0133] In one optional example, Figure 10 This is a schematic diagram illustrating the connection of four accelerators and a target shift register array according to an exemplary embodiment of this disclosure. The shift register array 612 in each of the four accelerators 61 is sequentially connected to the target shift register array 62, enabling data shifting between the accelerators. Synchronization between the multiple accelerators 61 can be achieved through an external clock. For the specific structure of the shift register array 612 and the target shift register array 62 in each accelerator 61, please refer to the examples in the foregoing method embodiments; further details will not be repeated here.

[0134] In an optional embodiment, the controller 611 of the first preset accelerator is further configured to control the target shift register array 62 to shift the second overlapping feature data required by the second preset accelerator in the third feature data to the shift register array 62 in the second preset accelerator based on a preset shift rule.

[0135] In an optional embodiment, the apparatus of this disclosure further includes a control module 63, which is connected to each neural network accelerator 61.

[0136] The control module 63 is used to segment the neural network feature map into non-overlapping feature data corresponding to each neural network accelerator 61 based on a preset segmentation rule, and to write the non-overlapping feature data corresponding to each neural network accelerator 61 into the memory 615 of each neural network accelerator 61. The preset segmentation rule can be set according to the size of the accelerator's operation array and shift register array, with the principle of enabling multiple accelerators to provide the required overlapping region feature data to the accelerator's operation array through data shifting in the shift register array. The preset segmentation rule can be, for example, a rule to segment the neural network feature map along the width direction, or a rule to segment it along the height direction.

[0137] The control module 63 can be a control logic device other than the accelerators in the device, used to control multiple accelerators, and can be specifically configured according to actual needs. The control module 63 can also be used to generate the operating clock for each accelerator to trigger the periodic operation of each accelerator.

[0138] Figure 11 This is a schematic diagram of the specific structure of an accelerator provided in an exemplary embodiment of this disclosure.

[0139] In an optional embodiment, the preset operation is a multiply-accumulate operation; the operation array 614 includes multiple multiply-accumulate operation units 6141, each multiply-accumulate operation unit 6141 including a multiplier mul and an adder add. Each multiply-accumulate operation unit 6141 can be connected to a register 6121 in the corresponding shift register array 612. The register 6121 provides the feature values ​​required for the multiply-accumulate operation unit 6141, and each multiply-accumulate operation unit 6141 can also be connected to a buffer 613, which provides the weight values ​​required for the multiply-accumulate operation unit 6141.

[0140] In one alternative embodiment, the shift register array 612 includes multiple rows and columns of registers 6121, and the shift register array can be shifted along the row direction (width direction) and / or the column direction (height direction), which can be set according to actual needs. Figure 11 The arithmetic array 614 is used as an example of a 2x2 array, and the shift register array 612 is used as an example of a 2x8 array (it can also be 8x2). The four gray registers provide feature values ​​for the four multiply-accumulate arithmetic units. The gray registers can be directly connected to the input of each multiply-accumulate arithmetic unit, or the feature values ​​in the registers can be read by the controller and transmitted to the input of the multiply-accumulate arithmetic unit. The specific configuration can be set according to actual needs. The four white registers store the feature values ​​to be processed after shifting. The feature values ​​stored in the white registers are shifted to the gray registers through shifting and provided to the multiply-accumulate arithmetic units.

[0141] In an optional embodiment, the first controller 611a is specifically used for:

[0142] For any multiply-accumulate operation unit 6141 in the first operation array 614a, determine the first feature value corresponding to the multiply-accumulate operation unit 6141 in the first feature data, and the first weight value corresponding to the multiply-accumulate operation unit 6141 in the first weight data; control the multiplier mul of the multiply-accumulate operation unit 6141 to perform a product operation on the first feature value and the first weight value to obtain a first product result; control the adder add of the multiply-accumulate operation unit 6141 to add the first product result to the previous accumulation result corresponding to the multiply-accumulate operation unit to obtain the current accumulation result corresponding to the multiply-accumulate operation unit, where the previous accumulation result is the multiply-accumulate result obtained by the multiply-accumulate operation unit in the previous operation; and take the current accumulation result corresponding to each multiply-accumulate operation unit 6141 in the first accelerator 61a as the first operation result.

[0143] In an optional embodiment, the preset shift rule includes a preset number of shifts and the shift method corresponding to each shift; the first controller 611a is specifically used for:

[0144] The current shift count is determined; in response to the shift mode corresponding to the current shift count being the first shift mode, the first shift register array 612a is controlled to shift the first overlapping feature data required by the second accelerator 61b from the first feature data to the second shift register array 612b of the second accelerator 61b based on the first shift mode; the shifted first shift register array 612a includes the third overlapping feature data from the third shift register array of the third accelerator among multiple neural network accelerators; in response to the shift mode corresponding to the current shift count being the second shift mode, the first shift register array 612a is controlled to shift its feature data according to the second shift mode.

[0145] The first controller 611a is also used for:

[0146] The system reads the shifted fourth feature data from the shifted first shift register array 612a and the fourth weight data corresponding to the fourth feature data from the first buffer 613a; it controls the first arithmetic array 614a to perform a preset operation on the fourth feature data, the fourth weight data, and the first operation result to obtain the fourth operation result; it uses the fourth feature data as the first feature data and the fourth operation result as the first operation result, and repeats the step of determining the current shift count; in response to the current shift count reaching the preset number, it uses the fourth operation result as the target operation result of the current operation cycle corresponding to the first accelerator 61a, the current operation cycle including the operation before shift and the operation after the preset number of shifts.

[0147] In an optional embodiment, it further includes a control module 63.

[0148] The first controller 611a is also used for:

[0149] The control module 63 reads the fifth feature data corresponding to the next operation cycle of the first accelerator 61a from the memory 615a in the first accelerator 61a; writes the fifth feature data into the first shift register array 612a of the first accelerator 61a; repeatedly executes the steps of reading the first feature data related to the neural network feature map from the first shift register array 612a in the first accelerator 61a and reading the first weight data corresponding to the first feature data from the first buffer 613a in the first accelerator 61a; in response to the first accelerator 61a completing the processing of the operation cycle related to the neural network feature map, and obtains the first output feature sub-map corresponding to the neural network feature map based on the target operation results obtained by the first accelerator 61a in each operation cycle; the control module 63 is used to obtain the output feature map corresponding to the neural network feature map based on the first output feature sub-maps corresponding to the multiple neural network accelerators respectively.

[0150] The specific control principle of the first controller 611a is described in the aforementioned corresponding method embodiments, and will not be repeated here.

[0151] In an optional embodiment, the accelerator 61 may also include other neural network-related computing units, such as computing units for loading, storing, and pooling, which can be specifically configured according to actual needs.

[0152] Exemplary electronic devices

[0153] This disclosure also provides an electronic device, including: a memory for storing computer programs;

[0154] A processor is configured to execute a computer program stored in the memory, wherein, when the computer program is executed, it implements the method for processing neural network feature maps using multiple accelerators as described in any of the above embodiments of the present disclosure.

[0155] Figure 12 This is a schematic diagram of an application embodiment of the electronic device disclosed herein. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0156] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0157] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0158] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0159] For example, the input device 13 may be the microphone or microphone array described above, used to capture the input signal of the sound source.

[0160] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0161] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0162] Of course, for the sake of simplicity, Figure 12 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0163] Exemplary computer program products and computer-readable storage media

[0164] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0165] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0166] Furthermore, embodiments of this disclosure may also be computer-readable storage media having computer program instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0167] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0168] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0169] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0170] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0171] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0172] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0173] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0174] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for processing feature maps of a neural network using multiple accelerators, comprising: First feature data related to the feature map of the neural network is read from a first shift register array in a first accelerator of a plurality of neural network accelerators, and first weight data corresponding to the first feature data is read from a first buffer in the first accelerator; Based on the first accelerator, a preset operation is performed on the first feature data and the first weight data to obtain a first operation result; Based on a preset shift rule, the first overlapping feature data in the first feature data that is needed by the second accelerator among multiple neural network accelerators is shifted from the first shift register array to the second shift register array of the second accelerator. The first accelerator and the second accelerator are adjacent accelerators. The preset shift rule is set based on the connection method between the first shift register array and the second shift register array. Read second feature data including the first overlapping feature data from the second shift register array in the second accelerator, and read second weight data corresponding to the second feature data from the second buffer in the second accelerator; The second accelerator performs a preset operation on the second feature data and the second weight data to obtain a second operation result.

2. The method according to claim 1, wherein, The shift register arrays in each of the neural network accelerators are connected to the target shift register arrays outside the multiple neural network accelerators according to a preset arrangement rule; Before reading the first feature data associated with the neural network feature map from the first shift register array in the first accelerator and the first weight data corresponding to the first feature data from the first buffer in the first accelerator, the method further includes: For each of the multiple neural network accelerators, based on the size of the shift register array in the accelerator, the feature data required for the current operation cycle of the accelerator is read from the memory in the accelerator and written into the shift register array in the accelerator. The current operation cycle includes the operation before the shift and the operation after the shift by a preset number of times. The feature data required for the current operation cycle includes the first feature data to be processed and the feature data to be processed after the shift. The third feature data of the next operation cycle of the first preset accelerator is read from the memory of the first preset accelerator among the multiple neural network accelerators, and the third feature data is written into the target shift register array. The third feature data includes the overlapping feature data required by the second preset accelerator among the multiple neural network accelerators.

3. The method according to claim 2, wherein, After performing a preset operation on the first feature data and the first weight data based on the first accelerator to obtain the first operation result, the method further includes: Based on the preset shift rule, the second overlapping feature data required by the second preset accelerator in the third feature data of the target shift register array is shifted to the shift register array in the second preset accelerator.

4. The method according to claim 1, wherein, Before reading the first feature data associated with the neural network feature map from the first shift register array in the first accelerator and the first weight data corresponding to the first feature data from the first buffer in the first accelerator, the method further includes: Based on a preset segmentation rule, the neural network feature map is segmented into non-overlapping feature data corresponding to each of the neural network accelerators; The non-overlapping feature data corresponding to each of the neural network accelerators are written into the memory of each of the neural network accelerators.

5. The method according to claim 1, wherein, The preset operation is a multiplication-accumulation operation; The step of performing a preset operation on the first feature data and the first weight data based on the first accelerator to obtain a first operation result includes: For any multiply-accumulate operation unit in the first accelerator, determine the first feature value corresponding to the multiply-accumulate operation unit in the first feature data, and the first weight value corresponding to the multiply-accumulate operation unit in the first weight data; Based on the multiply-accumulate operation unit, the first feature value and the first weight value are multiplied to obtain the first product result; The first product result is added to the previous accumulation result corresponding to the multiply-accumulate operation unit to obtain the current accumulation result corresponding to the multiply-accumulate operation unit. The previous accumulation result is the multiply-accumulate result obtained by the multiply-accumulate operation unit in the previous operation. The current accumulation result corresponding to each of the multiply-accumulate operation units in the first accelerator is taken as the first operation result.

6. The method according to claim 1, wherein, The preset shift rules include a preset number of shifts and the shift method corresponding to each shift; The step of shifting the first overlapping feature data, which is needed by the second accelerator among multiple neural network accelerators, from the first shift register array to the second shift register array of the second accelerator based on a preset shift rule includes: Determine the current shift count; In response to the shift mode corresponding to the current shift count being the first shift mode, based on the first shift mode, the first overlapping feature data required by the second accelerator in the first feature data is shifted from the first shift register array to the second shift register array of the second accelerator; the shifted first shift register array includes the third overlapping feature data from the third shift register array of the third accelerator among multiple neural network accelerators; In response to the shift mode corresponding to the current shift count being the second shift mode, the feature data in the first shift register array is shifted according to the second shift mode; The method further includes: Read the shifted fourth feature data from the shifted first shift register array, and read the fourth weight data corresponding to the fourth feature data from the first buffer; Based on the first accelerator, a preset operation is performed on the fourth feature data, the fourth weight data, and the first operation result to obtain the fourth operation result; Using the fourth feature data as the first feature data and the fourth operation result as the first operation result, the step of determining the current number of shifts is repeated. In response to the current number of shifts reaching the preset number, the fourth operation result is used as the target operation result of the current operation cycle corresponding to the first accelerator. The current operation cycle includes the operation cycle before the shift and the operation cycle after the preset number of shifts.

7. The method according to claim 6, wherein, After the step of taking the fourth operation result as the target operation result for the current operation cycle corresponding to the first accelerator in response to the current shift count reaching the preset number, the method further includes: Read the fifth feature data corresponding to the next operation cycle of the first accelerator from the memory in the first accelerator; Write the fifth feature data into the first shift register array of the first accelerator; Repeat the steps of reading first feature data related to the neural network feature map from the first shift register array in the first accelerator and reading first weight data corresponding to the first feature data from the first buffer in the first accelerator; In response to the first accelerator completing the processing of the operation cycle related to the neural network feature map, a first output feature sub-map corresponding to the neural network feature map is obtained based on the target operation results obtained by the first accelerator in each operation cycle. Based on the first output feature sub-maps corresponding to multiple neural network accelerators, the output feature map corresponding to the neural network feature map is obtained.

8. An apparatus for processing feature maps of a neural network using multiple accelerators, comprising: Multiple neural network accelerators; each neural network accelerator includes a controller, a shift register array, a buffer, and an arithmetic array for preset operations; For a first accelerator among a plurality of neural network accelerators, a first controller in the first accelerator reads first feature data related to the feature map of the neural network from a first shift register array in the first accelerator and reads first weight data corresponding to the first feature data from a first buffer in the first accelerator; The first controller controls the first computing array in the first accelerator to perform a preset operation on the first feature data and the first weight data to obtain a first operation result; The first controller controls the first shift register array based on a preset shift rule to shift the first overlapping feature data in the first feature data that is needed by the second accelerator among the multiple neural network accelerators to the second shift register array of the second accelerator; the first accelerator and the second accelerator are adjacent accelerators, and the preset shift rule is set based on the connection method of the first shift register array and the second shift register array; The second controller in the second accelerator reads second feature data, including the first overlapping feature data, from the second shift register array, and reads second weight data corresponding to the second feature data from the second buffer in the second accelerator; The second controller controls the second computing array in the second accelerator to perform preset operations on the second feature data and the second weight data to obtain the second computing result.

9. The apparatus according to claim 8, wherein, Each neural network accelerator also includes a memory; the device also includes a target shift register array; the target shift register array is connected to the shift register arrays in each of the neural network accelerators according to a preset arrangement rule; For each of the multiple neural network accelerators, the controller in the accelerator reads the feature data required for the current operation cycle of the accelerator from the memory in the accelerator and writes it into the shift register array in the accelerator, based on the size of the shift register array in the accelerator. The current operation cycle includes the operation before the shift and the operation after the shift by a preset number of times. The feature data required for the current operation cycle includes the first feature data to be processed at the moment and the feature data to be processed after the shift. The controller of the first preset accelerator in the plurality of neural network accelerators reads the third feature data of the next operation cycle of the first preset accelerator from the memory of the first preset accelerator, and writes the third feature data into the target shift register array. The third feature data includes the overlapping feature data required by the second preset accelerator in the plurality of neural network accelerators.

10. The apparatus according to claim 8, wherein, Also includes: The control module is connected to each of the neural network accelerators; The control module is used to divide the neural network feature map into non-overlapping feature data corresponding to each of the neural network accelerators based on a preset segmentation rule, and to write the non-overlapping feature data corresponding to each of the neural network accelerators into the memory of each of the neural network accelerators.

11. The apparatus according to claim 8, wherein, The preset operation is a multiply-accumulate operation; the operation array includes multiple multiply-accumulate operation units, and each multiply-accumulate operation unit includes a multiplier and an adder; The first controller is specifically used for: For any multiply-accumulate operation unit in the first operation array, determine the first feature value corresponding to the multiply-accumulate operation unit in the first feature data, and the first weight value corresponding to the multiply-accumulate operation unit in the first weight data; The multiplier of the multiply-accumulate unit performs a product operation on the first feature value and the first weight value to obtain the first product result; The adder of the multiply-accumulate operation unit adds the first product result to the previous accumulation result corresponding to the multiply-accumulate operation unit to obtain the current accumulation result corresponding to the multiply-accumulate operation unit. The previous accumulation result is the multiply-accumulate result obtained by the multiply-accumulate operation unit in the previous operation. The current accumulation result corresponding to each of the multiply-accumulate operation units in the first accelerator is taken as the first operation result.

12. The apparatus according to claim 8, wherein, The preset shift rules include a preset number of shifts and the shift method corresponding to each shift; The first controller is specifically used for: Determine the current shift count; In response to the shift mode corresponding to the current shift count being the first shift mode, the first shift register array is controlled to shift the first overlapping feature data required by the second accelerator in the first feature data to the second shift register array of the second accelerator based on the first shift mode; the shifted first shift register array includes the third overlapping feature data from the third shift register array of the third accelerator among multiple neural network accelerators; In response to the shift mode corresponding to the current shift count being the second shift mode, the first shift register array is controlled to shift its feature data according to the second shift mode; The first controller is also used for: Read the shifted fourth feature data from the shifted first shift register array, and read the fourth weight data corresponding to the fourth feature data from the first buffer; The first computing array is controlled to perform a preset operation on the fourth feature data, the fourth weight data, and the first operation result to obtain the fourth operation result; Using the fourth feature data as the first feature data and the fourth operation result as the first operation result, the step of determining the current number of shifts is repeated. In response to the current number of shifts reaching the preset number, the fourth operation result is used as the target operation result of the current operation cycle corresponding to the first accelerator. The current operation cycle includes the operation cycle before the shift and the operation cycle after the preset number of shifts.

Images