Data processing method, processor, artificial intelligence chip and electronic device

Abstract

The present disclosure relates to a data processing method, a processor, an artificial intelligence chip and an electronic device. The processing method is applied to a computing core of a processor, the processor comprises a plurality of computing cores, and the processing method comprises: generating and activating a convolution operation result based on sub-data to be processed stored in the computing core; exchanging the sub-data to be processed with different computing cores, and performing the following steps until each computing core stores each piece of sub-data to be processed: releasing the storage space occupied by the sub-data to be processed before the exchange; and generating and activating a convolution operation result based on the sub-data to be processed after the exchange. The data processing method provided in the embodiment of the present disclosure reduces the storage space occupied by the convolution operation result in a single computing core, and thus realizes that the sub-data to be processed can still be normally processed in the case of a shortage of hardware resources.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and more particularly to a data processing method, processor, artificial intelligence chip, and electronic device. Background Technology

[0002] Neuromorphic computing chips employ a decentralized many-core parallel processing architecture, where each computing core can operate independently and exchange data. Typically, when dealing with large amounts of data, neuromorphic computing chips need to divide the data into multiple incomplete sub-data sets. Therefore, how these sub-data sets are processed directly impacts the processing efficiency and memory usage of the neuromorphic computing chip. Summary of the Invention

[0003] According to a first aspect of this disclosure, a data processing method is provided, applied to a computing core of a processor, the processor including multiple computing cores, the processing method comprising: generating and activating a convolution operation result based on sub-data to be processed stored within the computing cores; exchanging sub-data to be processed with different computing cores, and performing the following steps until each computing core has stored each piece of sub-data to be processed: releasing the storage space occupied by the sub-data to be processed before the exchange; generating and activating a convolution operation result based on the sub-data to be processed after the exchange; wherein the sub-data to be processed is a part of the data to be processed, and the sub-data to be processed has the same depth as the data to be processed.

[0004] In one possible implementation, the processing method further includes: dividing the data to be processed into sub-data to be processed in at least one non-depth dimension; and storing each sub-data to be processed into a different computing core.

[0005] In one possible implementation, generating and activating the convolution operation result includes: generating the convolution operation result based on the weight data and the sub-data to be processed stored in each computing kernel; and activating the convolution operation result through a preset activation function.

[0006] In one possible implementation, dividing the data to be processed into sub-data to be processed in at least one non-depth dimension includes: dividing the data to be processed into sub-data to be processed in the last storage order dimension and non-depth dimension of the data to be processed; wherein the storage order dimension is used to determine the storage order of the data to be processed in different dimensions.

[0007] In one possible implementation, the exchange of unprocessed sub-data between different computing cores includes: determining the dynamic storage space number occupied by each piece of unprocessed sub-data to be exchanged based on the allocation order number and the number of exchanges for each computing core; wherein the allocation order number is used to indicate the allocation order of the unprocessed sub-data; and allocating dynamic storage space for each piece of unprocessed sub-data to be exchanged according to the dynamic storage space number.

[0008] In one possible implementation, the plurality of computing cores perform at least one of the following steps in a parallel synchronous operation: exchanging sub-data to be processed with different computing cores, releasing the storage space occupied by the sub-data to be processed before the exchange, and generating and activating the convolution operation result.

[0009] According to a second aspect of this disclosure, a processor is provided, the processor comprising a plurality of processors, the processor comprising a plurality of computing cores, the processor being configured to perform: generating and activating a convolution operation result based on sub-data to be processed stored within the computing cores; exchanging sub-data to be processed with different computing cores, and performing the following steps until each computing core has stored each piece of sub-data to be processed: releasing the storage space occupied by the sub-data to be processed before the exchange; generating and activating a convolution operation result based on the sub-data to be processed after the exchange; wherein the sub-data to be processed is a part of the data to be processed, and the sub-data to be processed has the same depth as the data to be processed.

[0010] In one possible implementation, the processor is further configured to: divide the data to be processed into sub-data to be processed in at least one non-depth dimension; and store each of the sub-data to be processed into a different computing core.

[0011] In one possible implementation, generating and activating the convolution operation result includes: generating the convolution operation result based on the weight data and the sub-data to be processed stored in each computing kernel; and activating the convolution operation result through a preset activation function.

[0012] In one possible implementation, dividing the data to be processed into sub-data to be processed in at least one non-depth dimension includes: dividing the data to be processed into sub-data to be processed in the last storage order dimension and non-depth dimension of the data to be processed; wherein the storage order dimension is used to determine the storage order of the data to be processed in different dimensions.

[0013] In one possible implementation, the exchange of unprocessed sub-data between different computing cores includes: determining the dynamic storage space number occupied by each piece of unprocessed sub-data to be exchanged based on the allocation order number and the number of exchanges for each computing core; wherein the allocation order number is used to indicate the allocation order of the unprocessed sub-data; and allocating dynamic storage space for each piece of unprocessed sub-data to be exchanged according to the dynamic storage space number.

[0014] In one possible implementation, the plurality of computing cores perform at least one of the following steps in a parallel synchronous operation: exchanging sub-data to be processed with different computing cores, releasing the storage space occupied by the sub-data to be processed before the exchange, and generating and activating the convolution operation result.

[0015] According to a third aspect of this disclosure, an artificial intelligence chip is provided, the artificial intelligence chip including the processor described in any one of the foregoing claims.

[0016] According to a fourth aspect of this disclosure, an electronic device is provided, the electronic device including the artificial intelligence chip described above.

[0017] This disclosure provides a data processing method applied to a processor's computing core. The method generates and activates convolution operation results based on the sub-data to be processed stored within the computing core; it exchanges the sub-data to be processed with different computing cores, releasing the storage space occupied by the sub-data before the exchange; and then generates and activates convolution operation results based on the exchanged sub-data. Since the sub-data to be processed is not divided in the depth dimension, the storage space occupied by its convolution operation results is relatively small, thus reducing the storage space occupied by the convolution operation results in a single computing core. This enables the data to be processed to still be processed normally even when hardware resources are scarce.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0019] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0020] Figure 1 This is a reference diagram illustrating the convolution operation process implemented using a step-by-step summation method in related technologies.

[0021] Figure 2 This is a reference diagram illustrating the process of convolution operation by loading an initial membrane potential in related technologies.

[0022] Figure 3 This is a reference schematic diagram of an artificial intelligence chip provided according to an embodiment of the present disclosure.

[0023] Figure 4 This is a flowchart of a data processing method provided according to an embodiment of the present disclosure.

[0024] Figure 5 This is a reference schematic diagram of a data processing method provided according to an embodiment of the present disclosure.

[0025] Figure 6 This is a flowchart of a data processing method provided according to an embodiment of the present disclosure.

[0026] Figure 7 This is a block diagram of an electronic device provided according to an embodiment of the present disclosure.

[0027] Figure 8 This is a block diagram of an electronic device provided according to an embodiment of the present disclosure. Detailed Implementation

[0028] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0029] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0030] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0031] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise expressly specified.

[0032] Please see Figure 1 , Figure 1 This is a reference diagram illustrating the convolution operation process implemented using a step-by-step summation method in related technologies.

[0033] Combination Figure 1 As shown, during the convolution operation, assuming the total depth of the data to be processed is 128, the data can be divided into four sub-data in the depth direction, and each sub-data is fed into kernels 0 to 3 for convolution. After the convolution operation, the partial sum data is obtained (the data to be processed is divided in the depth dimension; in this case, the value after convolution of the sub-data is called the partial sum, denoted as...). Figure 1 The values ​​of Sm (from Sm=0 to Sm=3) represent the convolution results of the sub-data to be processed. Kernel 0 stores partial sums at depths 0 to 31, kernel 1 stores partial sums at depths 32 to 63, kernel 2 stores partial sums at depths 64 to 95, and kernel 3 stores partial sums at depths 96 to 127. Through a step-by-step summation process, the partial sums are added together in computational phases 1 through 3. Then, in kernel 3, a preset activation function activates the added partial sums.

[0034] For example, in the first computation phase (Phase 1), computation core 0 sends the partial sum data stored at depths 0 to 31 to computation core 1. Then, computation core 1 adds the received partial sum data at depths 0 to 31 to the partial sum data stored in computation core 1 at depths 32 to 63, obtaining the sum result, which is the partial sum data at depths 0 to 63.

[0035] In the second computation phase, Phase 2, computation core 1 sends the stored partial sum data from depth 0 to 63 to computation core 2. Then, computation core 2 adds the received partial sum data from depth 0 to 63 to the partial sum data from depth 64 to 95 stored in computation core 2, obtaining the sum result, which is the partial sum data from depth 0 to 95.

[0036] In the third computation phase (Phase 3), computation core 2 sends the stored partial sum of data from depth 0 to 95 to computation core 3. Computation core 3 then adds the received partial sum of data from depth 0 to 95 to the partial sum of data from depth 96 to 127 stored within itself, obtaining the sum result, which is also the partial sum of data from depth 0 to 127, and the result of the convolution operation on the data to be processed. Computation core 3 then activates the convolution result to increase the non-linearity of the data and releases the partial sum of data from depth 0 to 127 before activation, thus completing the convolution operation on the data to be processed.

[0037] Therefore, in the above-described stepwise summation scheme, most computational cores are idle. For example, after computational core 1 receives the partial sum data sent by computational core 0, while computational core 1 performs partial summation operations at depths 0 to 63, computational cores 2 and 3 are in a waiting state. Among them, computational core 3 has the longest waiting time in the entire stepwise summation process, which easily leads to a waste of computational resources. In addition, the bit width occupied by the partial sum data is relatively large, usually four or sixteen times that of the data to be processed and the weight data. The computational cores may have insufficient remaining storage space due to the excessive storage space occupied by the partial sum data, and the routing delay for sending the partial sum data is also relatively long.

[0038] Please see Figure 2 , Figure 2 This is a reference diagram illustrating the process of convolution operation by loading an initial membrane potential in related technologies.

[0039] like Figure 2 As shown, the data Xn to be processed is not split along the depth direction, but the weight data W is split along the depth direction, that is, the weight data W is split along the depth direction of each convolutional kernel. The weight data can include N convolutional kernels. The weight data W is split into four parts along the depth direction: weight data W0, weight data W1, weight data W2, and weight data W3. For example, if the depth of the weight data W is 128, it can be split into four parts along the depth direction: W0 represents weight data with depths of 0-31, W1 represents weight data with depths of 32-63, W2 represents weight data with depths of 64-95, and W3 represents weight data with depths of 96-127.

[0040] During the T0 calculation cycle, the data to be processed Xn (n = 1-4) and the segmented weight data W0-W3 are sent to the calculation cores Core0-Core3 respectively. The calculation core Core0 stores the data to be processed X0 and the weight data W0, the calculation core Core1 stores the data to be processed X1 and the weight data W1, the calculation core Core2 stores the data to be processed X2 and the weight data W2, and the calculation core Core3 stores the data to be processed X3 and the weight data W3.

[0041] Each computation core performs a convolution operation on the received data to be processed and the segmented weight data to obtain the convolution result (partial sum). For example, computation core Core0 performs a convolution operation on the received data to be processed X0 and the weight data W0 to obtain the result Vout_0, i.e., Vout_0 = X0 * W0. Here, the result Vout_0 is the partial sum of the depth of the corresponding segmented weight data W0.

[0042] After the T0 computation cycle is completed, computation cores Core0 to Core3 send their stored weight data W0-W3 to other computation cores in rounds during the next computation cycle. In each subsequent computation cycle, each computation core synchronously sends and receives the corresponding weight data. After the weight data is sent and received, a convolution operation is performed between the weight data and the data to be processed. The result of the previous round (computation cycle) can be added while performing the convolution operation. The result (partial sum) of the previous round is called the initial membrane potential in the current round.

[0043] For example, computational core Core0 can send weight data W0 to computational core Core2 and receive weight data W1 sent by computational core Core1. The received weight data W1 is convolved with the unprocessed data X0 stored in computational core Core0. The convolution result X0*W1 is then added to the result Vout_0 from the previous round (calculation cycle T0) to obtain the result Vout_1, i.e., Vout_1 = X0*W1 + Vout_0. Here, the result Vout_1 is the partial sum of the depths of the corresponding segmented weight data W0 and weight data W1. Computational core Core0 can perform convolution operations while simultaneously adding the result Vout_0 from the previous round; the result Vout_0 from the previous round is referred to as the initial membrane potential in this round of computation.

[0044] However, the method of performing convolution operations based on loading the initial membrane potential still cannot reduce the storage space occupied by the convolution operation result in a single computing core. In the example above, the data finally stored in computing core Core3 is Vout_3=X0*W0+X0*W1+X0*W2+X0*W3), which will still occupy a large amount of storage space in a single computing core before activation.

[0045] In view of this, the present disclosure provides a data processing method applied to a processor's computing core. The processing method generates and activates convolution operation results based on the sub-data to be processed stored in the computing core; exchanges the sub-data to be processed with different computing cores to release the storage space occupied by the sub-data to be processed before the exchange; and then generates and activates convolution operation results based on the exchanged sub-data to be processed. Since the sub-data to be processed has not been divided in the depth dimension, the storage space occupied by its convolution operation results is small (which will be described in detail later with examples). That is, it reduces the storage space occupied by the convolution operation results in a single computing core, thereby enabling the data to be processed to still be processed normally when hardware resources are scarce.

[0046] See Figure 3 As shown, Figure 3 This is a reference schematic diagram of an artificial intelligence chip provided according to an embodiment of the present disclosure. Figure 3As shown, an artificial intelligence chip may include multiple processors.

[0047] In one possible implementation, such as Figure 3 As shown, each processor may include multiple computing cores, and data can be transferred between computing cores within each processor and between computing cores of different processors; wherein, each computing core includes a storage component, which is used to store data transferred with other computing cores.

[0048] In one possible implementation, such as Figure 3 As shown, each computing core may include a processing unit and a storage unit. The processing unit may include dendritic units, axonal units, cell body units, and routing units. The storage unit may include multiple storage units.

[0049] In one possible implementation, multiple processors can be integrated into a single neuromorphic computing chip. This neuromorphic chip, also known as a memory-based neuromorphic circuit, references the brain's processing model, simulating the transmission and processing of information by neurons in the brain to improve processing efficiency and reduce power consumption. Each processor can include multiple computing cores, which can independently process different tasks or process the same task in parallel to improve efficiency. Inter-core information can be transmitted between computing cores through routing units within the cores.

[0050] Within a computing core, processing units and storage units can be configured. The processing unit may include dendritic units, axonal units, cell body units, and routing units. The processing unit can simulate the information processing pattern of neurons in the brain, where dendritic units receive signals, axonal units transmit spike signals, cell body units integrate and transform signals, and routing units transmit information with other computing cores. The processing unit within the computing core can read and write to multiple storage units in the storage unit to interact with data within the core, and can each undertake its own data processing and / or data transmission tasks to obtain data processing results, or communicate with other computing cores. Communication with other computing cores includes communication with other computing cores within the same processor, as well as communication with computing cores within other processors.

[0051] In one possible implementation, the storage component includes multiple storage cells, wherein the storage cells can be Static Random Access Memory (SRAM). For example, it can include SRAM with a read / write width of 16 bytes and a capacity of 12 KB. This disclosure does not limit the capacity or bit width of the storage cells.

[0052] Based on the above artificial intelligence chips, please refer to Figure 4 , Figure 4 This is a flowchart of a data processing method according to an embodiment of the present disclosure. The processing method includes the following steps:

[0053] Step S100: Based on the sub-data to be processed stored within the computational kernel, generate and activate the convolution operation result. The sub-data to be processed is a part of the data to be processed, and the sub-data to be processed has the same depth as the data to be processed. For example, the data to be processed may be image, audio, text data, etc., used to multiply with weight data to generate the convolution operation result of the data to be processed. The weight data may include multiple convolution kernels to extract different types of features from the data to be processed. Researchers can set the relevant setting parameters of the convolution kernels according to the actual situation, and this embodiment of the disclosure is not limited thereto. In one example, the data to be processed may be stored in an external memory or a processor A in an artificial intelligence chip. The external memory or a processor A may be connected to another processor B to transfer the sub-data to be processed to some or all of the computational kernels of processor B.

[0054] In one example, step S100 may include: generating a convolution operation result based on the weight data and the sub-data to be processed stored in each computing core. The convolution operation result is then activated using a preset activation function. For example, the weight data stored in each computing core may be different. For instance, if the weight data includes 64 convolution kernels, and there are 4 computing cores, then the weight data stored in each computing core consists of 16 of the 64 convolution kernels. The data to be processed is also divided into 4 parts and sent to the aforementioned 4 computing cores. In other words, when the weight data is divided, the number of sub-data to be processed is the same as the number of weight data after division. If the weight data is not divided, the number of sub-data to be processed can be selected according to the actual hardware conditions. The activation function mentioned above can refer to activation functions in related technologies, such as the ReLU function, Sigmod function, Maxout function, etc., and this embodiment of the disclosure is not limited thereto.

[0055] Continue reading Figure 4 Step S200 involves exchanging the sub-data to be processed with different computing cores until each computing core has stored every piece of sub-data to be processed. In one possible implementation, step S200 may include: determining the dynamic storage space number occupied by each piece of sub-data to be processed based on the allocation order number and the number of exchanges for each computing core. Dynamic storage space is then allocated to each piece of sub-data to be processed according to the dynamic storage space number. The allocation order number indicates the allocation order of the sub-data to be processed.

[0056] For example, the allocation order number mentioned above is used to indicate the order in which the sub-data to be processed is allocated to different computing cores, that is, the storage order number of the sub-data to be processed in the storage dimension. The number of exchanges mentioned above is used to indicate the number of times the computing core exchanges data with other computing cores.

[0057] For example, the weight data corresponding to each computational core can be stored in static storage space, while the corresponding sub-data to be processed can be stored in dynamic storage space (i.e., the weight data does not change its value during the exchange process, while the sub-data to be processed in the computational core changes with each exchange). For example, the dynamic storage space can be divided into k regions, each region corresponding to a dynamic storage space number. The dynamic storage space numbers corresponding to the sub-data to be processed before the exchange and the sub-data to be processed after the exchange are different, so as to avoid two sub-data to be processed occupying the same dynamic storage space, thereby reducing the probability of data corruption.

[0058] For example, the dynamic storage space number corresponding to the above computing core in each swap process can be calculated according to the following Python pseudocode:

[0059] For(i = 0; i++; i <N):

[0060] For(T=0;T++;T <N):

[0061] Address_Core[i][T]=(i+T)%M

[0062] In the code above, i is the allocation sequence number of the computing core. N is the number of data partitions to be processed, i.e., the total number of computing cores to be swapped. T is the number of swaps, which can also be understood as the Tth time period. M is the number of dynamic storage space partitions. Address_Core[i][T] represents the dynamic storage space number occupied by the i-th computing core in the Tth time period.

[0063] For example: There are four sets of sub-data A, B, C, and D to be processed, which are stored in computing cores 0 to 3 respectively. The dynamic storage space in each computing core is divided into four parts, numbered 0 to 3.

[0064] During time interval T0 (i.e., the 0th exchange), computation core 0 stores the sub-data A to be processed into the dynamic storage space numbered 0. Computation core 1 stores the sub-data B to be processed into the dynamic storage space numbered 1 (i.e., (i+T)%M=(1+0)%4=1). Computation core 2 stores the sub-data C to be processed into the dynamic storage space numbered 2. Computation core 3 stores the sub-data D to be processed into the dynamic storage space numbered 3.

[0065] During time interval T1 (i.e., the first exchange), computation core 0 receives the sub-data B to be processed and stores it in dynamic storage space numbered 1. Computation core 1 receives the sub-data C to be processed and stores it in dynamic storage space numbered 2. Computation core 2 receives the sub-data D to be processed and stores it in dynamic storage space numbered 3. Computation core 3 receives the sub-data A to be processed and stores it in dynamic storage space numbered 0 (i.e., (i+T)%M=(3+1)%4=0).

[0066] The subsequent time periods T2 and T3 will not be described in detail in this embodiment. The above-mentioned allocation method of dynamic storage space number is only described as an example. Under the premise that the dynamic storage space of the sub-data to be processed before and after the exchange is different, the dynamic storage space occupied by the sub-data to be processed after the exchange can be arbitrarily selected.

[0067] Continue reading Figure 4 In step S200, steps S210 and S220 are executed, meaning that the kernel performs steps S210 and S220 each time the sub-data to be processed is swapped. Step S210 releases the storage space occupied by the sub-data to be processed before the swap. Step S220 generates and activates the convolution operation result based on the swapped sub-data to be processed.

[0068] For example, if there are n pieces of data to be processed, these pieces are distributed across n computational cores. Each core exchanges unprocessed pieces of data with other cores. After n-1 exchanges, each core stores all pieces of data. Upon receiving new data, each core releases the unprocessed data to increase its available storage space. Then, based on the exchanged data, it generates and activates the convolution operation result. In one example, different cores can perform any of these steps in parallel synchronous operation.

[0069] For example, if the data to be processed is three-dimensional data, it includes data depth Cin (this parameter will be ignored during the convolution process, which can be understood as the data depth becoming 1, and then being convolved by Cout convolution kernels in sequence, so the data depth becomes Cout), data width W, and data height H. In related technologies, it is divided into 4 parts in the depth direction. Each computing kernel uses Cout convolution kernels to perform convolution operations. Then the size of the sub-data to be processed (i.e., the partial sum) after convolution by each computing kernel is H*W*Cout. Each computing kernel exchanges the sub-data to be processed. Before activation, the data space occupied by the partial sum in each computing kernel is H*W*Cout*4. After activation as the result of the convolution operation, the data size becomes H*W*Cout.

[0070] In this embodiment of the disclosure, if the same data to be processed is divided into 4 parts in the non-depth direction, for example, divided into 4 parts in the height direction, then the size of the data to be processed after each kernel convolution is (H / 4)*W*Cout, and then it can be directly activated. After 4 data exchanges, the data size becomes H*W*Cout.

[0071] To visually demonstrate the differences between the two methods, this disclosure provides examples of their storage space usage over different time periods (also known as the number of swaps):

[0072] During time period T0, the storage space occupied by the convolution operation result in the computing kernel of the related technology is H*W*Cout, while the storage space occupied by the convolution operation result in the computing kernel of this embodiment is (H / 4)*W*Cout, which is one-quarter of that in the related technology.

[0073] During time period T1, the storage space occupied by the convolution operation result in the computing kernel of the related technology is 2*H*W*Cout, while the storage space occupied by the convolution operation result in the computing kernel of this embodiment is (H / 2)*W*Cout, which is one-quarter of that in the related technology.

[0074] During time period T2, the storage space occupied by the convolution operation result in the computing kernel of the related technology is 3*H*W*Cout, while the storage space occupied by the convolution operation result in the computing kernel of this embodiment is (3 / 4*H)*W*Cout, which is one-quarter of that in the related technology.

[0075] During time slot T3, the storage space occupied by the convolution operation result in the computation kernel of the related technology is 4*H*W*Cout. In the computation kernel of this embodiment, the storage space occupied by the convolution operation result is H*W*Cout, which is one-quarter of that in the related technology. Then, the related technology adds 4 H*W*Cout values ​​together and activates the data, changing the data size to H*W*Cout.

[0076] As shown in the example above, in some and inactive time periods of T0, T1, T2 and T3, the storage size of the convolution operation results in the computing kernel of the related technology is 4 times that of the embodiments of this disclosure, which greatly occupies the available storage space of the computing kernel.

[0077] Please refer to Figure 5 As shown, Figure 5 This is a reference schematic diagram of a data processing method provided according to an embodiment of the present disclosure.

[0078] like Figure 5As shown, the data to be processed consists of rows 0-13, 14-27, 28-41, and 42-55, each with a depth of 128. Rows 0-13 are assigned to Core0, rows 14-27 to Core1, rows 28-41 to Core2, and rows 42-55 to Core3. Figure 5 The T0 time period is defined in the computational core. Each core stores weighted data, specifically Core0-Core3, which stores weighted data W0-W3 sequentially.

[0079] During time interval T0, rows 0-13 of data in Core0 are multiplied by W0 to generate and activate the convolution operation result. Rows 14-27 of data in Core1 are multiplied by W1 to generate and activate the convolution operation result. Rows 28-41 of data in Core2 are multiplied by W2 to generate and activate the convolution operation result. Rows 42-55 of data in Core3 are multiplied by W3 to generate and activate the convolution operation result.

[0080] During time interval T1, Core0 releases rows 0-13 of data, multiplies the swapped rows 42-55 with W0, and generates and activates the convolution operation result. Core1 releases rows 14-27 of data, multiplies the swapped rows 0-13 with W1, and generates and activates the convolution operation result. Core2 releases rows 28-41 of data, multiplies the swapped rows 14-27 with W2, and generates and activates the convolution operation result. Core3 releases rows 42-55 of data, multiplies the swapped rows 28-41 with W3, and generates and activates the convolution operation result. Time intervals T2 and T3 are similar to the above time intervals and will not be described in detail here.

[0081] This disclosure provides a data processing method applied to a processor's computing core. The method generates and activates convolution operation results based on the sub-data to be processed stored within the computing core; it exchanges the sub-data to be processed with different computing cores, releasing the storage space occupied by the sub-data before the exchange; and then generates and activates convolution operation results based on the exchanged sub-data. Since the sub-data to be processed is not divided in the depth dimension, the storage space occupied by its convolution operation results is relatively small, thus reducing the storage space occupied by the convolution operation results in a single computing core. This enables the data to be processed to still be processed normally even when hardware resources are scarce.

[0082] Please see Figure 6 , Figure 6 This is a flowchart of a data processing method according to an embodiment of the present disclosure. In one possible implementation, the method further includes the following steps before step S100:

[0083] Step S300: Divide the data to be processed into sub-data to be processed in at least one non-depth dimension. For example, if the data to be processed is three-dimensional data, including depth, width, and height dimensions, then select at least one dimension from the width and height dimensions to divide the data to be processed into sub-data to be processed. In one example, the division dimension can be selected according to the following rule: divide the data to be processed into sub-data to be processed in the last storage order dimension of the data to be processed, which is not a depth dimension. Wherein, the storage order dimension is used to determine the storage order of the data to be processed in different dimensions. For example, if the storage order of the data to be processed is first the depth dimension, then the height dimension, and then the width dimension, then the data to be processed is divided in the width dimension to maintain the continuity of the data to be processed in dimensions, which facilitates the orderly allocation of the sub-data to be processed to the computing cores. This embodiment of the present disclosure reduces the storage pressure of a single computing core in storing the data to be processed by storing the sub-data to be processed in multiple computing cores. In the case of N computing cores (that is, when the weight data is divided into N groups), the data storage volume of each computing core is only 1 / N. In addition, the reuse rate of each sub-data to be processed is also higher, that is, each sub-data to be processed is called every time the core swap data is calculated.

[0084] Step S400: Store each of the sub-data to be processed into a different computing core. This embodiment of the present disclosure does not limit the method of allocating the sub-data to be processed to the computing cores; the computing cores only need to contain the weight data corresponding to the data to be processed. Exemplarily, steps S300 and S400 can be executed by a processor, computing cores, or a storage medium storing the data to be processed, and then the divided sub-data to be processed is sent to different computing cores.

[0085] The above steps can be followed by executing steps S100 and S200, which will not be described in detail here. However, if the data to be processed has been divided into multiple sub-data, steps S300 and S400 can be omitted.

[0086] In one possible implementation, this disclosure also provides a processor comprising multiple processors and multiple computing cores, the processors being configured to: generate and activate convolution operation results based on sub-data to be processed stored within the computing cores; exchange sub-data to be processed with different computing cores, and perform the following steps until each computing core has stored each piece of sub-data to be processed: release the storage space occupied by the sub-data to be processed before the exchange; generate and activate convolution operation results based on the sub-data to be processed after the exchange; wherein the sub-data to be processed is a part of the data to be processed, and the sub-data to be processed has the same depth as the data to be processed.

[0087] In one possible implementation, the processor is further configured to: divide the data to be processed into sub-data to be processed in at least one non-depth dimension; and store each of the sub-data to be processed into a different computing core.

[0088] In one possible implementation, generating and activating the convolution operation result includes: generating the convolution operation result based on the weight data and the sub-data to be processed stored in each computing kernel; and activating the convolution operation result through a preset activation function.

[0089] In one possible implementation, dividing the data to be processed into sub-data to be processed in at least one non-depth dimension includes: dividing the data to be processed into sub-data to be processed in the last storage order dimension and non-depth dimension of the data to be processed; wherein the storage order dimension is used to determine the storage order of the data to be processed in different dimensions.

[0090] In one possible implementation, the exchange of unprocessed sub-data between different computing cores includes: determining the dynamic storage space number occupied by each piece of unprocessed sub-data to be exchanged based on the allocation order number and the number of exchanges for each computing core; wherein the allocation order number is used to indicate the allocation order of the unprocessed sub-data; and allocating dynamic storage space for each piece of unprocessed sub-data to be exchanged according to the dynamic storage space number.

[0091] In one possible implementation, the plurality of computing cores perform at least one of the following steps in a parallel synchronous operation: exchanging sub-data to be processed with different computing cores, releasing the storage space occupied by the sub-data to be processed before the exchange, and generating and activating the convolution operation result.

[0092] In some embodiments, the processor provided in this disclosure may have functions or include modules that can be used to execute the methods described in the above method embodiments. The specific implementation of these methods can be referred to the description in the above method embodiments, and for the sake of brevity, they will not be repeated here.

[0093] In one possible implementation, embodiments of this disclosure also propose an artificial intelligence chip, said chip including the processor described above. Figure 3 As shown, the chip may include one or more processors, and the processor may include multiple computing cores. This disclosure does not limit the number of computing cores in the chip.

[0094] In one possible implementation, embodiments of this disclosure propose an electronic device that includes an artificial intelligence chip as described above.

[0095] Please see Figure 7 As shown, Figure 7This is a block diagram of an electronic device 1200 according to an embodiment of the present disclosure. Figure 7 As shown, the electronic device 1200 includes a computing processing unit 1202 (e.g., the aforementioned artificial intelligence processor including multiple computing cores), an interface device 1204, other processing units 1206, and a storage unit 1208. Depending on the application scenario, the computing processing unit may include one or more computing units 1210 (e.g., computing cores).

[0096] In one possible implementation, the computing processing apparatus of this disclosure can be configured to perform user-specified operations. In exemplary applications, the computing processing apparatus can be implemented as a single-chip artificial intelligence processor or a multi-chip artificial intelligence processor. Similarly, one or more computing devices included within the computing processing apparatus can be implemented as an artificial intelligence chip or a portion of the hardware structure of an artificial intelligence chip. When multiple computing devices are implemented as artificial intelligence chips or portions of the hardware structure of artificial intelligence chips, the computing processing apparatus of this disclosure can be considered to have a single-chip structure or a homogeneous multi-chip structure.

[0097] In exemplary operation, the computing processing device of this disclosure can interact with other processing devices through interface devices to jointly complete user-specified operations. Depending on the implementation, other processing devices of this disclosure may include one or more types of processors such as a central processing unit (CPU), a graphics processing unit (GPU), and an artificial intelligence processor, both general-purpose and / or special-purpose processors. These processors may include, but are not limited to, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing processing device of this disclosure can be considered to have a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, they can be considered to form a heterogeneous multi-core structure.

[0098] In one or more embodiments, the other processing device may serve as an interface between the computing processing device of this disclosure (which may be specifically embodied in a computing device related to artificial intelligence, such as neural network computing) and external data and control, performing basic controls including but not limited to data transfer, starting and / or stopping the computing device. In another embodiment, the other processing device may also cooperate with the computing processing device to jointly complete the computing task.

[0099] In one or more embodiments, the interface device can be used to transfer data and control commands between a computing processing device and other processing devices. For example, the computing processing device can obtain input data from other processing devices via the interface device and write it to on-chip storage (or memory) of the computing processing device. Further, the computing processing device can obtain control commands from other processing devices via the interface device and write them to on-chip control cache of the computing processing device. Alternatively or optionally, the interface device can also read data from the storage device of the computing processing device and transmit it to other processing devices.

[0100] Additionally or optionally, the electronic device disclosed herein may further include a storage device. As shown in the figures, the storage device is connected to both the computing processing device and the other processing device. In one or more embodiments, the storage device may be used to store data from the computing processing device and / or the other processing device. For example, the data may be data that cannot be fully stored in the internal or on-chip storage of the computing processing device or other processing device.

[0101] Depending on the application scenario, the artificial intelligence chip disclosed herein can be used in servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablets, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, camcorders, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound scanners, and / or electrocardiographs.

[0102] Please see Figure 8 , Figure 8 This is a block diagram of an electronic device provided according to an embodiment of the present disclosure.

[0103] For example, electronic device 1900 can be provided as a terminal device or a server. (See reference...) Figure 8The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0104] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Windows Server™, Apple's graphical user interface-based operating system (Mac OSX™), a multi-user, multi-process computer operating system (Unix™), a free and open-source Unix-like operating system (Linux™), an open-source Unix-like operating system (FreeBSD™), or similar.

[0105] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0106] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0107] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0108] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0109] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0110] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0111] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0112] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0113] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0114] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0116] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A data processing method applied to a compute core of a processor, the processor comprising a plurality of compute cores, the method comprising: The processing method comprises: ​ dividing the to-be-processed data into to-be-processed sub-data in at least one non-depth dimension; storing each of the to-be-processed sub-data into different computing cores; generating and activating a convolution operation result based on the to-be-processed sub-data stored in the computing cores; exchanging the to-be-processed sub-data between different computing cores, and performing the following steps until each computing core stores each of the to-be-processed sub-data: releasing the storage space occupied by the to-be-processed sub-data before the exchange; generating and activating a convolution operation result based on the to-be-processed sub-data after the exchange; wherein the to-be-processed sub-data is a part of the to-be-processed data, and the to-be-processed sub-data has the same depth as the to-be-processed data; the generating and activating a convolution operation result comprises: generating a convolution operation result based on the weight data and the to-be-processed sub-data stored in each computing core; and activating the convolution operation result by using a preset activation function.

2. The treatment method of claim 1, wherein, the dividing the to-be-processed data into to-be-processed sub-data in at least one non-depth dimension comprises: dividing the to-be-processed data into to-be-processed sub-data in the last storage order dimension and the non-depth dimension of the to-be-processed data; wherein the storage order dimension is used to determine the storage order of the to-be-processed data in different dimensions.

3. The treatment method of claim 1, wherein the exchanging the to-be-processed sub-data between different computing cores comprises: determining the dynamic storage space number occupied by each to-be-exchanged to-be-processed sub-data based on the allocation sequence number and the exchange number of each computing core; wherein the allocation sequence number is used to represent the allocation sequence of the to-be-processed sub-data; allocating a dynamic storage space for each to-be-exchanged to-be-processed sub-data according to the dynamic storage space number.

4. The treatment method of claim 1, wherein the multiple computing cores perform at least one of the following steps by using a parallel synchronous operation mode: the exchanging the to-be-processed sub-data between different computing cores, the releasing the storage space occupied by the to-be-processed sub-data before the exchange, and the generating and activating a convolution operation result.

5. A processor, the processor comprising a plurality of processors, the processor comprising a plurality of compute cores, wherein, the processor is used to perform: dividing the to-be-processed data into to-be-processed sub-data in at least one non-depth dimension; storing each of the to-be-processed sub-data into different computing cores; generating and activating a convolution operation result based on the to-be-processed sub-data stored in the computing cores; exchanging the to-be-processed sub-data between different computing cores, and performing the following steps until each computing core stores each of the to-be-processed sub-data: releasing the storage space occupied by the to-be-processed sub-data before the exchange; generating and activating a convolution operation result based on the to-be-processed sub-data after the exchange; wherein the to-be-processed sub-data is a part of the to-be-processed data, and the to-be-processed sub-data has the same depth as the to-be-processed data; the generating and activating a convolution operation result comprises: generating a convolution operation result based on the weight data and the to-be-processed sub-data stored in each computing core; and activating the convolution operation result by using a preset activation function.

6. The processor of claim 5, wherein, the dividing the to-be-processed data into to-be-processed sub-data in at least one non-depth dimension comprises: dividing the to-be-processed data into to-be-processed sub-data in the last storage order dimension and the non-depth dimension of the to-be-processed data; wherein the storage order dimension is used to determine the storage order of the to-be-processed data in different dimensions.

7. The processor of claim 5, wherein, The exchanging different computing cores between the to-be-processed sub-data, comprising: based on the allocation sequence number of each computing core, the exchange times, determining the dynamic storage space number occupied by each to-be-exchanged to-be-processed sub-data; wherein the allocation sequence number is used to represent the allocation sequence of the to-be-processed sub-data; According to the dynamic storage space number, each to-be-exchanged to-be-processed sub-data is allocated a dynamic storage space.

8. The processor of claim 5, wherein, The plurality of computing cores execute at least one of the steps of exchanging to-be-processed sub-data with different computing cores, releasing the storage space occupied by the to-be-processed sub-data before exchange, and generating and activating the convolution operation result in a parallel and synchronous operation manner.

9. An artificial intelligence chip, characterized by, The artificial intelligence chip comprises the processor according to any one of claims 5 to 8.

10. An electronic device, comprising: The electronic device comprises the artificial intelligence chip according to claim 9.

Images